Black Hat USA 2025 | Hack to the Future: Owning AI-Powered Tools with Old School Vulns

"Hack To The Future: Owning AI-Powered Tools With Old School Vulns" by Nils Amiet and Nathan Hamiel at Black Hat USA 2025: Core Thesis The integration of generative AI into developer productivity tools (like AI code reviewers and data analytics assistants) is creating massive new attack surfaces. While the underlying Large Language Models (LLMs) are not being "hacked," the applications wrapping them are poorly designed, overly permissive, and riddled with classic, "old-school" vulnerabilities like Remote Code Execution (RCE), Prompt Injection, and Insecure Direct Object Reference (IDOR). Because these AI…

Black Hat USA 2025 | Reinventing Agentic AI Security With Architectural Controls

"When Guardrails Aren't Enough: Reinventing Agentic AI Security With Architectural Controls," delivered by David Brauchler III from NCC Group. The Core Thesis The central argument of the presentation is that guardrails are not security boundaries. Much like Web Application Firewalls (WAFs) in the early days of the internet, AI guardrails are merely statistical heuristics. They reduce risk but do not provide "hard" security guarantees and can always be bypassed by a determined attacker. As AI evolves into "agentic" systems—where models can execute tool calls, read databases, and take actions—relying solely…

Black Hat USA 2025 | Invoking Gemini for Workspace Agents with a Simple Google Calendar Invite

"Invitation is All You Need! TARA for Targeted Promptware Attack against Gemini-Powered Assistants," presented by Ben Nassi, Or Yair, and Stav Cohen. Core Premise The presentation highlights a new, highly practical class of cyberattack called "Promptware," specifically targeting Large Language Model (LLM) powered personal assistants like Google's Gemini for Workspace and Android. The researchers demonstrate how an attacker can completely compromise a user's AI assistant simply by sending them a Google Calendar invitation containing hidden, malicious instructions. The Attack Mechanism: Indirect Prompt Injection Unlike traditional hacking that targets memory corruption or…

Black Hat USA 2025 | Training Specialist Models: Automating Malware Development

"Training Specialist Models: Automating Malware Development" explores how small, specialized Large Language Models (LLMs) can be trained to outperform massive generalist models in specific, highly technical tasks—specifically, the creation of evasive malware. Here is a summary of the key points: The Problem with Current ModelsAvery identifies a gap in the current AI landscape for offensive security professionals: Large Generalists (OpenAI, Anthropic): These models are highly capable but come with privacy concerns, high costs, and strict safety filters (refusals) that make them difficult to automate for red teaming. Small Local Models…

Black Hat USA 2025 | Watching the Watchers: Exploring and Testing Defenses of Anti-Cheat Systems

Introduction to the Anti-Cheat Ecosystem The World of Game Cheats: The speakers explore the fast-paced, high-stakes battleground between cheat developers (attackers) and anti-cheat systems (defenders) in modern competitive shooter games []. The Cheat Economy: Cheating is a massive industry. Cheats are often sold via subscription models by well-run, sometimes legally registered companies, with some cheats costing upwards of $200 a month []. Because it is so lucrative, the attack-defense cycle is incredibly rapid. The Shift to the Kernel: Historically, cheats operated in user mode. As anti-cheats adapted, the…

The difference of overflow and underflow

In computer science—and specifically in fuzzing and exploitation—the terms Overflow and Underflow mean different things depending on whether you are talking about Numbers (Arithmetic) or Memory (Buffers). Here is the breakdown of the differences. 1. Arithmetic (Integer) Context This refers to the value of a number going beyond what the variable type can hold. Integer Overflow (Too Big) Occurs when you try to store a value larger than the maximum limit. The value "wraps around" to the minimum. Analogy: A car odometer at 999,999 rolling over to 000,000.…

Keynote: Perspectives on Trust in Hardware Supply Chains

This talk, titled "Keynote: Perspectives on Trust in Hardware Supply Chains" [] by Bunny Huang, discusses the complexities and vulnerabilities within hardware supply chains. Key points from the talk include: Diversification and Simplification: In chaotic times, it's beneficial to diversify by having multiple, hyper-efficient locations rather than centralized single points of failure. Simplifying business processes and legal contracts can reduce complexity and improve understanding for everyone involved []. Trust in Hardware: The speaker suggests moving the "root of trust" or "source of truth" into hardware, such as using…

Tinker Tailor LLM Spy: Investigate & Respond to Attacks on GenAI Chatbots

In the "Tinker Tailor LLM Spy: Investigate & Respond to Attacks on GenAI Chatbots" talk by Black Hat, Ellen Scott discusses the increasing ubiquity of Generative AI chatbots and the security incidents that can arise from their misuse. The talk outlines three main incident scenarios and provides a playbook for investigation and response []. Here's a summary of the key takeaways: Chatbot Risk Classification []: Low Risk: Chatbots providing general information (e.g., a weather chatbot). Incidents primarily involve brand damage, like a chatbot giving Taylor Swift-themed weather reports…

Running the “Reflections on Trusting Trust” Compiler

Supply chain security is a hot topic today, but it is a very old problem. In October 1983, 40 years ago this week, Ken Thompson chose supply chain security as the topic for his Turing award lecture, although the specific term wasn’t used back then. (The field of computer science was still young and small enough that the ACM conference where Ken spoke was the “Annual Conference on Computers.”) Ken’s lecture was later published in Communications of the ACM under the title “.” It is a classic paper, and a…

Microarchitecture Vulnerabilities: Past, Present, and Future

Past Present Logic Issues Exploitation Techniques Physical Domain in Software Mitigation Efforts Physical hardware cannot be changed in the field Vendors build in "Survivability features" Microcode is the most common used tool for mitigations Other firmware is also used "Chicken bits" to disable / change behavior Some issues are best mitigated in software Mitigations are not always possible/reasonable and almost difficult and time-consuming to engineer Prevention Pre-silicon Post-silicon Future Take Aways