{"id":330,"date":"2026-03-13T06:47:59","date_gmt":"2026-03-13T06:47:59","guid":{"rendered":"https:\/\/haco.club\/?p=330"},"modified":"2026-03-13T06:47:59","modified_gmt":"2026-03-13T06:47:59","slug":"black-hat-usa-2025-reinventing-agentic-ai-security-with-architectural-controls","status":"publish","type":"post","link":"https:\/\/haco.club\/?p=330","title":{"rendered":"Black Hat USA 2025 | Reinventing Agentic AI Security With Architectural Controls"},"content":{"rendered":"\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Black Hat USA 2025 | Reinventing Agentic AI Security With Architectural Controls\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/iLX4OdAEznY?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p><strong>&#8220;When Guardrails Aren&#8217;t Enough: Reinventing Agentic AI Security With Architectural Controls,&#8221;<\/strong>&nbsp;delivered by David Brauchler III from NCC Group.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>The Core Thesis<\/strong><\/h3>\n\n\n\n<p>The central argument of the presentation is that&nbsp;<strong>guardrails are not security boundaries<\/strong>. Much like Web Application Firewalls (WAFs) in the early days of the internet, AI guardrails are merely statistical heuristics. They reduce risk but do not provide &#8220;hard&#8221; security guarantees and can always be bypassed by a determined attacker. As AI evolves into &#8220;agentic&#8221; systems\u2014where models can execute tool calls, read databases, and take actions\u2014relying solely on guardrails is a recipe for disaster.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>The Root Cause of AI Vulnerabilities<\/strong><\/h3>\n\n\n\n<p>In traditional software, trust is clearly defined (e.g., admins have high trust, standard users have low trust). However, Large Language Models (LLMs) inherently lack these boundaries.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Trust Inheritance:<\/strong>\u00a0An LLM is only as trustworthy as the least trusted input it receives. If an LLM ingests data from a third-party website, an untrusted user prompt, or a poisoned database via Retrieval-Augmented Generation (RAG), the model itself becomes untrusted.<\/li>\n\n\n\n<li><strong>Pollution Flows Downstream:<\/strong>\u00a0Attackers can inject malicious instructions (prompt injection) into data that an LLM will eventually read. The LLM then executes the attacker&#8217;s payload, leading to severe vulnerabilities.<\/li>\n<\/ul>\n\n\n\n<p><strong>Real-World Exploits Demonstrated:<\/strong><br>To prove this, Brauchler showcased two severe attacks:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Remote Code Execution (RCE):<\/strong>\u00a0By feeding malicious context to an AI developer assistant, attackers gained access to a Kubernetes cluster, Azure storage secrets, and internal source code.<\/li>\n\n\n\n<li><strong>Data Exfiltration via RAG (Cross-User Prompt Injection):<\/strong>\u00a0An attacker poisoned their own user profile. When an admin asked their AI assistant to summarize the attacker&#8217;s profile, the poisoned data hijacked the admin&#8217;s LLM, instructing it to quietly read the admin&#8217;s sensitive files (containing passwords) and exfiltrate them to an external server.<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Architectural Solutions (Mitigation Strategies)<\/strong><\/h3>\n\n\n\n<p>Because guardrails fail, organizations must build &#8220;zero-trust&#8221; architectural controls directly into the application layer surrounding the LLM. Brauchler outlined four key strategies:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Trust Binding (Pinning):<\/strong><br>Do not allow the LLM to independently dictate its own permissions. Instead, the backend application should &#8220;pin&#8221; the user&#8217;s authentication token (e.g., a JWT) to any tool call the LLM makes. This ensures the LLM can never perform an action that the human prompting it doesn&#8217;t already have permission to do.<\/li>\n\n\n\n<li><strong>I\/O Synchronization (Preventing Operator Evasion):<\/strong><br>A hijacked LLM can lie to a &#8220;human-in-the-loop.&#8221; It might ask the user to approve &#8220;buying a raincoat&#8221; but secretly send a backend tool call to &#8220;purchase 100 attacker books.&#8221; Applications must enforce strict synchronization, ensuring the backend only executes the exact parameters the user visually approved.<\/li>\n\n\n\n<li><strong>Trust Splitting:<\/strong><br>Instead of using one LLM for everything, split tasks between multiple models. Route highly sensitive actions (e.g., deleting accounts, transferring funds) to a &#8220;High-Trust LLM&#8221; that is\u00a0<em>never<\/em>\u00a0exposed to untrusted data. Route untrusted data processing (e.g., summarizing public web pages) to a &#8220;Low-Trust LLM&#8221; that has no access to critical backend tools.<\/li>\n\n\n\n<li><strong>Trust Isolation (Data Masking):<\/strong><br>If a high-trust LLM must process a prompt that contains untrusted data, the application layer should mask that data\u00a0<em>before<\/em>\u00a0it hits the LLM&#8217;s context window. Replace the untrusted data with a static placeholder, allowing the LLM to plan the action safely without being exposed to a potential prompt injection payload.<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Rethinking AI Threat Modeling<\/strong><\/h3>\n\n\n\n<p>To secure AI applications, security teams need to update their threat modeling approaches:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Trust Flow Tracking:<\/strong>\u00a0Map exactly where untrusted data enters the system and track its flow to see if it ever touches an LLM that has access to sensitive data sinks or high-privilege tools.<\/li>\n\n\n\n<li><strong>Models as Threat Actors (MATA):<\/strong>\u00a0When building data flow diagrams, security teams should literally replace the LLM icon with a &#8220;Threat Actor&#8221; icon. If that threat actor could compromise the system using the tools connected to it, the architecture is flawed and needs stricter controls.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Conclusion<\/strong><\/h3>\n\n\n\n<p>As AI systems are granted more agency to take actions on behalf of users, the attack surface expands exponentially. Security professionals must stop relying on prompt-engineering and AI guardrails for protection, and instead enforce strict, traditional application security boundaries\u2014segmentation, least privilege, and zero-trust\u2014around the AI models.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p>PS: <strong>Zero-Trust<\/strong>&nbsp;is a cybersecurity framework based on one core principle:&nbsp;<strong>&#8220;Never trust, always verify.&#8221;<\/strong><\/p>\n\n\n\n<p>Unlike traditional security models that assume everything already inside a network is safe (the &#8220;castle-and-moat&#8221; approach), zero-trust assumes that threats are everywhere\u2014both outside&nbsp;<em>and<\/em>&nbsp;inside the network. No user, device, or application is trusted by default.<\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;When Guardrails Aren&#8217;t Enough: Reinventing Agentic AI Security With Architectural [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[35,5],"class_list":["post-330","post","type-post","status-publish","format-standard","hentry","category-black-hat","tag-llm","tag-security"],"_links":{"self":[{"href":"https:\/\/haco.club\/index.php?rest_route=\/wp\/v2\/posts\/330","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/haco.club\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/haco.club\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/haco.club\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/haco.club\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=330"}],"version-history":[{"count":1,"href":"https:\/\/haco.club\/index.php?rest_route=\/wp\/v2\/posts\/330\/revisions"}],"predecessor-version":[{"id":331,"href":"https:\/\/haco.club\/index.php?rest_route=\/wp\/v2\/posts\/330\/revisions\/331"}],"wp:attachment":[{"href":"https:\/\/haco.club\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=330"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/haco.club\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=330"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/haco.club\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=330"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}