04 AI Model Data and Agent Application Security.en - jiale-li-orion/SecFusionAgent GitHub Wiki

Model, Data, and Agent Application Security

English | 中文

This document fixes SecFusionAgent's risk taxonomy, evidence chains, continuous monitoring targets and runtime security requirements for the model/data layer and the AI application/Agent layer. It follows the evidence/object/relation contract of 01-Data-Sources-and-Processing, the reuse-first policy of 02-CTI-Baseline-Reuse-and-Ownership, and the control / runtime policy relationship of 03-Normative-Knowledge-and-Policy. Package, artifact, image and dependency provenance in the supply chain remain owned by the supply chain topic.

The research snapshot date is 2026-09-24. Stable knowledge cites surveys, top-tier conference papers, vendor security advisories, GHSA/CVE, OWASP and MITRE ATLAS first; 2026 arXiv preprints serve to discover frontier insight and retain preprint status. Attack, defense and residual risk are recorded separately, and an effective result in a defense paper does not upgrade directly into "the risk is resolved".

Risk objects and evidence

The model and data layer mainly describes the security properties of training data, model artifacts, model behavior, privacy and retrieval/embedding; the AI application and Agent layer describes how a model acts through context, tool, identity, memory, network, sandbox and external systems. The two layers can share the same evidence, but factual ownership is decided by the concrete object.

Prompt injection is a typical cross-layer technique: when input or retrieved content influences model behavior it associates with the model/data layer; when that influence continues to drive a tool call, browser action, file write, network request or permission use it associates with the Agent/Application layer. The provenance and release chain of an MCP server, skill or plugin itself belong to the supply chain; when it causes tool misuse, RCE, credential abuse or state poisoning at runtime, the impact fact belongs to the Agent/Application layer.

The location where a risk occurs and the location where a mitigation is deployed are modeled separately. Data poisoning can be mitigated by dataset provenance, training pipeline validation and model scanning; control of jailbreak/prompt injection can sit in the model, context gate, tool policy, sandbox, network gateway or human approval; model extraction can be risk-reduced by deployment-layer measures such as API policy, rate limit and monitoring. The system stores the two relations RiskTechnique --affects--> Object and Control --mitigates--> RiskTechnique, and records mitigation_layer and residual risk. A security boundary that a model-layer defense cannot carry is carried by an independent system control.

Intelligence objects uniformly store the following fields:

risk_family
technique
affected_object
lifecycle_stage
precondition
attack_mechanism
impact
threat_model

evidence
reproduction / benchmark
mitigation
mitigation_layer
evaluation_condition
adaptive_attack / known_bypass
residual_risk

source_id
product_source_category
source_role / evidence_maturity
venue / vendor
publication_or_disclosure_date
revision

Paper conclusions additionally record claim, supports / contradicts / extends and the corresponding paper revision; vendor incident/advisory records the affected version, patched version, attack chain and remediation.

Model and data layer

Risk classification

The model and data layer is organized into five groups by where the attack occurs and by object. Operational taxonomies such as OWASP and MITRE ATLAS serve engineering mapping, while academic conclusions are supported by surveys, top-tier conference papers and benchmarks.

Risk family Main objects Typical risks Main evidence
Training / Fine-tuning Integrity training sample, dataset, fine-tuning corpus data poisoning, label poisoning, training backdoor, covert control dataset, paper, benchmark, training pipeline report
Model Artifact Integrity checkpoint, adapter, LoRA, model merge model poisoning, backdoored model, malicious artifact model hub, artifact metadata, paper, repo, security report
Inference / Alignment Integrity model behavior, instruction hierarchy jailbreak, adversarial prompting, safety bypass red-team report, paper, benchmark, provider report
Confidentiality / Privacy training data, model parameters, system information training-data extraction, membership inference, model extraction, system prompt leakage paper, incident, benchmark, provider report
Retrieval / Embedding Integrity document corpus, embedding, retriever, RAG context RAG poisoning, retrieval manipulation, embedding attack, indirect prompt injection paper, RAG corpus, retrieval benchmark, Agent/RAG report

Literature evidence chain

The model and data taxonomy uses two ACM Computing Surveys as survey entry points: Security and Privacy Challenges of Large Language Models: A Survey and Unique Security and Privacy Threats of Large Language Models: A Comprehensive Survey. The surveys carry the coverage boundary, and concrete risks continue to hang on representative attacks, defenses and subsequent bypass evidence.

The evidence chain of Training / Fine-tuning Integrity already covers from real-world feasibility to defense bypass. Poisoning Web-Scale Training Datasets is Practical at IEEE S&P 2024 shows the feasible attack surface of web-scale dataset poisoning in a real-world data collection pipeline; EmbedX at USENIX Security 2025 shows an embedding-space backdoor; BAIT at IEEE S&P 2025 studies LLM backdoor scanning. Cordyceps at USENIX Security 2026 further shows covert control formed through poisoning, and bypasses multiple classes of backdoor and prompt-injection defense under its threat model. This chain supports the data model of attack -> mitigation -> adaptive/bypass -> residual risk.

The evidence for Inference / Alignment Integrity and prompt injection needs to retain the adaptive attacker. StruQ at USENIX Security 2025 separates trusted instruction from data via structured query; The Attacker Moves Second at USENIX Security 2026 designs adaptive attacks against 12 recent jailbreak / prompt-injection defenses and obtains an attack success rate above 90% on most of those defenses. SecFusionAgent therefore records a defense's evaluation threat model, adaptive evaluation and known bypass, and does not interpret a low ASR on a static benchmark as a complete security boundary.

Retrieval / Embedding Integrity needs to treat retrieval itself as one segment of the attack chain. PoisonedRAG at USENIX Security 2025 proves that knowledge corpus poisoning can induce a target answer; Overcoming the Retrieval Barrier at USENIX Security 2026 further studies how malicious content is hit by a real retriever under natural queries, and gives an end-to-end attack across RAG and agentic workflow. Related intelligence therefore needs to store retrieval_success, execution_success and downstream impact at the same time.

The academic maturity of operational risks such as system prompt leakage and privacy leakage is not fully consistent. SecFusionAgent preserves this evidence-strength difference: an OWASP label can serve as the engineering entry point of a risk object, while a top-tier or validated attack paper decides how far we can speak about feasibility, impact and mitigation.

Model / data frontier insight

2026 papers need to carry the demonstrative role of "what has changed in existing understanding". Samples that can currently enter the first batch of insight directly include:

Evidence New insight Impact on the system schema
Cordyceps, USENIX Security 2026 poisoning can form covert control based on semantic association, and a fixed trigger taxonomy is insufficient add covert control, semantic trigger, defense bypass, persistence
The Attacker Moves Second, USENIX Security 2026 defense robustness requires verification by an adaptive attacker add adaptive evaluation, known bypass, residual risk
Overcoming the Retrieval Barrier, USENIX Security 2026 retrieval probability is a component of the indirect prompt injection attack chain add retrieval-stage precondition and retrieval success
Context Contamination, USENIX Security 2026 analytical input such as security logs can also carry passive prompt injection, and residual vulnerability remains after layered defense external evidence always retains the untrusted-data property
Evaluation of Prompt Injection Defenses, arXiv 2026 preprint experiments support evaluating independent application-code enforcement separately from model self-protection evidence_status=preprint, triggers follow-up tracking of the system-gate direction

The first four can serve as stable or peer-reviewed evidence; arXiv-only work keeps the preprint marker and waits for subsequent formal publication or independent reproduction.

AI application and Agent layer

Risk classification

Agent/Application risk is closer to software system security. The model handles planning and tool selection, and real impact is decided by capability, identity, execution boundary, network, memory and side effect. OWASP Top 10 for Agentic Applications 2026 provides an operational taxonomy; SecFusionAgent converges to six groups by actual trust boundary.

Risk family Broken boundary Typical objects
Goal / Context Hijack separation of authority between user intent and untrusted content web, email, repo, document, tool output
Tool / Action Misuse whether the model can turn an observation into a real side effect tool, API, browser action, shell, MCP
Identity / Privilege Abuse whom the Agent represents, what scope it holds, how long the credential lifecycle is user identity, service account, OAuth token, secret
Execution Boundary Failure whether controlled execution crosses the process/container/VM/host boundary shell, filesystem, runtime, sandbox, host bridge
Memory / State Poisoning whether temporary input can be promoted into trusted state across sessions memory, thread, preference, workspace state, config
Inter-Agent / Workflow Trust whether delegation and message passing expand the privilege of a poisoned agent orchestrator, peer agent, shared memory, workflow state

The Agent/Application graph maintains the following relations:

Agent reads Context
Agent invokes Tool
Agent acts-as Identity
Agent executes-in Sandbox
Agent reaches Network
Agent reads/writes Memory
Agent delegates-to Agent
Agent causes SideEffect

A security investigation focuses on which relation the attacker gains influence on, and which capability converts that influence into real impact.

Representative cases

Public evidence already covers tool, sandbox, network, MCP, memory, multi-agent and computer-use. Evidence maturity differs across subdomains: tool/sandbox/MCP have vendor advisories and CVE; memory and multi-agent still rest mainly on ATLAS, peer-reviewed papers and recent preprints.

Surface Representative evidence Reusable conclusion
Tool / command AWS-2025-019: prompt injection in Amazon Q/Kiro can drive local commands, DNS exfiltration or configuration-related code execution, with HITL added afterwards high-risk commands and side effects need an independent action policy, and a safe tool name does not mean safe parameters
MCP / repo trust GHSA-8q5r-mmjf-575q: .mcp.json in a malicious PR is automatically enabled by Claude Code Action, causing runner RCE and secret exfiltration repo-controlled MCP config is untrusted code/config, and server enablement needs provenance and explicit policy
MCP / authentication Microsoft 2026 telemetry: among its Defender for Cloud signals 15% of remote MCP servers are classified as critically insecure and allow unauthenticated access to sensitive internal capabilities the existence of MCP transport does not equal the existence of authorization; a remote server needs identity, scope and audit
Sandbox seam CVE-2026-39861 / GHSA-vp62-r36r-9xqp: a Claude Code sandboxed process creates a symlink, and an unsandboxed host process subsequently follows it and writes out of the workspace a cross-sandbox helper / host bridge needs fresh trust analysis, and components that are each restricted can still combine into privilege escalation
Tool crosses sandbox Microsoft Semantic Kernel CVE-2026-25592: the model can invoke DownloadFileAsync to write a file inside the sandbox to a host path of its choosing validate a model-controlled tool parameter as attacker-controlled input; the host bridge does not expose free path selection to the model
Persistent config CVE-2026-25725: code inside the sandbox can create a persistent Claude Code config, causing a later host session to execute a hook config/state has persistence, and write permission needs to be separated from ordinary workspace data
Network exfiltration OpenAI URL-based exfiltration: sensitive data can be encoded into a URL request; a trusted-domain allowlist is also affected by redirect filesystem and network are independent enforcement seams; egress policy needs to inspect the complete URL, redirect and destination
Computer Use OpenAI Operator System Card: the 2025 research preview treats a website as a potential adversarial actor and adopts confirmations, watch mode, prompt-injection monitor and browser sandbox GUI action is likewise a privileged side effect; high-risk actions need takeover/confirmation, and the browsing environment is separated from the local environment
Browser agent Anthropic browser-use prompt injection: content such as web pages and email can serve as an indirect injection surface, and the vendor explicitly retains residual risk browser content permanently retains the untrusted observation property, and model robustness cannot replace system enforcement
Memory MITRE ATLAS AML.T0080 divides Agent Context Poisoning into Memory and Thread; Bad Memory, arXiv 2026 shows that a payload already planted in persistent memory can affect a later session memory write, promotion and read scope need provenance, trust, TTL and independent authorization
Multi-agent Conjunctive Prompt Attacks, ACL 2026 shows cross-agent conjunctive attacks under star/chain/DAG routing; Overcoming the Retrieval Barrier, USENIX 2026 gives end-to-end exfiltration in a multi-agent workflow an agent message cannot inherit the sender's instruction authority; cross-agent routing needs provenance, role-scoped capability and end-action authorization
Web-agent adaptive attack MUZZLE, USENIX Security 2026 finds 44 new attacks and cross-application injection in 4 web apps a static prompt set cannot cover a real browser trajectory, and evaluation needs adaptive red-team and cross-application side effect
Action authorization AttriGuard, USENIX Security 2026 pushes the defense point from input classification to tool-call causal attribution a runtime gate can judge whether a proposed action is supported by user intent, instead of only judging whether the input "looks like an attack"

This round found no "multi-Agent vendor incident library" or "memory vendor CVE mainline" as mature as MCP / tool / sandbox. Multi-agent therefore currently adopts ACL/USENIX peer-reviewed research as stable evidence, memory adopts MITRE ATLAS + 2026 preprint as frontier evidence, and both continue into Research Intelligence Watch. The document does not write a direction with lower evidence maturity into an established large-scale production incident fact.

Memory / State

Agent memory is an independent trust boundary. Beyond content, a memory object also needs to store source, writer, scope, lifecycle and promotion reason:

value
source
source_trust
writer
created_at
scope
ttl
promotion_reason
read_policy
write_policy

Web pages, issues, email, MCP output and peer-agent messages enter session context as untrusted evidence by default. Promoting their content into durable memory needs an independent write policy; memory retrieval across users, tasks and agents is filtered by identity and scope. Detection and cleanup of memory poisoning cannot rely on the model "remembering to ignore" a piece of state.

MCP and Tool

MCP standardizes tool discovery, schema and invocation, and at the same time expands the attack surface of tool provenance, authorization and cross-server trust. The OWASP MCP Security Cheat Sheet already lists risks such as tool poisoning, rug pull, tool shadowing, confused deputy, over-scoped token, message replay and sandbox escape; these taxonomies serve monitoring and testing, and no project-specific MCP security taxonomy is created alongside them.

SecFusionAgent needs to establish the following control fields for MCP/tool in Technical Design:

server_identity
server_origin
transport
authentication
tool_schema_hash
schema_revision
approved_capability
credential_scope
side_effect_class
network_scope
sandbox_class
human_approval_policy

A change to a tool definition after first approval counts as a new revision; a change to the schema, server identity or credential scope of a high-impact capability triggers re-approval. Tool output continues to enter context as untrusted data.

Multi-Agent

Multi-agent adds trust propagation that a single agent does not have: a wrong judgement by a low-privilege agent can influence a high-privilege executor through message, shared memory or orchestrator routing. The security boundary is established per agent identity/capability, and "comes from an internal agent" is not used as a basis for trust.

Runtime security requirements include: every inter-agent message carries sender, task, provenance and message revision; delegation explicitly records the parent task and delegated scope; each agent's tool/credential is assigned by role; shared memory has writer and reader ACL; the executor redoes authorization for a high-impact action instead of reusing an upstream agent's natural-language "approved" conclusion.

The ACL 2026 conjunctive attack shows that multiple components that each look locally normal can form a malicious combination after routing, so boundary testing needs to cover star, chain, DAG and orchestrator/worker topologies rather than testing each agent independently.

Browser / Computer Use

Computer Use turns the traditional browser/OS GUI into a general tool surface. Web page text, DOM/accessibility tree, images, email, downloaded files and cross-application clipboard content can all carry data and adversarial instruction at the same time. The public security material of OpenAI Operator and Anthropic browser-use both treats indirect prompt injection as a persistent risk and adopts confirmation, monitor, browser isolation and user takeover mechanisms beyond model mitigation.

Computer-use policy needs to distinguish side-effect classes such as read, navigate, download, upload, type, submit, send, purchase, delete and credential entry. Login, credential entry and irreversible actions use takeover or explicit confirmation; an explicit bridge is established between the browser session and the host filesystem, credential store and local network. A downloaded file retains the untrusted artifact marker after entering the sandbox, and does not gain execution permission because it came from a browser download flow.

Frontier security intelligence monitoring

Model/data and Agent/Application share one frontier security intelligence monitoring chain. It monitors "whether new evidence changes existing risk understanding, attack surface, mitigation, residual risk or system design", and its data sources are not limited to papers. Academic papers provide reproducible methods and peer-reviewed evidence; vendor technical reports, system/model cards, security blogs and incident reports often expose production failure, deployment constraints and real control earlier; independent security research adds cross-vendor reproduction, vulnerability discovery and attack-defense observation. The three classes of source store authority and source type separately, and vendor self-report, preprint and peer-reviewed paper are forbidden from being compressed into one evidence level.

Source portfolio

The portfolio lane below is an information-role grouping inside the models/data and Agent/Application topic; it does not create another product source taxonomy. Every lane maps explicitly to the eight-category taxonomy from 01. Fixed monitoring, on-demand scholarly search, publication resolution and dynamic discovery share the same Source Definition / Evidence boundary.

Portfolio lane Product taxonomy Sources / entry points Main signal Default handling
Academic research academic arXiv, Crossref, OpenAlex, OpenReview, Semantic Scholar, formal proceedings, seasonal community curation attack/defense, benchmark, survey, adaptive evaluation persistent low-cost discovery; on-demand scholarly search narrows the space; formal paper/DOI owns publication resolution; community curation is discovery only
Frontier lab research vendor OpenAI Research/Security/System Cards, Anthropic Research/News/System Cards, Google DeepMind Responsibility & Safety, Meta AI safety/security frontier capability, incident, prompt injection, agent control, model/system evaluation preserve first-party authority; extract technical claims/experiments as evidence; product promotion does not automatically become Insight
China frontier lab vendor ByteDance Seed Publications, Tech Blog, Model Cards Agent/Coding/GUI, model/data, long-horizon evaluation, technical reports watch publication/report/card/major revision and classify by security/capability/architecture relevance
Platform / infra research vendor Microsoft Security Blog, NVIDIA Security for AI / AI Red Team, AWS Security Bulletins Agent framework CVE, MCP, identity, sandbox, network, runtime control incident/advisory becomes threat evidence; reference architecture/design guidance becomes control/design evidence
Independent security research independent Trail of Bits, Wiz, HiddenLayer, Unit 42, Talos MCP/tool poisoning, prompt injection, AI-infra exploit, agent incident, red-team, telemetry independent reproduction, telemetry and cross-product research validate vendor claims and reveal new attack patterns
Operational taxonomy / community knowledge independent OWASP GenAI / Agentic, MITRE ATLAS technique, failure mode, control vocabulary taxonomy mapping only; does not replace attack-feasibility or defense-effectiveness evidence

ByteDance Seed maintains public paper lists, Tech Blog and Model Card pages and continuously publishes Agent, Coding, GUI and foundation-model technical material. Its value in the watch is to cover production technical direction and benchmark change from a Chinese frontier lab rather than waiting for an arXiv keyword hit.

OpenAI, Anthropic, Google DeepMind, Microsoft and NVIDIA follow the same rule: monitor their source streams directly instead of relying on secondary discovery by a general search engine. First-party reports often precede formal papers and remain fixed M1 sources. When the same content also carries incident value, the executable source stream may support the incidents coverage category as well without changing source authority.

Discovery and update

Different sources use different discovery paths:

Source form Discovery Update semantics
arXiv API / metadata query; cs.CR + cs.CL + cs.AI + cs.LG + cs.IR and keyword clusters low-cost periodic discovery; check new versions for priority arxiv_id values
on-demand scholarly search Crossref, OpenAlex, OpenReview / Semantic Scholar when available, and general Web search search on demand from M4/M5 investigation questions; prioritize DOI, venue, year, formal paper pages and candidate revisions without mirroring whole venues locally
conference proceedings formal venue pages such as USENIX Security, IEEE S&P, CCS, NDSS, ACL Anthology and NeurIPS resolve publication state after a candidate is found; acceptance, camera-ready and DOI changes update status
conference curation seasonal GitHub guides, categorized accepted-paper repositories and conference explorers provide high-recall navigation, topical grouping and code/project links across thousands of papers; remain discovery hints rather than authority evidence
vendor publication / blog official index, RSS/Atom (where present), sitemap or list page diff new article, technical report, system/model card, incident update; a content change at the same URL stores a new revision
security research blog official research/category index, RSS/Atom or sitemap new disclosure, telemetry, follow-up, fix update; associate CVE/GHSA/vendor response
model/system card official model release/card index new model, card revision, security evaluation or capability boundary change

Web crawling records canonical_url, published_at, updated_at, author/team, source class, content hash and fetch time. When a page only changes layout and the body hash is unchanged, no insight is regenerated; when body, conclusion, experimental number or remediation is updated, a new observation is formed.

Scholarly search boundary

The academic paper space is much larger than vulnerability feeds and fixed vendor watches. Venues such as SaTML, ICML, ICLR, EMNLP, RAID, TDSC, TIFS, TSE and TOSEM belong to the search space of on-demand scholarly search; SecFusionAgent does not create a dedicated full-volume continuous crawler for every venue. When M4/M5 needs to answer questions such as whether an attack has paper evidence, whether a defense has adaptive evaluation, or whether a recent conference introduced a new attack surface, the search tool constrains retrieval by keyword, venue, year, object and evidence type, then resolves candidate papers against formal sources.

The search chain has three layers:

investigation question
  -> structured scholarly search
     Crossref / OpenAlex / available scholarly APIs / Web search
  -> publication resolution
     official proceedings / DOI / arXiv / author project
  -> seasonal curation hints
     GitHub conference guides / categorized accepted-paper lists

Community curation answers discovery questions such as which papers appeared this year, how they were categorized and which ones expose code or project pages. It can reduce the search space from thousands of papers, but its classification, summaries and code-link completeness are secondary material. Title, authors, acceptance status, revision and formal links resolve back to venue / DOI / paper sources. Paper revisions actually consumed by investigation, QA or evaluation then enter durable_managed and continue through the existing same-work-as, supports / contradicts / extends and evidence_status relations.

Useful seasonal forms include GitHub repositories that classify all accepted papers for one conference and topic-focused repositories that filter ICLR / ICML accepted-paper lists for Agent-related work. When registered, these sources use a discovery / curation role and never own peer-review status or experimental claims.

API accessibility is a scholarly-search provider-selection and fallback concern. A lightweight probe on 2026-09-26 observed structured responses from Crossref, usable OpenAlex responses with one connection timeout, and stable GitHub REST reads of conference curation repositories; the current runtime encountered challenge, throttling, bot-check or egress-compatibility issues with OpenReview, Semantic Scholar, DBLP and arXiv. These observations describe the current environment rather than a long-term source contract. Implementation needs provider-specific timeout, retry, cache, API-key and browser/manual fallback policies.

Query clusters

Continuous academic discovery and on-demand scholarly search share high-recall keyword clusters; vendor and independent research sources are first discovered in-site by source stream, then relevance-classified with the same taxonomy, to avoid missing important system design when a vendor title does not explicitly write security.

Cluster Query terms
Jailbreak / alignment jailbreak, adversarial prompt, alignment attack, safety bypass, guard model
Prompt injection prompt injection, indirect prompt injection, instruction injection, context contamination
Poisoning / backdoor data poisoning, model poisoning, backdoor, fine-tuning poisoning, semantic trigger, covert control
Privacy training data extraction, memorization, membership inference, model inversion, model extraction, model stealing
RAG / retrieval RAG poisoning, retrieval poisoning, retrieval attack, embedding attack, retriever attack
Agent runtime agent security, sandbox, runtime policy, network egress, identity, computer use, browser agent
MCP / tool MCP security, tool poisoning, tool misuse, capability, function calling, skill security
Memory / multi-agent memory poisoning, persistent memory, multi-agent security, delegation, inter-agent, shared memory
Defense jailbreak defense, prompt injection defense, guardrail, LLM firewall, action authorization, causal attribution

Insight promotion

After new content is discovered, an Insight Candidate is generated first. What the product displays is "what understanding has changed", not a paper/blog update feed.

source_item
source_class
canonical_url
revision
risk_domain
risk_family
affected_object
what_is_new
what_previous_assumption_changed
attack / defense / measurement / incident / capability / design
threat_model_or_operational_context
key_result
related_existing_claim
supports / contradicts / extends
new_relation_or_schema_needed
evidence_status
confidence

A candidate is promoted only when it satisfies one of the following changes: a new attack surface is added; a change in model or Agent capability changes the threat model; a theoretical risk is pushed to production/observed; an existing defense is broken; a new control with independent enforcement meaning is proposed; the residual-risk judgement changes; a new benchmark/telemetry is produced; a modification of taxonomy, relation or Technical Design assumption is required. An ordinary product release, an opinion without evidence and an article repeating an existing claim remain searchable but do not enter frontier insight display.

Evidence promotion and identity

preprint, formal paper, vendor report, security advisory and independent reproduction exist as different observations. The v1/v2/v3 of the same arxiv_id belong to the same work; after a DOI / conference publication is found, same-work-as is established. When a CVE/GHSA, external reproduction or paper later appears for a vendor article, supports / contradicts / extends is established without overwriting the old evidence.

A first-party incident report has high authority on "what this vendor observed and what fix it adopted", while general effectiveness still needs external research or cross-product evidence. An independent research finding needs the disclosure timeline and vendor response stored. The security benchmark of a system card / model card retains the concrete model version, evaluation configuration and date, and extrapolation to other model versions is forbidden.

The front end can display Latest AI Security Insights and What Changed Our Understanding. Each card shows source type and evidence status at the same time, so that sources such as "USENIX peer-reviewed result", "arXiv preprint", "OpenAI/Anthropic first-party incident" and "Wiz/Trail of Bits independent research" do not lose their evidence boundary in the UI.

Agent Security Control Plane requirements

The production design of Agent/Application adopts a control plane that separates model reasoning from real capability. The later DESIGN can start probing from the following logical components:

Control seam Responsibility
Context / Trust Gate mark the source and authority of external content, repo, web, email, tool output and peer-agent message
Capability Broker decide whether an invocation is allowed, based on task, tool, parameter and side-effect class
Identity Broker provide a scoped, short-lived credential for a single task/session, and record delegated identity
Execution Sandbox isolate filesystem, process, resource, kernel and host bridge
Network Gateway control destination, DNS, protocol, redirect, request budget and egress logging
Memory / State Store manage provenance, writer, scope, TTL, promotion and read/write policy
Side-effect Gate apply policy, staging or human approval to actions such as send/write/delete/pay/publish
Audit / Telemetry record context, tool call, credential use, network request, sandbox event, state write and final effect

DSec: isolation class reference

DeepSeek Elastic Compute (DSec) is a preprint published in 2026-09 on production sandbox infrastructure, used for large-scale agentic training/evaluation; it provides four backend classes, FnCall, container, microVM and full VM, and brings resource, lifecycle and network rule into a unified sandbox API. The main use of DSec is production execution infrastructure, so within SecFusionAgent it serves as a Technical Design reference rather than as evidence that some class of defense is already security-complete.

The SecFusionAgent interface needs to retain isolation class rather than a single sandbox=true/false:

Isolation class Applicable task Main boundary
semantic-only pure retrieval, parsing, reasoning no executable capability
fixed tool / FnCall stable API, query, validator capability schema and parameter policy
process sandbox controlled shell, small local task namespace / Landlock / seccomp / filesystem policy
container coding, parser, general tool execution filesystem, process, resource isolation
microVM untrusted repo, security analysis, arbitrary code independent kernel and stronger host isolation
full VM browser/desktop/COTS OS, high-risk complete environment complete OS boundary and independent lifecycle

Technical Design 2 now owns the concrete v1 runtime selection: controlled execution defaults to OpenShell + OCI container, while high-risk arbitrary-code work uses a Firecracker microVM profile without silent downgrade; browser/full-VM remains an interface under probe. This document keeps only the security semantic that task risk determines isolation class and does not duplicate deployment parameters.

GitHub Agentic Workflows: control plane reference

GitHub Agentic Workflows Security Architecture separates substrate isolation, declarative configuration, network firewall, API proxy, MCP Gateway and staged execution. The egress of its agent container is controlled through firewall/proxy, and the MCP server is managed by the gateway inside an isolated container; the document also makes explicit that an in-container identifier such as agent ID cannot be treated as a strong authentication boundary.

This public design supports three runtime security requirements: filesystem/process isolation and network egress are enforced separately; MCP/tool capability is carried by a gateway/broker; the proposal/artifact generated by the Agent is separated from the privileged stage that actually writes an external system. Technical Design 2 preserves these enforcement seams, while implementation probes decide concrete runtime/provider reuse.

NVIDIA OpenShell: runtime security boundary reference

The Agent Security Stack / OpenShell design published by NVIDIA in 2026 places model, agent and harness in the behavior layer, and places runtime, identity, policy, credential, network and audit below an independent security boundary. Its public design requires that a higher layer can only propose an action and a lower layer decides whether to execute it; every external effect passes through policy, a credential uses just-in-time scope, and a subagent can only obtain a delegated child runtime and cannot break through the ceiling of its parent.

This design is complementary to DSec and GitHub: DSec provides a multi-level execution substrate; GitHub shows the production control plane of MCP/network/staged side effect; OpenShell strengthens the contract that "policy must sit at an enforcement boundary the Agent cannot bypass by itself". Technical Design 2 now fixes ToolImplementation / CapabilityContract / CapabilityBinding and the logical Execution Control Plane responsibilities, and selects OpenShell as the v1 sandbox supervisor/enforcement runtime; Identity/Network provider reuse, browser/full-VM, and finer backend composition remain implementation probes.

Agent security incident schema

Agent/Application intelligence uses the complete capability path rather than storing only a vulnerability label:

incident / technique
entry_surface
untrusted_context
goal_or_state_effect
agent / role
capability
identity / credential scope
tool / MCP server
execution_boundary
network_effect
memory_or_state_effect
side_effect
persistence
impact
control
enforcement_point
known_bypass
residual_risk
vendor_advisory / CVE / GHSA / paper

For example, the .mcp.json case can be expressed as an attacker-controlled PR providing untrusted config, the Agent runtime automatically enabling the MCP server, the server obtaining runner execution capability, and finally producing RCE and secret exfiltration. The same schema can also express browser injection, memory poisoning, peer-agent propagation and sandbox escape.

M1-M8 Ownership

Module Model/data and Agent/Application responsibility
M1 monitor low-cost academic discovery, frontier lab technical report/system/model card, vendor and independent security research, model/data platform, vendor advisory, GHSA/CVE, Agent framework, MCP and browser/computer-use security material
M2 store the revision and authority of paper, technical report, system/model card, blog research, advisory, incident and repo evidence, and maintain same-work and subsequent publication/disclosure relations
M3 form risk, technique, control, benchmark, agent capability path and cross-layer relation
M4 store confirmed fact, conflict, unknown, residual risk and evidence chain in an investigation
M5 use on-demand scholarly search for paper tracking, repo/advisory investigation and tool/Agent attack-chain completion; candidate papers return to formal sources for revision and publication-status resolution
M6 answer attack feasibility, affected object, mitigation, known bypass and applicable control, citing concrete evidence
M7 maintain model/data benchmark, adaptive attack, Agent prompt-injection, tool, MCP, memory, multi-agent and sandbox regression case
M8 enforce SecFusionAgent's own capability, identity, sandbox, network, memory and audit control

M7's Agent security testing covers single-Agent and multi-Agent topology, persistent memory, MCP/tool, browser/computer-use, network exfiltration, sandbox seam and side-effect authorization. The benchmark score of a model-layer defense and the actual enforcement of a system-layer control are reported separately.

Items fixed at the requirements stage

The model and data layer uses five groups of risk family, and retains a traceable evidence chain of attack, mitigation, adaptive bypass and residual risk. Model/data and Agent/Application share frontier security intelligence monitoring: low-cost academic discovery, on-demand scholarly search, frontier lab technical report and system/model card, platform security research and independent security research all enter Insight Candidate; only content that changes an existing claim, threat model, control, telemetry or schema enters frontier insight display, and the evidence boundaries of preprint, peer-reviewed, first-party and independent research are retained. Seasonal community curation only assists discovery; formal venue / DOI / paper sources own publication state and revision resolution.

The AI application and Agent layer uses six trust boundaries: goal/context, tool/action, identity/privilege, execution boundary, memory/state, and inter-agent/workflow. Classic cases rest mainly on vendor advisory, GHSA/CVE and public incident research, while multi-Agent and memory are currently supplemented by peer-reviewed research, MITRE ATLAS and frontier preprint, with evidence maturity explicitly retained.

Agent/Application runtime security fixes control-plane separation across context provenance, capability, identity, execution sandbox, network, memory/state, side effects and audit/telemetry. Technical Design 2 now fixes Task/Role, ToolImplementation/Capability/Binding, Policy decision points, delegation ceilings, and the Sandbox v1 contract. DSec, GitHub Agentic Workflows and OpenShell remain references for isolation/runtime/gateway enforcement, while the concrete capability catalog, provider-reuse boundary, and implementation parameters are decided by TD2 coding/evaluation probes.

Main reference sources

Model / Data

Agent / Application

Frontier monitoring source indexes

Academic search / conference curation

Technical Design reference