Build AI Agents for Enterprise Security Review

A security reviewer needs to follow a customer request from authentication to the data returned and any action taken. If the engineering team can only show the chat transcript, much of that path is missing.

I would build the review packet while building the agent: the data-flow diagram beside the retrieval code, the permission matrix beside the tool registry, and the failure tests beside the release checks. Reconstructing all three during procurement is avoidable work.

This guide covers preparation. Passing a buyer's review depends on that buyer's requirements, the proposed use, and the tested implementation; no checklist guarantees acceptance.

Enterprise AI Security Review Is About Trust Boundaries

An enterprise security reviewer needs to assess:

Data exposure.
Tenant isolation.
Identity and authorization.
Prompt injection.
RAG leakage.
Tool abuse.
Excessive autonomy.
Logging and auditability.
Response planning.
Vendor governance.
Secure development practices.
Legal, privacy, and procurement risk.

NIST's Generative AI Profile is intended to help organizations identify unique generative AI risks and align risk-management actions with their goals and priorities.

Treat deployment and operations as part of the application boundary: include the people and services that can change credentials, inspect logs, disable tools or restore queued work.

1. Build An Architecture Diagram Reviewers Can Understand

An unclear architecture makes even a narrow review difficult. Show the actual runtime and configuration, including paths that bypass the main orchestrator.

You need a diagram showing:

User / tenant
  -> application frontend
  -> backend orchestrator
  -> model provider / model gateway
  -> RAG / knowledge layer
  -> tools / MCP servers / APIs
  -> SaaS systems / databases / workflow engines
  -> logging / monitoring / audit store

For each component, define:

Field	Example
Owner	Product engineering, platform, security, vendor.
Data handled	Prompt, retrieved document, customer record, token.
Trust level	Trusted internal, customer-controlled, public, untrusted.
Authentication	User OAuth, service account, API key, workload identity.
Authorization	Tenant ACL, RBAC, ABAC, policy engine.
Logging	Prompt trace, retrieval trace, tool-call trace, output trace.

A reviewer should not have to infer where the model sits, where data goes, or what tools are reachable.

2. Treat Prompts As Untrusted Input

Explain prompt injection through the inputs and permissions in your own workflow.

Your review packet should explain how you handle:

Prompt Source	Risk
User input	Direct prompt injection.
Retrieved documents	Indirect prompt injection.
Tool output	Tool-response injection.
Webpages	Hidden or adversarial content.
Emails/tickets	Attacker-controlled instructions.
Memory	Persistent manipulation.

Show how the system classifies user-provided and retrieved content as untrusted, preserves instruction hierarchy, enforces authorization server-side, validates tool parameters outside the model, and regression-tests known prompt-injection scenarios. A prompt instruction alone cannot demonstrate those controls.

3. Enforce Authorization Before Context Enters The Model

Enterprise buyers care deeply about whether one customer can see another customer's data.

For RAG and document workflows, you need to show that authorization happens before the model sees content.

Bad answer:

The model is instructed not to reveal unauthorized data.

Good answer:

Retrieval is constrained server-side by tenant, user, role, source, classification, and document-level ACLs before any content enters the model context.

Show:

Artifact	Why It Helps
RAG data-flow diagram	Shows where authorization happens.
Retrieval policy summary	Explains tenant/role/document filtering.
Negative test cases	Shows cross-tenant queries fail.
Source attribution	Shows answers map to authorized sources.
Logging fields	Shows retrieval can be reconstructed.

4. Classify Tools By Business Impact

If your agent can use tools, enterprise reviewers will ask what those tools can do.

Tool inventory is mandatory.

Tool	Action Type	Data Touched	Risk	Control
`search_docs`	Read	Internal docs	Medium	Tenant/role filtering.
`create_ticket`	Write	Support workflow	Medium	User attribution.
`send_email`	External send	Email content	High	Draft-first approval.
`update_customer`	Write	Customer record	High	User-scoped auth + confirmation.
`run_query`	Query	Database	High	Read-only, parameterized, logged.
`deploy_change`	Deploy	Production	Critical	Disabled or manual approval.

Document how read tools are separated from write tools, destructive tools are disabled by default, high-impact actions require approval, broad service credentials are avoided, parameters are validated before execution, and tool-call traces are preserved.

5. Do Not Let The Model Authorize Itself

The model can request an action. The backend should authorize it.

Decision	Should The Model Decide?	Better Control
Is the user allowed to see this document?	No	Server-side ACL check.
Can this tool be called?	No	Policy engine / allowlist.
Is this action destructive?	No	Action classification.
Should this email be sent externally?	No	Draft-first + human approval.
Can this query run?	No	Parameterized query + DB permissions.
Can this file be deleted?	No	Explicit approval or deny.

Show the enforcing code and a denied request for each boundary. A policy description is much easier to assess when it points to an implementation and a current test.

6. Design For Auditability

Enterprise security review is partly about what happens when something goes wrong.

You need to show that the system records enough artifacts to support response, customer review, abuse review, legal/compliance review, internal debugging, and security regression testing.

Minimum agent trace fields:

Field	Purpose
User ID	Who initiated the workflow.
Tenant ID	Which boundary applied.
Session ID	Correlates interaction.
Prompt reference, digest and retention policy	A digest verifies retained content; an authorized content store is needed for reconstruction.
Retrieved source IDs	Shows what entered context.
Tool name	Shows which action was requested.
Tool parameters	Shows what the model proposed.
Authorization result	Shows policy decision.
Approval event	Shows human confirmation.
External action result	Shows downstream impact.
Model/provider metadata	Supports debugging and reproducibility.
Trace ID	Connects all events.

7. Prepare and Test an AI Response Plan

Enterprise reviewers may ask what happens if the agent leaks data, sends the wrong message, modifies a record, or is manipulated into unsafe behavior.

Event Type	Response Requirement
Prompt injection success	Preserve trace, disable affected workflow, patch controls, regression test.
RAG leakage	Identify source documents, affected users, exposed output, access-control failure.
Tool misuse	Revoke token, disable tool, review approval/policy logic.
Memory poisoning	Quarantine memory, inspect scope, delete poisoned state.
Cost/loop event	Rate-limit, kill workflow, inspect retry/autonomy logic.
External communication error	Identify recipients, preserve content, initiate customer/legal process.

Exercise the response plan with the people who will disable workflows, revoke access and handle affected records.

8. Prepare A Security-Review Packet Before It Is Requested

A strong enterprise AI security packet should include:

Artifact	Purpose
AI system overview	Explains what the agent does and does not do.
Architecture diagram	Shows components, trust boundaries, and data flows.
Data inventory	Lists data classes handled by the system.
RAG authorization summary	Shows retrieval controls and tenant isolation.
Tool inventory matrix	Shows agent capabilities and action controls.
Prompt-injection control summary	Explains boundaries and tests.
Logging/audit summary	Shows artifact trail and retention.
Response summary	Explains containment and escalation.
Secure SDLC summary	Shows how AI-related changes are reviewed.
Known limitations	Identifies unsupported uses and unresolved control gaps.

NIST SP 800-218A extends secure software development practices for generative AI and dual-use foundation models across the lifecycle and is intended for AI model producers, AI system producers, and acquirers.

9. Have A Clear Answer For Customer Data Use

Your answer should clearly cover:

Topic	What To Explain
Model provider	Which provider or model class is used.
Data retention	Whether prompts/outputs are retained.
Training use	Whether customer data trains models.
Fine-tuning	Whether any customer-specific tuning occurs.
Logs	What is logged and for how long.
Redaction	How sensitive data is filtered or protected.
Subprocessors	Which vendors touch customer data.
Contract controls	DPA, security terms, enterprise settings.

Document the provider settings and contractual terms that apply to customer data. The provider name alone does not answer those questions.

10. Enforce and Test Operational Stops

A trustworthy agent system has hard stops.

Hard Stop	Why It Matters
Tool disable switch	Rapid containment.
Per-tenant feature flags	Limit rollout and exposure.
High-risk action denial	Prevent destructive autonomy.
Rate limits	Prevent cost and loop failures.
Retrieval deny rules	Prevent sensitive source exposure.
Human approval gates	Prevent external or irreversible action.
Memory reset/quarantine	Contain persistence failures.
Provider failover policy	Prevent uncontrolled degradation.

Test these stops under the normal operator role. Record what remains in flight, which queues need draining, and who takes over work after disablement.

Enterprise AI Agent Readiness Checklist

These are proposed criteria to adapt with the buyer, not completed checks for your product. Attach a dated artifact and owner to each row.

Area to verify	Proposed acceptance criterion
Architecture diagram exists and is current	No open gap.
Data flows are documented	No open gap.
Tenant isolation is enforced server-side	No open gap.
RAG retrieval is authorization-aware	No open gap.
Prompt injection has negative tests	No open gap.
Tool inventory is complete	No open gap.
Destructive actions are gated	No open gap.
Credentials are scoped and attributable	No open gap.
Logs preserve prompt/retrieval/tool/approval events	No open gap.
AI response plan exists	No open gap.
Customer data training/retention answer is clear	No open gap.
Known limitations are documented	No open gap.
Go/No-Go launch decision is documented	No open gap.

Keep the packet tied to the release

Record the commit, deployed configuration, model identifier, tool schemas, retrieval policy version, and test date. List unresolved gaps in the same packet as passing results. Reviewers should not have to discover that a claimed control exists only in a planned release.

The next useful step is a walkthrough with someone who did not build the agent. Have them trace one permitted action and one denied action from the user to the downstream result. Where they lose the thread, improve the implementation or its supporting artifacts.

Preparing AI Agents for Enterprise Security Review