Preparing AI Agents for Enterprise Security Review

A security reviewer needs to follow a customer request from authentication to the data returned and any action taken. If the engineering team can only show the chat transcript, much of that path is missing.

Build the review materials alongside the agent: the data-flow diagram beside the retrieval code, the permission register beside the tools, and the failure tests beside the release checks. Reconstructing all three during procurement creates avoidable work.

A buyer's requirements and the proposed use determine acceptance. This guide helps a team explain and test its implementation; it cannot guarantee a buyer's approval.

Draw the system that will run

Start with one workflow and its deployed configuration:

User / tenant
  -> application frontend
  -> backend orchestrator
  -> model provider / model gateway
  -> RAG / knowledge layer
  -> tools / MCP servers / APIs
  -> SaaS systems / databases / workflow engines
  -> logging / monitoring / audit store

For each connection, identify the data sent, credential used and authorization check. Mark who can change source content and who operates each component. Include alternate SDK calls, background workers and local tools that bypass the main orchestrator.

Deployment and recovery belong in the diagram. Show the people and services that can change credentials, inspect logs, disable tools or restore queued work. A reviewer should be able to follow a request without guessing which vendor receives the documents or which account performs a write.

Show where permission is enforced

For retrieval, point to the code that resolves the user, tenant and document access. Test it before content enters a model or external reranker. An instruction asking the model not to reveal unauthorized records cannot establish that boundary once the records are in context.

For tools, record the exact operation and permitted targets. Separate drafting from sending and preview from production. Provider access alone may not enforce your ticket assignment, recipient restrictions or document ACLs.

A useful review pairs the implementation with a permitted case and a neighboring denied case:

These are examples to adapt with the buyer. Attach the tested version, expected result, actual observation and owner. A row marked “no open gap” without those details is not an acceptance criterion.

Test hostile input without relying on refusal

Use the inputs the application really processes: customer tickets, retrieved documents, tool responses and saved memory. A hostile instruction in one of those sources may change a proposed action even when the source has no authority to grant permission.

Preserve source labels and use structured inputs where they help. Then test the enforcement code with an unauthorized request. Validate parameters and business rules outside the model, and require a bound approval or an approved automation policy for consequential actions.

Retain the legitimate case too. The buyer needs a working workflow, so a fix that blocks every tool call is not enough.

Explain customer data use with the actual settings

List the model and other processors receiving customer content. For each, document the applicable retention, training-use and contractual settings. The provider's name does not answer whether this account retains prompts or whether a particular feature changes the data path.

Include embeddings, caches, telemetry, error reports and support access. Explain how source permissions propagate to derived copies and what happens after deletion or revocation. Name any delay the design permits rather than promising immediate removal that the system cannot perform.

Keep the inventory linked to the relevant deployment settings and agreements. Review it when a provider, model, logging option or fallback route changes.

Make an operation traceable without copying every secret

An operator needs to connect the initiating user and tenant to source references, a proposed action, policy result, approval reference and downstream outcome. Record model and configuration identifiers so a later test can reproduce the conditions.

Use safe parameter projections or protected content references where full bodies would create another sensitive data store. A digest can verify retained content but cannot reconstruct content that was never retained. Define access, redaction and retention for the trace itself.

Distinguish a proposed tool call from provider acceptance and observed completion. That difference matters when a request times out, a message remains queued or a worker retries after a partial failure. The audit-trail guide follows one such uncertain update.

Exercise the response plan under an operator account

Have the people who will handle failures disable a tool, revoke a grant and stop a queued workflow. Record what remains in flight and how they establish its outcome. Test whether a restart or restore can re-enable work that was stopped.

For a retrieval leak, identify the source records and returned content. For an unauthorized write, preserve the operation and resulting state before deciding how to correct it. For poisoned memory, inspect derived copies as well as the original record. Escalation and notification decisions should follow the organization's established process.

Rate limits, per-tenant rollout controls and time budgets reduce the scale of a failure. Check their behavior under concurrency and retry; a configuration field alone does not show that the limit holds.

Assemble a packet a reviewer can use

Keep the packet small enough to follow: a system overview, current diagram, data and tool registers, control tests, unresolved findings and operating restrictions. Link to detailed logs and source artifacts under appropriate access controls. Identify what was excluded from the review.

NIST's Generative AI Profile can help organize risk-management actions. NIST SP 800-218A addresses AI-related secure development practices. If you map to either, state the practice and work performed; the framework name does not establish implementation or acceptance.

Record the commit, deployed configuration, model identifier, tool schemas, retrieval policy version and test date. Keep unresolved gaps beside passing results. A planned control should not appear as a released one.

Before sending the packet, ask someone who did not build the agent to trace one permitted operation and one denial to their downstream results. Where they lose the thread, improve the implementation or the material explaining it.

Sources

• NIST AI RMF Generative AI Profile
• NIST SP 800-218A
• OWASP 2025 LLM Top 10