Building a SOC 2 audit trail for AI agent decisions

What SOC 2 actually requires for AI systems

SOC 2 (System and Organization Controls 2) is built on the AICPA Trust Services Criteria. The criteria don't mention AI explicitly — but they do address system operations, change management, logical access, and monitoring controls. As AI becomes a core system component, auditors have become increasingly specific about what evidence they expect.

The relevant criteria for AI outbound systems are:

• CC6.1 — Logical and physical access controls: who can configure AI policies and approve AI outputs?
• CC7.2 — System monitoring: how do you detect and respond to anomalies in AI-generated outputs?
• CC8.1 — Change management: how are policy changes to AI systems reviewed and approved?
• A1.2 — Availability monitoring: is there evidence the AI system operated within defined parameters?

The five evidence gaps auditors find in AI deployments

1. No audit log of AI-generated outputs

The most common gap. Teams have application logs but no structured, queryable record of "what did the AI generate, what was the policy decision, and what happened to the output." Application logs capture HTTP requests; they don't capture semantic decisions.

2. No change management for AI policies

When auditors ask "show me the change log for modifications to your AI content policies over the audit period," teams find they have no versioned policy history — only the current system prompt and whatever's in git blame.

3. No access controls on AI policy configuration

CC6.1 requires evidence that only authorized personnel can modify production controls. If your AI policy is "whoever has access to the repo can change the system prompt," you don't have a logical access control.

4. No demonstrated effectiveness of controls

"We have guardrails" is a design assertion, not an operating effectiveness assertion. SOC 2 Type II tests operating effectiveness over time. Auditors want to see that the control fired, that it fired correctly, and that responses to control firings were appropriate. This requires specific log entries with timestamps that auditors can sample.

5. No evidence of exception handling

When an AI control fires (blocks or queues an output), auditors want to see evidence that exceptions were reviewed and resolved. "It was blocked and that's the end of it" is insufficient — auditors check whether blocked items were reviewed, patterns were analyzed, and the control was adjusted accordingly.

What the DataVibe audit export gives you

The DataVibe audit export generates a JSON bundle that includes, for each AI output decision in the requested time window:

• Unique decision ID (audit_id)
• Timestamp (ISO 8601, millisecond precision)
• AI output hash (SHA-256 of the original payload — proves it hasn't been modified)
• Policy version that evaluated the output
• Rules that fired and their severities
• Decision: ALLOW, QUEUE, or BLOCK
• If QUEUED: reviewer identity, approval/rejection, timestamp of resolution
• SHA-256 chain hash linking each record to the previous one (tamper-evidence)
• Dispatch record: provider, message ID, timestamp (if ALLOW or approved QUEUE)

This bundle is the SOC 2 evidence package for your AI content control. You hand it to the auditor, they verify the hash chain, they sample specific dates, they confirm the control fired and exceptions were handled. Done.

// Request an audit bundle for a specific period
// GET /api/workspaces/:slug/evidence?from=2026-01-01&to=2026-03-31

{
  "bundleId": "evd_3k9xmqp7n2",
  "workspaceId": "ws_acmecorp",
  "rangeStart": "2026-01-01T00:00:00Z",
  "rangeEnd": "2026-03-31T23:59:59Z",
  "rowCount": 4821,
  "manifestHash": "a3f7b2c901d4e5...",
  "signature": "HMAC-SHA256:7e2a91bc...",
  "signedAt": "2026-04-01T09:15:22Z",
  "rows": [
    {
      "id": "audit_abc123",
      "action": "gate.block",
      "resourceType": "outbound_message",
      "actorType": "API_KEY",
      "metadata": {
        "policy_version": "v3.2",
        "rule_fired": "unauthorized_discount",
        "severity": "BLOCK",
        "payload_hash": "d41d8cd98f00b2..."
      },
      "createdAt": "2026-01-15T14:23:11.483Z",
      "prevHash": "f9e8d7c6...",
      "hash": "1a2b3c4d..."
    }
  ]
}