SOC 2 by industry

SOC 2 for AI companies: what happens to the customer's prompt.

Buyers of AI products ask a question older software never faced: does my data teach your model? A SOC 2 report cannot answer every AI-specific worry, but it can show, with testing, where prompts and files travel, who can read them, how long they stay and whether training use matches what you promised. This guide shows how to scope it.

Back to SOC 2 services

01 · Who asks

Who asks AI companies for a SOC 2 report

The buyer decides the scope and the weight of the report, so start with who is asking.

Asked by	What they want to see
Enterprise security and legal reviewers	Whether customer inputs are used for training, which model providers receive them, and how long anything is retained.
Data protection officers	Where personal information in prompts goes, including cross-border transfers to model providers.
Customers in regulated sectors	Logging of who at your company viewed their conversations, and whether human review of outputs happens.
Platform partners embedding your model	Change control over model versions, so a silent swap does not break their product.

02 · What is in scope

What sits inside the system boundary

The report covers one system. In this sector the boundary usually takes in:

Inference services: The API and app that accept prompts and files, call a model and return output.
Prompt, output and file stores: Conversation history, uploaded documents, embeddings and vector indexes built from customer content.
Training and fine-tuning pipelines: Datasets, labelling tools and the jobs that turn data into model weights, if any customer data can enter them.
Model registry and release process: Where model versions are stored, evaluated and promoted to production.
Human review tooling: Interfaces where staff or contractors read prompts for quality, safety or abuse review.

03 · Categories

Which Trust Services Categories matter here

Security is always in. The rest depend on what you promise customers.

Category	Weight in this sector	Why
Security	Core	Always included. Access to stored prompts and to model weights is the core question.
Confidentiality	Core	Customers send proprietary documents and code. C1.1 and C1.2 cover identifying, protecting and disposing of it, which is what training-use promises are about.
Privacy	Often	Relevant when individuals interact with your product directly, or personal information in prompts is under your own notice and choice.
Availability	Often	Customers building on your API need capacity planning, especially when a model provider throttles you.
Processing Integrity	Rare	Model outputs are probabilistic. Committing to 'accurate' output is hard to specify; it fits only for deterministic steps around the model.

04 · Vendors in the report

The vendors that appear in your report

Each one is a subservice organization or a carve-out, and the report has to say which.

Foundation model provider: A subservice organization when it processes customer prompts. The description should name its retention and training-use terms you rely on.
GPU cloud or inference host: Carved out, with CSOCs for physical security and tenant separation on shared accelerators.
Vector database service: Holds embeddings derived from customer content. Treat it as holding customer data.
Labelling or annotation contractors: If people outside your company see customer data, they are inside your vendor management, and often your access controls.

05 · Sector controls

Controls that this sector makes necessary

Examples only. Your own wording is what the auditor tests.

S.1
Training use follows a recorded customer settingEach tenant's training opt-in or opt-out is stored, and dataset builds filter on it automatically.
S.2
Dataset lineage is keptEvery training or fine-tuning run records which datasets it used, so a removal request can be traced.
S.3
Prompt access is need-based and loggedStaff can read customer conversations only through a review tool that records who looked and why.
S.4
Model provider terms are checked before useA new provider or model is approved only after its retention and training terms are compared with your customer commitments.
S.5
Model releases go through evaluation gatesA model version is promoted only after defined evaluations pass, and the result is stored with the release.
S.6
Retention applies to derived dataWhen a customer deletes a document, its embeddings and cached outputs are deleted on the same schedule.

06 · Traps

Where companies in this sector slip

Predictable, and cheaper to fix before fieldwork.

Embeddings treated as not being customer data. A vector built from a contract is still derived from the contract. Leave it out of deletion and the confidentiality story breaks.
Logs that quietly keep every prompt. Debug logging captures full prompts in a log platform with a long retention. The tool then belongs in scope.
Switching model provider mid-period. A new provider is a new subservice organization. The description, vendor review and customer notices must change with it.
Promising 'never used for training' with no control. A policy line is not a control. Show the filter in the dataset build and test it.
Annotators outside the access review. Contractors in a labelling tool see real prompts but are missing from the quarterly access review.

07 · Where a prompt goes

Mapping where a prompt travels

Draw this map before you draw the system boundary. Every stop is a scoping decision.

Entry. The prompt and any file arrive at your API. Decide what is logged and for how long.
Enrichment. Retrieval pulls related content from your vector store. That store holds customer data.
Model call. The prompt goes to your own model or to a provider. Record which, and under what retention terms.
Output. The response is returned and possibly stored as history. History is customer data with its own deletion rule.
Feedback and review. Ratings or flagged outputs may reach human reviewers. Name who, and how access is controlled.
Training. Only data whose tenant setting allows it may enter a dataset. This is the step buyers ask about most.

Each stop has an owner. Someone answers for the data at that point.
Each stop has a retention rule. Written, configured and evidenced.
Each external stop is a named vendor. Reviewed, with its terms matched to your commitments.

08 · Evidence

Evidence to have ready

What an auditor will ask for, in the order they usually ask.

A data-flow diagram from prompt to storage, model and training
Tenant training settings and the dataset build logs that applied them
Model provider agreements and your review of their assurance reports
Access logs from the prompt review tool, and the review of those logs
Model release records with evaluation results
Deletion records covering documents, embeddings and history

SOC 2 is not an AI management standard

The Trust Services Criteria do not address model fairness, transparency or AI risk management as topics of their own. Buyers who want that may ask for ISO/IEC 42001, a separate certifiable management-system standard, or refer to the NIST AI Risk Management Framework. Laws such as the EU AI Act set their own obligations. A SOC 2 report supports none of these as a compliance finding.

Sources: AICPA Trust Services Criteria (TSP section 100), Confidentiality criteria C1.1 and C1.2 and Privacy criteria P1 to P8; AICPA Description Criteria (DC section 200); ISO/IEC 42001; NIST AI Risk Management Framework.

More about SOC 2 for AI

We only call a third-party model API. Do we still need our own controls over prompts?

Yes. You collect and send them.

The provider controls its side. You still decide what is logged, how long history is kept, who on your team can read it and which provider receives it. Those are your controls, and the report tests them.

Should model weights be treated as confidential information?

If you say so, protect them as such.

Your own weights are usually your intellectual property rather than customer data. If a model is fine-tuned on one customer's data, the weights carry that customer's information and should sit under the same confidentiality controls.

Can the report say our model is accurate?

Not in the way buyers hope.

A SOC 2 tests whether controls operated, not whether outputs are right. You can show that evaluations were defined and run before each release. That is evidence of discipline, not of accuracy.

A customer asked us to delete their data from a trained model. What does SOC 2 expect?

That you do what you promised.

SOC 2 has no rule on removing data from trained models. It tests the commitment you made. If you promise exclusion from future training runs, show lineage and filtering. Avoid promising removal from existing weights unless you can actually do it.

Scope your AI SOC 2 around what buyers ask.

A readiness assessment turns this into your boundary, your categories and your gaps.

Get a SOC 2 readiness assessment Book a free consultation Back to SOC 2 services