Evidence and testing
Penetration testing, seen from the auditor's chair.
No Trust Services Criterion orders a penetration test by name. Once your own control says you run one, though, the auditor tests that sentence: the report, its date, its scope, who did it, and what happened to every finding. This page is about making that trail hold up.
Back to SOC 2 services01 · Your wording
The control you write is the control they test
Each clause in the control becomes a separate thing to evidence.
Penetration testing appears in the criteria only as an illustration of the evaluations management might use to monitor controls. Whether it is required for you depends on what your system description commits to. Our page on pen tests and recovery tests explains why; this one covers what happens next.
| If your control says | The auditor will look for |
|---|---|
| “An external penetration test is performed annually” | A report dated inside the period, from a tester outside the team that builds the product |
| “…and after significant changes” | Your list of significant changes, and a test or a documented decision for each |
| “High and critical findings are remediated within a stated window” | Every high and critical finding with open and close dates, compared with the window |
| “Remediation is verified by retest” | A retest letter or updated report showing each fix held |
| “Results are reviewed by the security lead” | A dated record of that review, by that role |
02 · Reading the report
What an examiner reads in a test report
They read it for coverage and follow-through, not for the exploits.
- Scope statement. The applications, APIs, networks and environments tested, matched to the system boundary in your description.
- Test dates. When testing ran and when the report was issued, both inside the period.
- Tester independence. Whether the testers are separate from the people who built and run the system.
- Qualifications. Who the testers are and the basis for their competence, such as experience and recognised credentials.
- Method. Approach and depth: authenticated or not, black or grey box, the standard followed.
- Findings. Each with a severity, the affected asset and a clear description.
- Remediation and retest. What was fixed, when, and proof the fix was checked.
Most SOC 2 reports do not list individual pen test findings. The auditor's interest is whether your process for handling them operated as described.
03 · Findings to evidence
How a finding turns into evidence
The report is the start of the trail, not the end.
Log
Every finding enters your tracker the day the report lands.
- What happens
- Create one ticket per finding
- Copy severity from the report
- You get
- A dated population of findings
- Good practice
- Ticket links back to the report section
- Watch for
- One ticket for the whole report
Triage
Assign an owner and a due date from your own policy.
- What happens
- Confirm severity
- Set the due date from the stated window
- You get
- Ownership the auditor can trace
- Good practice
- Downgrades recorded with a reason and an approver
- Watch for
- Silent severity changes
Fix
Remediate through your normal change process.
- What happens
- Raise a change
- Review and deploy
- You get
- Change records tied to each finding
- Good practice
- The fix passes the same change control as any release
- Watch for
- Hotfixes with no approval trail
Retest and close
Close only when the fix is confirmed.
- What happens
- Retest by the tester or by documented internal verification
- Close the ticket
- You get
- Proof the risk is gone
- Good practice
- Accepted risks signed by a named leader
- Watch for
- Closing on “deployed” alone
04 · Scope mistakes
Scope mistakes that weaken a good test
A thorough test of the wrong thing is still the wrong evidence.
- Testing the marketing site. The customer-facing product and its APIs are what the boundary covers.
- Staging only, without saying so. Fine if staging mirrors production and the report says so; a problem if nobody can tell.
- Skipping the cloud control plane. Misconfigured identity and storage settings are often where real exposure sits.
- Leaving out a region or product. If the description includes it and the test doesn't, expect a question.
- An old report reused. A test from before the period does not evidence a control that runs during it.
05 · With scanning
How it fits with vulnerability scanning
Scanning is the steady signal; the pen test is the deep check.
| Question the auditor asks | Scanning answers it with | A pen test answers it with |
|---|---|---|
| Did you look for weaknesses all period? | Scheduled scan runs across the period | Not on its own |
| Did someone try to chain weaknesses together? | Not on its own | The test narrative and findings |
| Were findings fixed on time? | Open and close dates per finding | Remediation tracker and retest |
| Was the whole boundary covered? | Asset list matched to scan targets | Scope statement matched to the description |
When both are described, both are tested. A finding the pen tester raised that your scanner also flagged months earlier is a question about your scanning process, not just the test.
06 · What to hand over
The package to hand the auditor
One folder, in this order, answers most follow-up questions.
- The control text as it appears in your description.
- The final test report, with scope, dates and tester details.
- The engagement letter or statement of work showing scope and the tester's independence from your team.
- Your findings tracker export: each finding, severity, owner, due date, close date.
- Change records for each fix.
- The retest letter or updated report.
- Any risk acceptances, signed and dated.
Where Auditify fits
Auditify performs penetration testing as a separate service and prepares teams for SOC 2. The test report is evidence for your control; the independent CPA firm decides what it concludes from it.
Sources: TSP section 100, 2017 Trust Services Criteria with revised points of focus (2022), CC4.1 and CC7.1; AICPA attestation standards AT-C section 205; AICPA Guide, SOC 2 Reporting on an Examination of Controls at a Service Organization.
Questions about pen tests in a SOC 2
Can our own engineers do the penetration test?
Only if your control allows it, and objectivity is a hard sell.
Nothing in the criteria forbids an internal test, but the auditor considers the objectivity and competence of whoever performed it. People testing systems they built are hard to describe as independent. If your control says “external”, it must be external.
What if the test finds something serious during the period?
That is the test working. What matters is what you do next.
A finding is not an exception. Failing to triage, fix or retest it within the window your control states can be. Log it, own it, fix it and keep the trail.
Does the auditor need the full report or just a summary letter?
Usually the full report, under confidentiality.
A summary letter suits customers. The examiner typically needs enough detail to compare scope with the boundary and trace findings to fixes, which a one-page letter rarely gives.
Our test happened two weeks before the period started. Does it count?
Not for an annual control that runs during the period.
The examination covers the period. A test before it can be context, but it does not show the control operating inside the window. Plan the test date with the period in mind.
Make the test count twice.
A SOC 2 readiness assessment checks that your pen test control, scope and remediation trail match what the auditor will sample.