Test of controls
Also known as: Operational effectiveness testing, Compliance testing, Control validation
An audit procedure used to determine if a specific security or financial control is operating effectively over a defined period. It verifies that the process described in policy—such as an ISO 27001 requirement—is actually followed in practice. This goes beyond checking if a rule exists to confirming it consistently works.
Why it matters
Failure here means the auditor cannot rely on the control to prevent or detect errors, fraud, or breaches. They will conclude there is a "material weakness" or "significant deficiency," which often triggers more expensive and time-consuming substantive testing of raw data. This can lead to a qualified audit opinion, damaging an organization's reputation with clients and regulators. Ultimately, the CISO or CFO is held accountable for these failures during board reporting.
In practice
An external auditor or internal compliance officer performs this after verifying that the control design is adequate but before issuing a final report. They request evidence such as system-generated logs, signed approval forms, or configuration screenshots from across the audit window. The results are documented in an Audit Work Paper (AWP) detailing the sample size and any exceptions found. In a first-year engagement, auditors typically test larger samples to establish a baseline of trust. For repeat engagements, they may reduce the sample size if previous years showed consistent compliance.
Worked example
FinTechFlow sought SOC 2 certification for their cloud environment in 2023. The auditor focused on the "User Access Review" control, which requires managers to certify employee permissions every quarter. The auditor requested a random sample of 15 access reviews conducted between January and December. Upon inspection, they found that two reviews in Q3 were signed off by a manager who had already left the company at the time of the signature. This was marked as an exception, leading to a "non-effective" rating for that specific control.
Common mistakes
- Confusing design with operation; verifying that a policy is written but failing to check if staff actually follow it.
- Using biased samples by allowing the client to provide "representative examples" instead of selecting random dates or users.
- Accepting screenshots without timestamps or system metadata, which allows for retrospective manipulation of evidence.
- Testing too few items—such as a single instance—and incorrectly concluding that a control is operational across an entire year.
Frequently asked questions
What is the difference between a test of controls and substantive testing?
A test of controls checks if a process (like a password policy) works to prevent errors, while substantive testing examines the actual data (like checking individual transactions) to see if an error actually occurred. One tests the fence; the other looks for holes in the ground.
How many samples should I pick for a control test?
Sample sizes depend on the frequency of the control and the audit standard used, such as AICPA or ISACA guidelines. For a daily control, an auditor might test 25-30 instances; for an annual control, they may only test one.
When is the best time to perform these tests during an engagement?
They should occur after the "walkthrough" phase, where you confirm how the control is supposed to work. Testing too early risks checking a process that hasn't been fully implemented or documented yet.
What constitutes valid evidence for a test of controls?
Valid evidence must be objective and verifiable, such as system-generated logs, tickets in Jira, or timestamped emails. Verbal confirmation from a manager is generally considered insufficient "hearsay" and will not pass a rigorous audit.
Can a control fail the test even if it is perfectly documented in the policy?
Yes; this is a failure of operational effectiveness rather than design. The documentation proves the organization knows how to be secure, but the test proves they are not actually doing it.
More terms
Management representation letter · General-purpose AI model · Audit evidence · Residual risk · Records of processing activities · Disclaimer of opinion · Scoping · Unqualified opinion