SOX Testing: What Operating Effectiveness Actually Requires

Design test, operating test and evidence file, the three parts of SOX operating-effectiveness testing

Operating-effectiveness testing evaluates whether a control operated as designed during the relevant period, by people with appropriate authority and competence, using evidence sufficient for the control’s risk and frequency.

A well-written control description is not proof that the control worked. For management and auditors, the conclusion depends on what happened in practice and whether the available evidence supports that conclusion.

What is operating effectiveness in SOX?

Operating effectiveness asks whether a properly designed control was actually performed as intended during the period being assessed. It is different from design effectiveness.

Design effectiveness asks whether the control, if performed as designed, could prevent or detect a material misstatement on a timely basis.
Operating effectiveness asks whether the control operated as designed, whether it was performed by someone with appropriate authority and competence, and whether the evidence supports that conclusion.

A monthly reconciliation can be designed appropriately yet fail in operation if it was skipped, performed late, or reviewed without sufficient precision. A control can also be performed consistently and still be ineffective because its design does not address the relevant risk. Consistent execution cannot cure a design deficiency.

How do auditors test operating effectiveness?

PCAOB AS 2201 identifies four types of procedures: inquiry, observation, inspection, and reperformance. Auditors normally use a combination based on the nature and risk of the control.

Inquiry

Inquiry helps the tester understand who performs the control, how it works, what judgments are made, and how exceptions are handled. Inquiry alone is not sufficient evidence of operating effectiveness.

Observation

Observation shows the control being performed at a point in time. It can be valuable for activities that leave limited documentation, but it may not demonstrate that the control operated consistently throughout the period.

Inspection

Inspection examines records showing that the control operated: approvals, reconciliations, exception reports, review notes, system records, and follow-up evidence. The quality of the evidence matters as much as its existence.

Reperformance

Reperformance involves independently executing all or part of the control. It can provide persuasive evidence, but it is not automatically required for every control. The tester selects procedures that match the control risk.

How much SOX testing evidence is enough?

There is no universal sample size for every control. Under PCAOB AS 2201, the evidence required increases as the risk associated with the control increases. Frequency, complexity, judgment, prior results, expected failure rate, and the consequences of failure all affect the testing approach.

A practical program therefore avoids applying one sample size or one procedure to every control. High-risk, judgmental controls may need stronger or more extensive evidence. Lower-risk controls may support a proportionate approach.

What should the evidence file contain?

For each control, the file should allow a reviewer who was not present to understand what occurred and how the conclusion was reached.

A precise control description, including the risk addressed and the level of review.
The control owner and reviewer, including their roles and relevant authority.
The complete population from which any sample was selected.
The date or period of performance.
The evidence produced by the control.
The exceptions identified, how they were investigated, and how they were resolved.
The testing procedure, result, and conclusion.

A control with no exceptions can be entirely valid. Still, a long series of zero-exception results may justify asking whether the review was sufficiently precise and whether the file shows what the reviewer actually evaluated.

Why do management review controls require special attention?

Management review controls often depend on judgment. A signature or approval may prove that a review occurred, but not what the reviewer considered. Strong evidence identifies the expectations used, the threshold for investigation, the items challenged, and the resolution of those items.

For example, ‘management reviewed results’ is too vague. A stronger control states that the controller compares actual results with forecast by account, investigates variances above a defined threshold, documents the explanation, and resolves open matters before close approval.

How does filer status affect SOX testing?

Filer status can affect whether the company’s internal control over financial reporting is subject only to management’s assessment or also to external-auditor attestation under Section 404(b). Public float is an important input, but it is not the entire analysis. Reporting history, prior annual-report filing, smaller-reporting-company eligibility, emerging-growth-company status, and transition rules can also matter.

Companies approaching auditor attestation should confirm their status early and align management testing with the evidence expectations that will apply. Waiting until the fourth quarter can leave too little time to address design gaps or accumulate evidence after remediation.

What about controls performed by service organizations?

When financial reporting depends on a payroll provider, hosted ERP, revenue platform, or another service organization, management still needs to address the related ICFR risks. A SOC 1 report may provide relevant evidence, but it must be evaluated for scope, period coverage, exceptions, subservice organizations, and complementary user entity controls.

If the report does not cover the relevant controls or period, management may need bridge-period procedures, complementary controls, or controls that test the completeness and accuracy of information received from the provider.

Where does operating-effectiveness testing most often break down?

The control population cannot be shown to be complete.
The control operated, but the evidence does not show the precision or substance of the review.
Exceptions were identified but not evaluated or resolved.
The control owner lacked sufficient authority to challenge or approve the activity.
Testing began too late to allow remediation and a sufficient period of effective operation.

What should management do this quarter?

Confirm the control population, test design before testing operation, establish complete populations, select procedures based on risk, and begin early enough to investigate exceptions. If auditor attestation may apply, confirm that status before the testing plan is finalized.

Prepare for SOX 404(b) auditor attestation with A2Q2

FAQ

Does operating effectiveness require testing every occurrence?

No. The tester obtains sufficient evidence based on the control’s risk, frequency, nature, and other relevant factors. Complete populations may be necessary even when only a sample is tested.

Can a control pass operating testing if its design is ineffective?

No overall effectiveness conclusion can rest on an ineffective design. A control may be performed consistently, but consistent performance does not correct a design that fails to address the relevant risk.

Is inquiry enough to test a SOX control?

No. Inquiry should be combined with other procedures such as observation, inspection, or reperformance.