#39 | SOX System Report Testing

Welcome to SOX system reports testing. Report testing has received a lot of focus by external auditors and the PCAOB recently so we want to address it today. The reason these system-generated reports are important is because management uses them to make judgements for booking journal entries or recording financial transactions and this impacts the financial statements.

These are the questions that we’re going to answer today:

• How do we know which system generated reports are in scope for SOX?
• What are the considered standard reports?
• What are non-standard reports?
• How do we to test them?
• What sample size should we use?
• What roll forward testing should we do?

I suggest you watch the video. It’s easier to understand if you are a visual/audio learner. The content below is the same as the video. It’s for those who learn by reading.

How do we know which system-generated reports are in scope for SOX?

The first part is whether the system generated reports should be in scope for SOX purposes.  There are three criteria to consider:

1. Serves a financial purpose and not just operational reporting
2. Associated with a key control activity
3. Materiality

   

Criteria 1: Does it serve a financial reporting purpose instead of a purely operational. An example is you could get a system report that shows you the ERP system uptime or the amount of time the ERP was available for any user. That’s great for operations but it actually has no financial reporting impact and it doesn’t have financial implications for our results.

Criteria 2: system generated report is associated with the key control activity. In the day-to-day running of a business you will run a lot of reports. In order to make sure that we scope and only look at reports for SOX purposes, we look at areas associated with key control activity such as the review and recording AR reserve or inventory reserve.

Criteria 3: materiality. Even if it’s for financial reporting purposes and is related to a key control, maybe the amount on this report is not material so we wouldn’t want to test it. We want to scope it out. It’s good practice to test it but it’s not required.

What are considered standard reports?

A standard report is a report that has not been customized or where the users don’t have access to make changes to the reports. An example is if you implemented a new ERP system like NetSuite or Oracle and the trial balance is a key report that every customer and user has access to and you can’t change the logic of the trial balance report.

If that’s the case, then there’s no testing needed to in order to rely on the trial balance report if your IT general controls are effective.

What are non-standard reports?

Now let’s focus on non-standard reports. By definition, any report that is not standard is non-standard. There are two things that we have to test for non-standard reports.

How do we test non-standard reports ?

1. Testing logical access which is system access to make changes to the report. We rely on the fact that the general IT controls around logical access have been working. If that’s the case, there’s no additional testing needed because the access hasn’t changed and no one has to access the report.

However, we do need to test the integrity of the report at some point.

2. Testing integrity of the report is the completeness and accuracy of the report. There are 4 ways to test the integrity of a report.
3. Test result of the original user acceptance testing– what are the results from when the system was originally implemented and the users wrote the requirements? The IT team wrote the report and then the user tested the report. If the user tested the report and no changes have been made to the report since the original approval, no additional testing is needed because that report continues to have integrity.
4. Do a logic review – read the query steps or the logic behind the report to see if it’s correct. An example would be an AR aging report. If you look at the query language it says to take the invoice date and if the invoice date is more than one day overdue and between 30 days put in the 30 category. If you read that logic and the logic make sense with what the report is supposed to do, that’s all you need to do.
5. Do a parallel simulation – re-create the report on your own and compare it to the original for the same results. If you create the report and you get the same results as the report generate from the system, you know however the system is doing it, it must be correct.
6. Corroborative inquiry and watching the user – watch the user generate the report. You also ask the user questions to understand the report. What’s the logic of the report? What results do you expect? And then watch them generate the report. Then you verify the results to ensure the report matches with what the user told you.

What sample size should we use to test system reports?

One (1). So many people push for automated controls because we only need to test 1 sample. That’s because we know that systems always act the same way, so we only need to test one report to know that it operates the same way each time.

What roll forward testing do we need to do?

The roll forward testing period is basically the period from your last tests until the end of the year.  If your IT general controls continue to work in Change Management then there’s no additional testing needed.

If you tested the report at one point in time, and the change management controls  continue to work, then nothing should change on that report. Or the changes were made and they followed the right test protocol then you can rely on the report.

What if you find an exception in the change management process?

You have to look at that exception and evaluate it to see if you might have to do additional testing.

Summary

To recap, here is what we covered today

• How do we know which system generated reports are in scope for SOX?
• What are the considered standard reports?
• What are non-standard reports?
• How do we to test them?
• What sample size should we use?
• What roll forward testing should we do?