#25 | Part 9 – Evidence to Get is Based on Risk in Demystifying SOX 404 – Auditing Standard 5
Welcome to part 9 of Auditing Standard 5. This training series gives context to SOX 404 requirements. In part 8, we talked about selecting controls to test, testing the design of controls and testing if the controls operate effectively.
In this section, we will discuss how much testing to do and the amount of evidence you need to gather. It covers:
- How much evidence to get is based on risk
- Factors that affect the risk
- Deviation vs exception
- Types of test procedures
- Documents supporting controls
- Timing of test of controls
- Extent of testing
I suggest you watch the video. It’s easier to understand if you are a visual/audio learner. The content below is the same as the video. It’s for those who learn by reading.
Take a look at the table of contents to find the sections that are of interest or focus to you. You can watch those videos or read those blog sections to get an idea of what the section means.
How much evidence to get is based on risk
“Relationship of risk to the evidence to be obtained” is a complex way of saying the amount of support you need depends on how high your risk is.Paragraph 46 says as the risk increases the evidence to support that the risk is under control has to increase.
The note in paragraph46 says the auditor has to get evidence about the relevant assertion, not necessarily test every control for effectiveness. For example,a particular assertion like existence of assets may have 20 controls. You don’t have to test all 20 controls to say that assertion is covered. You may pick 2 or 3 controls or any combination to give you comfort and the level of evidence needed varies. The level of evidence can change based on what that assertion is. This is great news because it means we don’t have to test every control that’s listed and that’s why we pick only key controls.
Factors that affect risk
Paragraph 47 talks about the factors that affect the risk associated with the control and some of them you’ve already seen because we’ve done the risk assessment.
Nature and materiality means how big the amount is. If the amount is going to be big, then that’s a higher risk in terms of a misstatement.But if your petty cash was $200 or $500 off, do you really care? Is that really material to your financial statements?
Inherent risk is about the accounts themselves. People are more likely to take cash because it’s “fungible” (just a fancy word to say cash could walk off). People don’t steal prepaid assets. Who can steal a prepaid when it’s an amount already paid to someone else like the insurance company? All you’re doing is amortizing (decreasing) the insurance premium over time. So the risk for cash getting stolen is inherently higher than the risk for prepaids.
Another factor to consider in risk is the volume of transactions. If you have millions of transactions, this will have more risk of a mistake happening than if you just have 10or 100 transactions.
Next is nature of transactions. If you think about a software company that is selling perpetual software licenses and you have to determine VSOE and you do carve outs of separate arrangements, which are more complex than if you have a software company that sells software-as-a-service. In a SaaS model, you pay a certain amount each month or year to use the software. The nature of the SaaS transaction is pretty homogeneous and consistent and there’s not much judgment involved.
There’s also the effectiveness of the entity level controls and whether some of those entity level controls monitor other controls. An example is the budget to actual variance analysis that smaller companies often use as a key control. When the operations are fairly simple or the CEO or CFO look at the detailed budget to actual variances across different departments and at a particular level, that can give you comfort and reduce your risk.
There is also the frequency of the control. If your control happens only once a year versus daily or monthly, that also changes the risk profile. The more a control happens, the more likely that you can catch something or prevent something sooner than if you only did it once a year.
Another factor is the how complex the accounting system (also called ERP) is. Using an off-the-shelf software is easier to get evidence that it works than when you have a home-grown system. With of-the-shelf systems, we can test information technology general controls (ITGC) and that’s why we integrate the manual financial controls with ITGC.
The competence of the personnel or the person doing the work is a factor. Do they actually understand what they’re doing? Does this person really know what they’re supposed to do? Did they receive training? Do they have experience doing it?
The next point is automated controls which generally have lower risk versus a manual control. Think about it. A computer or a machine will do a sequence of steps exactly the same way every time. It doesn’t get tired. It didn’t go out late last night and can’t read a particular report or transpose a number. Generally, automated controls have lower risk or higher reliance and that’s why we want to push as much as possible to automated controls.
Another risk factor is the amount of the judgment that needs to be made. Remember that a lot of the big fraud cases are mostly about management making judgments and saying “Do I want to reserve a higher or lower amount?” You can’t really argue whether it’s clearly right or wrong. There’s just varying degrees of support. Again, judgment is key to this.
Deviation vs Exception
Note that it’s easier to say a control is not working than to say a control is working. Some of our clients have deficiencies because the evidence isn’t there to support that the control is working.
Paragraph 48 talks about deviations. Eventhough a deviation sometimes happens, it does not mean the control has failed.
Let me give you an explanation between deviations and exceptions. A deviation example is normally there’s a control where the controller will review the AR reserve on a quarterly basis. They look at all the relevant reports, does the tie outs and signs off. What happens if the controller goes on vacation,is out sick, is on leave or absent for whatever reason? That control can be delegated to another person who is equally competent and can do the job like the VP Finance or the CFO.
That is a deviation from the normal process but it doesn’t mean the control is not effective. It may be more effective because a fresh set of eyes is now looking and maybe they’re questioning things that the same person might not question. Deviation is something that isn’t exactly the way it’s documented but when you look at it, it makes business sense.
In an exception, the controller goes on vacation and no one reviews the AR reserve. A staff accountant prepares the AR reserve, books the reserve and no one takes a look at it. This is a change in the process and it’s an exception because you don’t have a qualified person doing a review. That’s the difference between a deviation and exception.
Types of Test Procedures
Paragraph 49 says with each control, you can use different combinations of types of testing and timing to get enough evidence. An example is a walkthrough. If you do a walkthrough in combination with asking people, watching them do the task, inspecting the reports they have and re-performing the control. You would be able to say that I’ve now got enough evidence for that particular sample and it’s operating effectively.Paragraph 49 is saying that you have to change the combination of the testing depending on the control.
Paragraph 50 talks about the type of testing you could do and how much comfort each test gives you, from the least comfort to the most comfort.
Inquiry or asking someone is the least supportive because there’s no proof other than their word.
Observation is the next level you can rely on. Observing someone doing something is better than asking them because you can see it.
What’s even better than that is inspecting the relevant documents. You can watch them looking at the report and see what they particularly look at. Ask them which pages, which subtotals? What did you do? That’s inspecting it.
The one test that gives the most comfort in terms of operating effectiveness is re-performing the control. In this case, you can ask the controller, “did you do the AR reserve?” and they’ll say yes. Then you can watch them look through the AR aging and do the calculation. Then you can take the actual AR reserve and look at various things. Once you actually go through and do all the steps the controller should have done and come up with the same results within a reasonable amount, you’ve re-performed that control. You know for sure that this person has done it correctly.
The note in Paragraph 50 says that inquiring alone is not sufficient evidence to conclude a control is operating effectively. Inquiry is good when you’re documenting a process but it’s not enough when you’re doing the test of effectiveness.
Supporting documents of controls
Paragraph 51 is about the physical documentation or soft copy or the actual evidence of the control. How much documentation you keep in your records depends on the degree of risk.This is where you have done the review of a control. How much supporting documentation do you keep as evidence that you’ve done your job? The higher the risk, the more support documentation you need to keep.
This note says that smaller companies sometimes do all these controls but they may not keep all the formal documentation. So you have to use your judgment and use a combination of testing techniques to get comfortable that a control is effective. It’s just recognizing again that smaller companies may not have the same level of formal documentation that large companies may have.
Timing of test of controls
Paragraph 52 talks about the timing of the test of controls. I highlighted two separate colors in the picture above to help you focus on two separate thoughts. The first one is the length of test period. The longer period of time you test, the more support and more comfort you have that a control is working. Fairly logical. If you test the controls over a 9-month period and results shows that they work well,that’s more comforting than if you only test controls for three months. Based on only 3 months of evidence, could you say that the controls worked for the whole year?
The next part is when to test the controls. The closer you are to the measurement date (also known as management assertion date or the year-end date), the more comfort you have. An example is if you do all of your tests in the 1st and 2nd quarter and they all passed. You didn’t do other testing and on December 31st, would you know if the controls are still working? That is less comfort than if you did testing for Q1, Q2, Q3 and some minor testing in Q4. The closer you get to date, the more comfortable you know that things haven’t changed.
Paragraph 53 says that if the company implemented changes to make things more efficient or effective later in the year, don’t go back and test the earlier periods. Test the controls that were implemented later because they reflect the process closest to the year-end.
Extent of testing
Paragraph 54 talks about the extent of testing or how much testing you want to do. It says the more extensive a control is tested, the greater the evidence you have to get and keep.
Paragraph 55 introduces the concept of “rollforward” procedure. We often do a lot of testing in Q1, Q2 and Q3 and then less testing inQ4. Q4is also called a “stub period”. Think of a“stub”as a smaller piece. How do you make sure thatall the testing you’ve done in those three periods continues to work? What you do to testthat remaining, shorter period is called “rollforward procedure”.
Paragraph 56 says that what testing you do in that rollforwardperiod depends on the control. How long is the stub period?Meaning, we’ve tested everything up to November, and we only havethe month of December left. That gives us more comfort versus we tested everything through August and now we have to test September through December. How many changes happened in the last four months? Maybe the company implemented a whole new accounting system that just went live for 2 or 3 months.
There’s more risk the closer the changes happen towards the end of the year.
In some cases, when the risk is low in that stub period, you can really reduce the test work.AS5 saysthat inquiry alone might be sufficient. Even though the literature says that, often times the auditors have different guidelines for 4th quarter. Auditors are not going to rely on just inquiry; they’re really going to inspect or do something more than inquiry.
Summary
We covered a lot of ground in this session. To recap, we covered
- How much evidence to get is based on risk
- Factors that affect the risk
- Deviation vs exception
- Types of test procedures
- Documents supporting controls
- Timing of test of controls
- Extent of testing
Leave a Reply