Charta is on Google Play: practise on the go. Get the app
Charta

Describe automating controls, designing internal controls and testing them, noting the risks and difficulties in each and how to make them work better.

FRM® Part II · Operational Risk and Resilience · Risk Mitigation · FRM2.ORR.5.c

The idea

Automation reduces manual error and cost and applies a control the same way every time. But a wrong rule or configuration is also applied consistently and at speed, and staff can lose the skills to notice failures. Automated controls rely on general IT controls such as change management, access security and backups. Without them the control can be altered or disabled. Good design starts from mapped processes and identified risks, concentrates on key controls, gives each an owner, and avoids piling on duplicate controls that staff sign off without checking. Testing has two parts. Design effectiveness asks whether the control would work if performed as described. Operating effectiveness asks whether it actually worked over the period, usually by sampling and re-performance. Testing should be independent of the control owner, and deficiencies need remediation and retesting. The exam often asks you to tell design from operating effectiveness.

A practice question

From the question bank, with the reason each wrong option is wrong.

A reviewer walks through a control as written and concludes that, if performed as described, it would address the identified risk. Which assessment has been made?

  1. Operating effectivenessWhy not: Operating effectiveness is tested by examining whether the control was actually performed correctly over a period.
  2. Inherent riskWhy not: Inherent risk is the exposure before any controls; it is not an assessment of a particular control.
  3. Design effectiveness - correct
  4. Residual riskWhy not: Residual risk is what remains after controls; the reviewer has judged the control itself.

Design effectiveness asks whether the control, as designed, can mitigate the risk. Whether it was carried out correctly in practice is operating effectiveness.

A harder one

A key reconciliation control passed its design review. A test of 60 transactions finds 3 not reconciled, and management's tolerable exception rate is 4%. What is the appropriate conclusion?

  1. The control is effective because 95% of the sample passed and only a few items failedWhy not: Ignores the tolerance; the comparison should be exceptions against the tolerable rate, not the pass rate against 100%.
  2. Operating deficiency: the 5.0% exception rate exceeds the 4% tolerance - correct
  3. The control has a design deficiency because exceptions occurredWhy not: Confuses design with operation; the design passed, and the failures show the control was not operated reliably.
  4. The result is inconclusive until at least 100 items are sampledWhy not: Invents a minimum sample size; the observed rate already exceeds tolerance and calls for remediation.

Exceptions are 3 / 60 = 5.0%, above the 4% tolerance. Since the design was sound, the problem lies in how the control is being operated, so it is an operating effectiveness deficiency to be remediated and retested.

The hardest tier

An operations team tests its own reconciliation controls each year and reports them effective, yet losses from reconciliation breaks keep rising. Which change would most improve the testing?

  1. Having the same team test more frequently with the same methodWhy not: More frequent self-testing keeps the lack of independence and the same blind spots.
  2. Replacing testing with an annual attestation signed by control ownersWhy not: An attestation is weaker evidence than testing and relies on the same owners.
  3. Independent testing by a second or third line function, with re-performance - correct
  4. Testing a control only after a loss event has been traced to it by the businessWhy not: Waiting for losses makes testing reactive; testing should find weaknesses before they cause loss.

Self-assessment by the owners is useful but not independent evidence. Testing by a function outside the process, with re-performance of actual controls, gives a more reliable view of operating effectiveness.

In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.

Start free →

More in Risk Mitigation