Charta is on Google Play: practise on the go. Get the app
Charta

Account for the ways of supervising cyber-resilience, the approaches to testing and responding to incidents, and the metrics for cybersecurity and resilience.

FRM® Part II · Operational Risk and Resilience · Cyber-resilience: Range of practices · FRM2.ORR.8.c

The idea

Supervisors assess cyber-resilience through off-site reviews, on-site inspections, thematic reviews, self-assessments and sometimes supervisor-led testing. Institutions test with a ladder of methods. Vulnerability scans find known weaknesses, and penetration tests try to exploit them. Tabletop exercises rehearse decisions and communication without touching systems. Threat-led red teaming imitates real adversaries, using threat intelligence against critical functions, often without telling the defenders so that detection and response are genuinely observed. Incident response needs tested plans, escalation paths, recovery objectives, reporting to authorities and lessons fed back into the framework. Metrics fall into groups. Protection metrics include patch compliance and phishing-test results. Detection and response metrics measure the time to detect, contain and recover. Incident counts form a third group. No standard set exists. Beware averages distorted by outliers and compliance rates with the wrong denominator. Choose the test or metric that fits the stated objective.

Formulas

Patch compliance rate

compliance = patched ÷ (in scope − exempt)

patched
systems patched within the deadline
in scope
all systems covered by the metric
exempt
systems with formally approved exemptions

The denominator must follow the stated definition. Leaving exempt systems in the denominator understates compliance.

A practice question

From the question bank, with the reason each wrong option is wrong.

Which type of test uses current threat intelligence to imitate a real adversary's methods against an institution's critical functions?

  1. An automated vulnerability scanWhy not: A scan lists known weaknesses; it does not imitate an adversary's campaign against critical functions.
  2. A tabletop exerciseWhy not: A discussion-based walk-through of an incident, with no live attack on systems.
  3. Threat-led penetration testing (red teaming) - correct
  4. A business continuity plan reviewWhy not: A paper review of recovery arrangements, not a simulated attack.

Threat-led penetration testing or red teaming draws on intelligence about realistic attackers and attacks critical functions as they would. Scans, tabletops and plan reviews are different tools.

A harder one

In a threat-led penetration test, only a small control team knows the test is under way and the security operations staff are not told. What is the main reason?

  1. To avoid having to report any of the findings to the supervisorWhy not: Findings are normally shared and remediated; secrecy during the test is about realism, not avoiding reporting.
  2. To observe genuine detection and response behaviour - correct
  3. To keep staff from patching systems before the testers use known exploitsWhy not: The aim is to see real detection and response; a patching rush is not the principal reason.
  4. Because the test is carried out against a copy of the system, so staff have no roleWhy not: Tests are typically run against live critical systems with safeguards, and the role of staff is exactly what is being observed.

Keeping defenders uninformed shows whether the bank would detect and respond to a real intrusion. The control team can stop the test if it threatens live services.

The hardest tier

Detection times for five incidents were 2, 3, 4, 5 and 46 hours. Which figure or approach is most appropriate for reporting detection speed to the board?

  1. Report only the 12-hour mean, because averages are the sole acceptable summaryWhy not: Uses the arithmetic mean alone, which one outlier distorts, and hides the typical performance and the tail.
  2. Report 3.5 hours after dropping the 46-hour incident as an anomalyWhy not: Excludes the worst case, which is exactly the one that matters most for resilience.
  3. Report only the 46-hour maximum, since the worst case alone captures the riskWhy not: Ignores the typical experience; a single maximum gives no sense of overall detection performance.
  4. Report the median of 4 hours alongside the 12-hour mean and investigate the 46-hour incident - correct

Mean = (2+3+4+5+46)/5 = 12 hours; median = 4 hours; excluding the outlier gives 3.5. Skewed data call for the median plus a look at the tail and the outlier's cause.

In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.

Start free →

More in Cyber-resilience: Range of practices