Charta is on Google Play: practise on the go. Get the app
Charta

Outline the drawbacks of using simulation to solve financial problems.

FRM® Part I · Quantitative Analysis · Simulation and Bootstrapping · FRM1.QA.13.g

The idea

Simulation is powerful but has real drawbacks. It is computationally expensive, because the standard error falls only with the square root of the number of replications. Halving it needs four times the runs, and nested or high-dimensional problems multiply the cost. Results are estimates, not exact values, and they differ across seeds and replications. A standard error must be reported, and differences between runs compared with it. Results are specific to the model and parameters assumed. A poor data generating process, such as normal returns when the data are fat-tailed, gives a precise answer to the wrong question, and more replications cannot repair it. Simulation can also be hard to debug and to explain. The exam point: separate sampling error, which more draws reduce, from model error, which they do not.

A practice question

From the question bank, with the reason each wrong option is wrong.

Which is a fundamental drawback of using simulation to solve financial problems?

  1. Simulation cannot produce results for problems with no closed-form solution.Why not: Simulation is mainly used precisely when no closed-form solution exists.
  2. The results are only as good as the assumed model, such as normal returns. - correct
  3. Simulation results are exact once the program has run once.Why not: Results are estimates with sampling error that falls only as 1/sqrt(N).
  4. Simulation cannot be applied to path-dependent payoffs.Why not: Path-dependent payoffs are a standard use of simulation.

Output depends on the data generating process and parameters supplied. If returns are fat-tailed but the model assumes normality, the simulation will faithfully give the wrong answer. Simulation also has sampling error and can be computationally costly.

A harder one

A simulation with 10,000 replications has a standard error of 0.8 and takes 20 minutes. If run time is proportional to the number of replications, how long will it take to reach a standard error of 0.2, using the same method?

  1. 320 minutes - correct
  2. 80 minutesWhy not: Scales the run time by the ratio of standard errors rather than by its square.
  3. 40 minutesWhy not: Takes the square root of the ratio of standard errors instead of squaring it.
  4. 1,280 minutesWhy not: Cubes the ratio of standard errors; the required factor is the square.

The error scales with 1/sqrt(N), so a fourfold cut needs 4² = 16 times as many replications (160,000). Run time is 20 x 16 = 320 minutes, about 5.3 hours.

The hardest tier

Two analysts run the same simulation with different seeds. They get 4.18 and 4.31, each with an estimated standard error of 0.06. Which conclusion is correct?

  1. The z-statistic is 2.17, which exceeds 1.96, so the results are inconsistent and the code is probably faulty.Why not: Uses a single standard error for the difference; the standard error of a difference of two independent estimates is sqrt(2) x 0.06.
  2. z = 1.53 (difference over sqrt(2) x 0.06), within sampling error at 5%, so no evidence of a bug. - correct
  3. The z-statistic is 1.08, which is within sampling error, but only because the standard errors were added.Why not: Adds the two standard errors (0.12) instead of combining them in quadrature; the conclusion is right by accident but the statistic is wrong.
  4. The z-statistic is 3.06, which exceeds 1.96, so the seeds must have been poorly chosen.Why not: Divides the standard error by sqrt(2) instead of multiplying, so the difference looks too significant.

The two estimates are independent, so the standard error of the difference is sqrt(0.06² + 0.06²) = 0.0849. z = 0.13 / 0.0849 = 1.53, below 1.96. Differences of this size arise from sampling error, and results differ across runs and seeds.

In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.

Start free →

More in Simulation and Bootstrapping