Model RiskMedium

Every price that is not a traded price is the output of a model, and a model is an argument with assumptions in it. Model risk is what happens when the argument stops holding and the number keeps printing.

7 min read · 1 285 words

What a model is doing

Two kinds of number appear on a trading screen. The first is a price somebody actually paid. The second is a price nobody paid, computed from prices people did pay, plus assumptions about how the world connects the two.

Almost everything past the simplest instruments is the second kind. An option that trades has a price; the same option one strike further out, three months longer, on a name that trades twice a week, does not — and it still has to be valued, margined and risk-managed every day. A model is what fills that gap. It takes what is observable and produces what is not.

Model risk is the risk that the filling is wrong, and that nothing about the output says so. That last clause is the whole subject. A model does not fail loudly. It produces a plausible number, in the right format, to the usual number of decimal places, on a day when it has stopped being right.

Calibration is not estimation, and the difference matters

• Estimation asks what the world is like: measure historical volatility, fit a default probability from observed defaults, estimate a correlation from returns. It is a statement about reality, testable against more reality.
• Calibration asks what parameters make this model reproduce the prices the market is quoting today. It is not a statement about the world at all; it is a statement about the market's own current view, translated into the model's language.

Both are legitimate and they answer different questions. The failure is confusing them — treating a calibrated parameter as a measurement of the world, or a historical estimate as a market price. A volatility that reproduces today's option prices is not a forecast of how much the underlying will actually move, and volatility sets out how far apart those two numbers routinely sit.

The assumptions that fail first

Named in roughly the order in which they cause trouble:

• That you can trade continuously and in size. Most derivative pricing rests on being able to rebalance a hedge as the market moves. When the market gaps, or the hedge instrument stops quoting, the argument that justified the price is gone while the price itself carries on being computed. Liquidity is the mechanism.
• That the distribution has thin tails. Returns are not normally distributed, everyone knows it, and models built on the assumption remain in use because the alternative is harder. The error is small in the middle and enormous exactly where the money is lost.
• That correlations are stable. A portfolio's risk depends on how its parts move together, that number is estimated from a calm period, and it changes in a stress — usually towards one, which removes the diversification the position was sized against. Diversification covers what that does to a portfolio.
• That the past sample contains the event. A model fitted to a period without a particular kind of shock has no view on that shock. The output does not mention this.
• That the inputs are right. The most common real-world failure is not an exotic mathematical flaw. It is a curve built from a stale quote, a dividend assumption nobody updated, a holiday calendar, a day-count convention. Unglamorous, frequent, and the reason product control exists.

The fair-value hierarchy is a model-risk disclosure

Accounting frameworks sort valuations into three levels, and read correctly the classification is a statement about how much of a firm's reported value is somebody's opinion:

• Level one — a quoted price in an active market for the identical instrument. No model.
• Level two — a model, but every significant input is observable: quoted prices for similar instruments, curves and surfaces the market publishes.
• Level three — a model with at least one significant input that is not observable. The firm supplies the number; nobody outside can check it.

A large level-three balance is not evidence of anything improper. It is evidence that a material part of the reported figure cannot be verified externally, which is a different and entirely legitimate thing to know. How a position hits the books covers why day-one profit on a level-three trade is deferred rather than recognised, which is the framework applying exactly this scepticism.

How firms are supposed to contain it

• Independent validation. A team that did not build the model rebuilds the answer another way, tests it at the boundaries, and documents what it is not to be used for. The last part is the part that gets ignored.
• A model inventory with owners. Every model in production, what it is used for, when it was last reviewed. Models applied outside their stated purpose are a recurring finding, and it usually happens because the model was good and somebody reached for it.
• Reserves and adjustments. A valuation adjustment held against model uncertainty, unobservable parameters, close-out costs and funding. It is an explicit admission that the headline number is not exact, expressed in money.
• Backtesting. Compare what the model said would happen with what did. A risk model that breaches its own confidence level more often than it should is telling you something, and the number of exceptions is a supervisory input rather than an internal curiosity.
• Benchmarking. A second model, deliberately different in structure, run in parallel. Agreement is weak evidence; disagreement is strong evidence, and it is the disagreement you are paying for.

Where it has been visible in public

• LTCM in 1998 — positions sized on relationships that had held historically, in size large enough that unwinding them moved the very prices the model assumed were exogenous. The model was not wrong about the relationships; it was silent about what happens when the model's own holder has to sell.
• Metallgesellschaft in 1993 — a hedge that was defensible over the life of the contracts and produced cash demands in the meantime that the framing had not accounted for. A model of the final outcome is not a model of the path.
• Volmageddon in 2018 — products whose mechanical rebalancing was a known, published feature, sized by holders against a distribution of moves that did not contain the move that occurred.
• 2008 — correlation assumptions inside structured credit, where the parameter that mattered most was the one estimated from the least relevant data.

None of these is a story about arithmetic being done wrong. In each, the arithmetic was right and the frame around it was too small.

The uncomfortable conclusion

A model that is obviously wrong is comparatively safe: nobody sizes a position against it. The dangerous model is the one that has been right for long enough that the assumptions inside it have stopped being read as assumptions and have become the way people describe the world.

Which suggests the only durable defence, and it is a habit rather than a control: be able to say, for any number you are relying on, what would have to stop being true for it to be wrong. If that question has no answer, the number is not a measurement — it is a belief with decimal places. Getting it right in writing is the same discipline applied to a sentence rather than a position.

What to take away

• Any price that is not a traded price is a model output, and the model carries assumptions the output does not display.
• Calibration reproduces today's market; estimation describes the world. They are not interchangeable.
• Continuous trading, thin tails, stable correlations and a representative sample are the four assumptions that fail first.
• The fair-value hierarchy tells a reader how much of a firm's value nobody outside can check.
• The dangerous model is the one that has been right long enough for its assumptions to stop being visible.