How to Read Any Market NumberStart here

Before believing a figure, five questions. Applied to returns, volatility, ratings, back-tests and the statistics that most often mislead.

The five questions

• 1. What exactly is being measured? "Return" can mean price return, total return, gross of fees, net of fees, in one currency or another. All six are different numbers for the same holding.
• 2. Over what period, and why that one? The period is chosen by whoever is showing you the number. A start date one year earlier or later frequently reverses the conclusion.
• 3. Compared against what? An unstated benchmark is the most common way to make an ordinary result look remarkable — and a price index is the most common unstated benchmark.
• 4. Who is missing from the sample? Funds that closed, companies that delisted, strategies that stopped being reported.
• 5. Is the sample large enough to support the claim being made? Almost always the answer is no, and almost never is that stated.

Question 1 in practice: the definition problem

Convert everything to one basis before comparing — the volatility converter and the tax-equivalent yield calculator exist for exactly this step.

Question 4 in practice: survivorship and selection

• Survivorship bias: a database of funds that exist today omits the ones that closed, and funds close because they performed badly. The average of the survivors overstates the average of the population, historically by a meaningful margin.
• Selection bias in indices: an index of companies that met listing requirements throughout a period is not the same as the investable universe at the start of it.
• Backfill bias: strategies join a database with their history attached, and they join after the history was good.
• The question that catches all three: "what happened to everything that started alongside this and is not in the chart?"

Question 5 in practice: how much evidence is in a track record

• Statistically distinguishing genuine skill from luck needs far more data than intuition suggests. With an information ratio of 0.5, roughly sixteen years of returns are required before the result clears a conventional two-standard-error threshold — and that is in sample.
• The information-ratio calculator computes the t-statistic and the years required directly. Most track records people rely on are far shorter than the number it returns.
• Multiple testing makes it worse. Examine a thousand strategies and a handful will look excellent by chance alone. Nothing about the winner's statistics reveals which case it is.
• Three years of good performance is close to no evidence. This is not scepticism; it is the arithmetic of small samples, and it applies equally to results you like and results you do not.

The specific numbers that mislead most reliably

• A back-test. It was constructed by someone who already knew what happened. The only honest use is to check whether an independently motivated idea would have been catastrophic — never to establish that it works.
• An annualised figure from a short period. Annualising three months of a good run produces a number with no meaning and considerable persuasive power.
• A cumulative return chart. It compresses the drawdown that would have ended most people's participation. Discrete calendar years show what actually happened; see the factsheet playbook.
• A correlation from a calm sample. The number is accurate and useless, because it describes the regime in which you did not need it.
• A rating or a risk class. Both are ordinal buckets, not measurements. A class 2 product can lose a third of its value; the risk-class calculator shows the volatility band each bucket actually represents.
• Any statistic quoted without its denominator. "Doubled" is meaningless without knowing from what and over how long.

The habit worth building

• Ask what the number would look like if the claim were false. If the answer is "about the same", the number carries no information regardless of how impressive it is.
• Prefer measures with fewer choices in them. A tracking difference over a fixed period has almost no discretion; an attribution analysis has a great deal.
• Treat costs as the most reliable statistic available. They are stable, forward-looking, and among the few figures whose predictive value is not in dispute.
• Then check the arithmetic yourself. That is what every calculator on this site is for — not to produce a valuation, but to reproduce a claimed number and see whether it survives.