How Christoffersen VaR Backtesting Algorithms Detect Bad Coverage and Exception Clustering: Likelihood Ratios, Markov Hits, Conditional Coverage, Small-Sample Failure and Diagnostics

Reader question: A Value-at-Risk model can produce roughly the right number of breaches over a year and still fail exactly when markets become volatile. How do we test both the frequency and the time pattern of VaR exceptions?

Christoffersen-style backtesting turns the daily hit sequence into two statistical questions. First: is the unconditional exception rate correct? Second: are exceptions independent through time, or do they cluster? The combined conditional-coverage test asks whether both properties hold at once.

This article owns one narrow computational job: likelihood-ratio backtesting of VaR exception sequences for unconditional coverage, first-order independence and joint conditional coverage. It does not own VaR forecasting itself, Expected Shortfall modelling, CAViaR estimation, Basel traffic-light capital rules or general stress testing.

This is public mathematical and computational education. It is not financial advice, a regulatory approval opinion or a guarantee that a model passing a backtest is safe.

1. Turn forecasts into a binary hit sequence

Let L_t be the realised loss and VaR_t the one-step-ahead loss threshold at confidence level 1−α.

Define:

I_t = 1{L_t > VaR_t}.

Under a correctly specified VaR model, the hit probability should be:

P(I_t=1 | information available at t−1) = α.

For a 99% VaR, α = 0.01.

The word conditional matters. A model should not merely average 1% exceptions over a long sample; after conditioning on yesterday’s information, today’s breach probability should still be 1%.

2. Two models can have the same hit count but different quality

Consider 250 daily forecasts with four exceptions.

Model A has hits on days 20, 81, 146 and 220.

Model B has hits on days 100, 101, 102 and 103.

Both have the same unconditional exception rate:

4/250 = 1.6%.

But Model B failed in a cluster. That pattern is exactly what we expect when a risk model is slow to adapt after volatility jumps.

A hit-count-only test cannot distinguish the two sequences. Christoffersen’s independence test can.

3. Unconditional coverage

Let T be the sample size and x the total number of exceptions.

Under the model null:

x ~ Binomial(T, α)

if hits are independent.

The unrestricted maximum-likelihood exception probability is:

π̂ = x/T.

The restricted likelihood is:

L₀ = (1−α)^T−x α^x.

The unrestricted likelihood is:

L₁ = (1−π̂)^T−x π̂^x.

The likelihood-ratio statistic is:

LR_uc = −2 log(L₀/L₁).

Asymptotically, under standard conditions:

LR_uc ~ χ²(1).

This is the familiar Kupiec/POF-style unconditional coverage component.

4. Correct coverage is two-sided

Too many exceptions mean VaR is too small or too slow to react.

Too few exceptions can also signal misspecification: VaR may be excessively conservative, the forecast horizon may be misaligned, or the backtest may contain stale positions or a sign error.

Therefore the null is not “few breaches are good.” It is:

exception probability equals α.

5. First-order independence as a Markov-chain test

Christoffersen’s independence test uses adjacent hit pairs.

Count transitions:

n₀₀: no hit followed by no hit;
n₀₁: no hit followed by hit;
n₁₀: hit followed by no hit;
n₁₁: hit followed by hit.

Under first-order dependence, estimate:

π̂₀₁ = n₀₁/(n₀₀+n₀₁)

and:

π̂₁₁ = n₁₁/(n₁₀+n₁₁).

If hits are independent, the probability of a hit tomorrow should not depend on whether today was a hit:

π₀₁ = π₁₁.

6. Independence likelihood ratio

Under the independence restriction, estimate one transition-to-hit probability:

π̂ = (n₀₁+n₁₁)/(n₀₀+n₀₁+n₁₀+n₁₁).

The restricted likelihood is:

L_ind,0 = (1−π̂)^n00+n10 π̂^n01+n11.

The unrestricted Markov likelihood is:

L_ind,1 = (1−π̂₀₁)ⁿ⁰⁰ π̂₀₁ⁿ⁰¹ (1−π̂₁₁)ⁿ¹⁰ π̂₁₁ⁿ¹¹.

Then:

LR_ind = −2 log(L_ind,0/L_ind,1)

is asymptotically χ²(1).

7. Conditional coverage combines the two failures

The joint statistic is:

LR_cc = LR_uc + LR_ind.

Under the joint null of correct unconditional coverage and first-order independence:

LR_cc ~ χ²(2)

asymptotically.

This decomposition is valuable diagnostically:

high LR_uc, low LR_ind → wrong average coverage;
low LR_uc, high LR_ind → right count, wrong timing;
both high → both calibration and dynamics are suspect.

8. Why clustered exceptions matter

Suppose a volatility model underestimates a sudden regime shift.

After the first breach, tomorrow’s VaR remains too low. A second breach becomes more likely. The hit process then displays:

P(I_t=1 | I_t−1=1) > P(I_t=1 | I_t−1=0).

The first-order Markov test is not a complete volatility diagnostic, but it detects a simple and important form of dynamic misspecification.

9. Inputs and outputs

Inputs can include:

timestamped ex-ante VaR forecasts;
corresponding realised P&L or loss;
target exception probability α;
forecast horizon;
VaR sign convention;
P&L definition (actual or hypothetical);
missing-day/calendar policy;
finite-sample or asymptotic inference setting.

Outputs can include:

hit sequence;
exception count and observed rate;
n₀₀, n₀₁, n₁₀, n₁₁;
LR_uc and p-value;
LR_ind and p-value;
LR_cc and p-value;
exception dates and sizes;
clustering plots;
rolling-window diagnostics;
small-sample warning flags.

10. A minimal algorithm

Align each VaR forecast only with the loss realised over its stated horizon.
Resolve sign convention before constructing hits.
Create I_t.
Count x and compute LR_uc.
Construct adjacent hit pairs and transition counts.
Compute LR_ind.
Add them for LR_cc.
Compute asymptotic or exact/bootstrapped p-values as appropriate.
Inspect breach dates and sizes rather than reporting p-values alone.
Run alternative diagnostics if the failure mode is not first-order Markov clustering.

11. Zero transition counts are a numerical edge case

At 99% VaR with only 250 observations, n₁₁ is often zero.

Terms such as:

0 × log(0)

should be evaluated by their limiting value zero.

Code that computes log(0) first can generate NaNs even though the likelihood contribution is well-defined.

Falsifier: test synthetic sequences containing no adjacent exceptions, all exceptions, one exception and alternating hits.

12. Asymptotic χ² inference can be poor in small samples

The likelihood-ratio reference distributions are asymptotic.

At low exception probabilities, samples can contain only a handful of hits. Transition counts are then extremely discrete and some cells are empty.

Recent exact finite-sample implementations show that the usual χ² approximation can have material size distortion for extreme coverage levels and small samples.

Diagnostic: where the decision is sensitive, compare asymptotic p-values with exact dynamic-programming, Monte Carlo or carefully justified bootstrap inference.

13. One percent VaR creates a power problem

At α = 0.01 and T = 250, the expected hit count is only:

Tα = 2.5.

There is little information with which to distinguish:

a true 1% model;
a mildly misspecified 1.5% model;
a model with occasional clustering.

Failing to reject the null may therefore mean “insufficient evidence,” not “validated with high precision.”

14. The independence test only targets one form of dependence

The first-order Markov alternative asks whether yesterday’s hit changes today’s hit probability.

A model can still fail through:

clusters separated by two or three days;
seasonal exception risk;
dependence on volatility state rather than yesterday’s hit;
long-memory clustering.

Passing LR_ind does not prove the entire hit sequence is iid.

15. Duration tests use more temporal information

Christoffersen and Pelletier proposed testing the waiting times between breaches.

If hits are iid Bernoulli with probability α, no-hit durations have a memoryless structure. Duration-based methods can detect dependence patterns that a first-order transition table misses and can have better power in realistic small samples.

Alternative: use duration tests when the central concern is clustering over variable-length intervals.

16. Dynamic Quantile tests ask a richer conditional question

CAViaR and Dynamic Quantile tests examine whether hit residuals are predictable from lagged hits, VaR forecasts and other information.

That is a broader conditional-calibration test than the two-state first-order Markov alternative.

Christoffersen remains valuable because it is transparent, compact and easy to diagnose.

17. Exception sizes are ignored by the binary test

A loss exceeding VaR by $1 and a loss exceeding VaR by $100 both create:

I_t = 1.

Coverage tests therefore assess quantile calibration, not tail severity beyond the threshold.

Falsifier: if a model passes coverage tests but exceptions are routinely enormous, tail risk remains under-described. Add Expected Shortfall and tail-loss diagnostics.

18. VaR backtesting is not Expected Shortfall backtesting

Modern market-risk frameworks use Expected Shortfall for important capital calculations, but daily VaR-style backtesting remains useful because a quantile produces an observable breach indicator.

Expected Shortfall asks about the mean loss beyond a tail threshold and requires different statistical tools.

Do not interpret LR_cc as a test of ES accuracy.

19. Current Basel context

The Basel Committee’s current market-risk backtesting framework continues to use a 99% one-day VaR comparison for backtesting. For 250 observations, its traffic-light-style zones are:

green: 0–4 exceptions;
amber: 5–9;
red: 10 or more.

Those zone counts are supervisory rules based on binomial probabilities. They are not the same algorithm as Christoffersen conditional coverage.

A model can sit in a favourable count zone while still showing statistically suspicious clustering.

20. Evidence polarity

Evidence for confidence includes:

observed hit rate near α over sufficiently long samples;
LR_uc does not reject;
π̂₀₁ and π̂₁₁ are similar;
LR_ind does not reject;
LR_cc does not reject;
results are stable across reasonable inference methods;
rolling windows do not reveal persistent local failure;
exception sizes and stress periods have plausible explanations.

Evidence against confidence includes:

too many or too few hits;
π̂₁₁ materially above π̂₀₁;
repeated post-shock clusters;
results change dramatically under exact versus asymptotic inference;
breaches coincide with stale volatility estimates;
backtest alignment differs from forecast horizon;
passing full-sample tests but failing repeatedly in rolling windows.

21. Counterexample: correct count, concentrated crisis failure

A 1% VaR model generates exactly ten breaches over 1,000 days, but seven occur in one 10-day stress window.

Unconditional coverage looks excellent.

Falsifier: inspect n₁₁, LR_ind, duration distributions and the timeline. Correct count cannot rescue dynamic failure.

22. Counterexample: zero breaches

Zero exceptions may look reassuring.

But for a 95% VaR over 1,000 observations, zero breaches is wildly inconsistent with the target 5% exception rate.

Falsifier: coverage tests are two-sided with respect to calibration. Excessive conservatism is also a forecast error.

23. Counterexample: sign convention is reversed

If VaR is stored as a positive loss magnitude but code tests:

return < VaR

instead of comparing loss to the threshold, almost every observation can be labelled a breach.

Falsifier: hand-check several days around the threshold before running any statistic.

24. Counterexample: overlapping multi-day VaR

A 10-day VaR forecast backtested every day with overlapping 10-day realised losses creates mechanical dependence between adjacent outcomes.

The iid-hit null is then inappropriate without adjustment.

Falsifier: use non-overlapping evaluation windows or an inference method that explicitly handles overlap.

25. Counterexample: reusing today’s loss to build today’s forecast

If the model is recalibrated using information from the same day whose loss it predicts, the hit sequence contains look-ahead leakage.

Falsifier: reconstruct every forecast using only the information that genuinely existed before the forecast horizon began.

26. Counterexample: p-value worship

Suppose LR_cc has p = 0.08 at a 5% threshold.

“Pass” is too strong. The evidence is simply insufficient to reject at 5%.

Falsifier: report effect sizes—hit rate, transition probabilities, exception clusters and sample size—alongside p-values.

27. Rolling diagnostics

A model can pass over ten years because good periods dilute a recent breakdown.

Track:

rolling exception rate;
rolling LR statistics;
time since last hit;
n₁₁ concentration;
exception size;
forecast volatility versus realised volatility.

This does not replace formal inference; it localises when and how the model fails.

28. Connections to surrounding BTT knowledge

CAViaR algorithms own direct dynamic quantile forecasting and DQ tests.

FRTB market-risk algorithms own the wider regulatory capital and backtesting architecture.

GARCH/EWMA volatility algorithms explain a common source of dynamic VaR forecasts whose exception clustering can then be tested here.

30. What would falsify confidence?

Confidence should be withdrawn if hit frequency is persistently inconsistent with α; if exceptions cluster; if exact finite-sample inference contradicts a marginal asymptotic pass; if rolling windows repeatedly fail; if breach sizes reveal severe tail underestimation; or if the backtest cannot be reproduced from archived ex-ante forecasts and realised P&L.

31. Verification and update triggers

Preserve the forecast timestamp, VaR horizon/confidence, P&L definition, sign convention, hit sequence, transition table, LR formulas, p-value method and all exception dates.

Revalidate when:

VaR model changes;
volatility estimator changes;
portfolio composition changes materially;
forecast horizon changes;
P&L construction changes;
market regime changes;
exception clustering appears;
the regulatory/backtesting use case changes.

32. Primary and high-quality references

Peter F. Christoffersen, Evaluating Interval Forecasts, International Economic Review, 1998.
Peter Christoffersen and Denis Pelletier, Backtesting Value-at-Risk: A Duration-Based Approach, Journal of Financial Econometrics, 2004.
Basel Committee on Banking Supervision, Internal Models Approach: Backtesting and P&L Attribution Test Requirements, current Basel Framework page.
Basel Committee on Banking Supervision, Guidance on Use of the Internal Models Approach, including 250-observation backtesting zones.
Engle and Manganelli’s Dynamic Quantile framework is covered in the linked CAViaR article as a complementary conditional-calibration approach.

Educational boundary: A Christoffersen pass means the available hit sequence did not provide enough evidence against the tested coverage/first-order-independence null. It does not prove model truth, tail-severity accuracy, future robustness or regulatory acceptability.

Small Group Tutorials