How Kalman-Filter Dynamic Nelson–Siegel Algorithms Track Yield-Curve Factors: State-Space Loadings, Prediction–Update Recursions, Missing Maturities, Likelihood Calibration and No-Arbitrage Limits

Reader question: A static Nelson–Siegel curve can fit today’s yields, but how can an algorithm track the hidden level, slope and curvature factors through time when market yields are noisy, maturities are occasionally missing and the factors themselves evolve?

A Kalman-filter Dynamic Nelson–Siegel (DNS) model turns the yield curve into a linear state-space system once the decay parameter is fixed. The cross-section of yields becomes the observation equation; the evolving level/slope/curvature factors become the hidden state. At every date the algorithm predicts the factors, compares predicted yields with observed yields, measures the innovation, then updates the factor estimate according to the relative uncertainty of the state forecast and the observed data.

This article owns one exact computational job: state-space filtering, smoothing, forecasting and likelihood estimation of dynamic Nelson–Siegel yield-curve factors. It does not own static Nelson–Siegel/Svensson curve fitting, arbitrage-free affine term-structure modelling, HJM dynamics, Hull–White calibration, or interpolation between individual curve knots.

This is public mathematical and computational education. It is not a bond-trading recommendation, an interest-rate forecast for personal decision-making, or a guarantee that a statistically well-filtered yield curve is arbitrage-free.

1. The static Nelson–Siegel measurement shape

For maturity τ, the Diebold–Li form of the Nelson–Siegel yield curve is:

y_t(τ) = β_1,t + β_2,t[(1−e^−λτ)/(λτ)] + β_3,t{[(1−e^−λτ)/(λτ)] − e^−λτ} + ε_t(τ).

The three loadings have intuitive shapes:

• level: loading 1 at every maturity;
• slope: starts near 1 at short maturity and decays toward 0;
• curvature: starts near 0, rises to a hump, then falls toward 0.

With λ fixed, the loadings are known numbers for each maturity. The unknown quantities at each date are the three factors.

2. Why the factors become a state vector

Define:

f_t = [β_1,t, β_2,t, β_3,t]^T.

If yields are observed at maturities τ₁,…,τ_m, define the loading matrix Λ(λ) with one row per maturity.

The observation equation is then:

y_t = Λ f_t + ε_t,

with:

ε_t ~ N(0,H).

H is the measurement-error covariance matrix. A diagonal H says yield-fitting errors at different maturities are conditionally independent; a full H allows cross-maturity measurement-error dependence.

3. The factor dynamics are the state equation

A common dynamic specification is a VAR(1):

f_t − μ = A(f_t−1 − μ) + η_t,

where:

η_t ~ N(0,Q).

μ is the long-run factor mean, A controls persistence and cross-factor dynamics, and Q is process-noise covariance.

If A is diagonal, each factor follows its own AR(1). A full A allows, for example, yesterday’s slope to help predict tomorrow’s curvature.

4. Prediction step

Suppose the filtered factor estimate after date t−1 is:

f̂_t−1|t−1

with covariance:

P_t−1|t−1.

The one-step state prediction is:

f̂_t|t−1 = μ + A(f̂_t−1|t−1 − μ).

The predicted state covariance is:

P_t|t−1 = A P_t−1|t−1A^T + Q.

Q prevents the factor uncertainty from collapsing to zero even when the model has observed many historical curves.

5. Predict the whole yield curve

Given the predicted factors:

ŷ_t|t−1 = Λ f̂_t|t−1.

The innovation is:

v_t = y_t − ŷ_t|t−1.

This is the vector of yield surprises across maturities.

The innovation covariance is:

F_t = ΛP_t|t−1Λ^T + H.

F combines uncertainty about the latent factors with measurement noise in the observed yields.

6. The Kalman gain chooses how much to trust the new curve

The Kalman gain is:

K_t = P_t|t−1Λ^TF_t⁻¹.

The updated state is:

f̂_t|t = f̂_t|t−1 + K_tv_t.

If measurement noise H is large, the algorithm moves the factors only modestly toward the new yields. If prior state uncertainty P is large and yields are measured precisely, the new curve receives more weight.

This is why the Kalman filter is not simply a moving average: its update is uncertainty-weighted and maturity-aware.

7. Update the state uncertainty

A compact covariance update is:

P_t|t = (I − K_tΛ)P_t|t−1.

Numerically cautious implementations may use the Joseph form:

P_t|t = (I−KΛ)P(I−KΛ)^T + KHK^T,

which better preserves symmetry and positive semidefiniteness under floating-point error.

8. A numerical intuition

Suppose the model predicts a 10-year yield of 4.00%, but the observed yield is 4.08%.

If the model believes the 10-year measurement is very precise and factor uncertainty is substantial, the innovation receives a large gain and the level factor may rise materially.

If the observed curve is known to be noisy because of sparse trading, the same 8-basis-point difference may produce only a small factor update.

The innovation is not automatically “truth”; it is evidence whose weight depends on F.

9. Why λ matters so much

λ controls the decay speed of the slope and curvature loadings.

Diebold and Li famously fix λ at 0.0609 when maturity is measured in months, which places the curvature loading’s maximum near 30 months in their application.

Fixing λ has an important computational effect: the observation equation becomes linear in the three factors.

If λ is estimated jointly with everything else, the likelihood becomes more nonlinear and can have local optima or weak-identification regions.

Diagnostic: profile the likelihood over λ rather than trusting one nonlinear optimizer run.

10. Static Nelson–Siegel and Dynamic Nelson–Siegel are different owners

Nelson–Siegel–Svensson yield-curve algorithms own the cross-sectional fitting problem at one date.

Dynamic Nelson–Siegel with a Kalman filter owns the time-series problem:

how do latent curve factors evolve, and how should today’s noisy curve update yesterday’s factor beliefs?

11. Maximum likelihood comes almost for free from the innovations

For a linear Gaussian state-space model, the Kalman filter produces the prediction-error likelihood.

At date t, with m observed yields:

ℓ_t = −½[m log(2π) + log|F_t| + v_t^TF_t⁻¹v_t].

The total log-likelihood is:

ℓ = Σ_t ℓ_t.

Parameters such as μ, A, Q, H and sometimes λ can therefore be calibrated by maximizing the same likelihood used to run the filter.

12. Why the likelihood can be deceptive

A high likelihood does not guarantee:

• economic stability;
• good multi-step forecasts;
• no arbitrage;
• well-identified Q versus H;
• reasonable factor persistence.

For example, a model may explain nearly all variation as measurement noise H and very little as process noise Q, or vice versa, while producing similar fitted yields.

Falsifier: inspect factor paths, innovation autocorrelation, Q/H sensitivity and out-of-sample forecast errors, not just the optimized likelihood.

13. Missing maturities are naturally handled

Suppose a 20-year yield is missing on one date but the 2-, 5-, 10- and 30-year yields are present.

The filter does not need to manufacture the missing observation.

Use only the rows of y_t, Λ and H corresponding to observed maturities when forming:

v_t, F_t, K_t.

The factor state still predicts the missing maturity through:

ŷ_missing,t = λ_missing^T f̂_t|t.

This is one major advantage of the state-space formulation over ad hoc interpolation of missing time-series points.

14. Filtering versus smoothing

Filtering estimates f_t using information available up to date t.

Smoothing estimates f_t using the full sample, including later dates.

A Rauch–Tung–Striebel smoother runs backward after the forward Kalman pass.

Smoothed factors are useful for historical analysis and parameter estimation, but they must not be presented as real-time estimates.

Falsifier: for forecast backtests, confirm that only filtered—not smoothed—states feed the forecast origin.

15. Multi-step forecasts

For h steps ahead:

f̂_t+h|t = μ + A^h(f̂_t|t − μ).

The predicted yield curve is:

ŷ_t+h|t = Λf̂_t+h|t.

The forecast covariance grows as future process shocks accumulate.

A usable engine should output both point forecasts and uncertainty, because a forecasted 10-year yield of 4.2% with a 5-basis-point standard deviation is fundamentally different from the same point forecast with 80-basis-point uncertainty.

16. Persistence and stationarity diagnostics

If the eigenvalues of A lie inside the unit circle, the factor VAR is stationary around μ.

Estimated level factors are often highly persistent. An eigenvalue extremely close to one can make long-horizon forecasts sensitive to tiny parameter changes.

Diagnostic: report eigenvalues of A, half-lives where meaningful, and long-horizon factor forecasts.

Falsifier: if small re-estimation windows flip a factor root from 0.98 to 1.02, the stationarity conclusion is fragile.

17. Innovation diagnostics

Under the model, standardized innovations:

z_t = F_t^−1/2v_t

should be approximately:

• mean zero;
• serially uncorrelated;
• cross-sectionally consistent with identity covariance after standardization;
• Gaussian if the Gaussian likelihood is taken literally.

Persistent autocorrelation means factor dynamics are missing information. Heavy tails can indicate outliers, crisis regimes or measurement-error misspecification.

18. Evidence polarity

Evidence for confidence includes:

• small, structureless yield innovations;
• innovation covariance close to model expectations;
• stable factor loadings and λ;
• factor paths with plausible level/slope/curvature interpretation;
• good out-of-sample yield forecasts versus random-walk/AR benchmarks;
• robust estimates across reasonable initial states;
• missing-maturity predictions that reconcile when observations return;
• likelihood and forecast conclusions stable across nearby specifications.

Evidence against confidence includes:

• serially correlated innovations;
• large maturity-specific residual bias;
• extreme sensitivity to λ;
• Q or H collapsing toward numerical boundaries;
• factor transition roots near or outside one without economic justification;
• forecast performance worse than simple benchmarks;
• smoothed factors accidentally used in real-time evaluation;
• high likelihood but implausible yield dynamics.

19. Counterexample: measurement noise is set too small

Suppose H is nearly zero.

The filter treats every observed yield wiggle as genuine movement in the latent factors.

Factors become excessively jagged and Q may be estimated too large.

Falsifier: inspect innovation variance and factor turnover. If tiny market microstructure changes produce large factor jumps, H is probably understated.

20. Counterexample: process noise is set too small

If Q is nearly zero, the model assumes factors move almost deterministically through A.

A genuine monetary-policy shock then appears as a giant measurement error rather than a state innovation.

Falsifier: compare shock dates with standardized innovations. Persistent large curve-wide residuals indicate that factor evolution is too rigid.

21. Counterexample: a structural break

A single stationary VAR may not describe both low-volatility and crisis periods.

A regime shift can change factor means, persistence and shock covariance.

Falsifier: rolling innovation diagnostics, parameter stability tests and forecast-break analysis.

Possible alternatives include Markov-switching DNS, time-varying parameter models or nonlinear filters, but extra complexity must earn its place out of sample.

22. Counterexample: λ is weakly identified

When maturities do not cover the region where the curvature loading bends strongly, many λ values can produce similar fitted curves.

The optimizer may report a precise-looking optimum that is actually a shallow likelihood ridge.

Falsifier: plot profile likelihood and fitted loading functions across a λ grid. If broad λ ranges have near-identical likelihood, report identification weakness instead of one spurious decimal estimate.

23. Counterexample: missing data are not random

A maturity may disappear precisely because the instrument is illiquid or stressed.

Kalman filtering can handle a missing observation mechanically, but it does not make the missingness mechanism harmless.

Falsifier: compare residuals and liquidity indicators before/after missing episodes. If missingness is systematically associated with stressed pricing, the measurement model needs an explicit liquidity/error treatment.

24. DNS does not impose no-arbitrage

The empirical Dynamic Nelson–Siegel model is prized for parsimonious fit and forecasting, but the standard DNS dynamics do not by themselves impose no-arbitrage restrictions across time.

Christensen, Diebold and Rudebusch developed the arbitrage-free Nelson–Siegel (AFNS) class to reconcile the Nelson–Siegel loading structure with affine no-arbitrage term-structure restrictions.

Boundary: a Kalman-filter DNS forecast can be statistically coherent while still not being a risk-neutral pricing model for derivatives.

25. Relation to Hull–White

Hull–White calibration algorithms own a different job: fit a short-rate dynamics model used for derivative pricing, usually under risk-neutral assumptions.

DNS Kalman filtering is primarily a statistical factor/filtering framework for the observed term structure.

26. Relation to Hagan–West interpolation

Hagan–West monotone-convex algorithms answer the within-date interpolation question: how should forward rates behave between observed curve knots?

DNS answers a lower-dimensional factor question across maturities and time.

27. Relation to Smith–Wilson

Smith–Wilson algorithms own long-end extrapolation toward an ultimate forward rate.

DNS can extrapolate because its factor loadings have limiting behaviour, but it does not implement the same regulatory/economic long-end constraint.

28. Relation to particle filters

Particle-filter algorithms are useful when the state-space system is nonlinear or non-Gaussian enough that the Kalman recursion is no longer exact.

The Kalman filter remains the preferred baseline when the model is genuinely linear Gaussian because it is exact, fast and diagnostically transparent.

29. Inputs and outputs

Inputs can include:

• yield observations by date and maturity;
• maturity vector τ;
• fixed or calibrated λ;
• initial factor mean/covariance;
• state transition A and mean μ;
• process covariance Q;
• measurement covariance H;
• missing-observation mask;
• forecast horizon;
• optimizer and parameter transforms for likelihood calibration.

Outputs can include:

• filtered level/slope/curvature factors;
• smoothed historical factors;
• fitted yields;
• innovations and standardized innovations;
• state covariance matrices;
• log-likelihood;
• multi-step yield forecasts;
• forecast uncertainty;
• missing-maturity estimates;
• parameter-stability and residual diagnostics.

30. Weak links

• λ units inconsistent with maturity units;
• poor state initialization;
• Q/H confounding;
• non-positive-semidefinite covariance matrices from numerical error;
• measurement-error independence assumed without evidence;
• factor VAR underfits breaks/regimes;
• optimizer stops at local likelihood optimum;
• smoothed state leakage into forecasts;
• missingness correlated with liquidity stress;
• DNS used for derivative pricing as though no-arbitrage were imposed.

31. What would falsify confidence?

Confidence should be withdrawn if innovations remain serially structured; if λ and covariance estimates are unstable across nearby samples; if simple random-walk or AR yield forecasts outperform the model; if missing-maturity estimates fail when observations return; if factor roots are implausibly unstable; or if the intended downstream job requires no-arbitrage pricing restrictions that ordinary DNS does not provide.

32. Verification and update triggers

Preserve the exact maturity grid, units, loading convention, λ, parameter transforms, A/Q/H matrices, initial-state rule, optimizer, likelihood values, innovation diagnostics and backtest windows.

Revalidate when:

• the maturity set changes;
• curve data source/conventions change;
• liquidity or measurement-error behaviour changes;
• monetary regime shifts materially;
• factor persistence changes;
• forecast residuals become autocorrelated;
• new no-arbitrage pricing requirements arise;
• the filtering engine changes numerical linear-algebra routines.

33. Primary and high-quality references

• Francis X. Diebold and Canlin Li, Forecasting the Term Structure of Government Bond Yields, Wharton/Rodney L. White Center working paper; published in Journal of Econometrics, 2006.
• Francis X. Diebold and Glenn D. Rudebusch, Dynamic Nelson–Siegel, in Yield Curve Modeling and Forecasting, Princeton University Press/Oxford Academic.
• Jens H. E. Christensen, Francis X. Diebold and Glenn D. Rudebusch, The Affine Arbitrage-Free Class of Nelson–Siegel Term Structure Models, NBER Working Paper 13611; later Journal of Econometrics.
• James Durbin and Siem Jan Koopman, Time Series Analysis by State Space Methods, Oxford University Press.
• Federal Reserve Board, Nominal Yield Curve, current public methodology/data page for daily estimated zero-coupon curves and Svensson parameters.

Educational boundary: Kalman filtering makes the dynamic factor-estimation problem explicit and testable. It does not convert empirical factor dynamics into a guaranteed arbitrage-free pricing model.