Reader question: Many financial algorithms repeatedly apply the same mapping until nothing changes. If ordinary fixed-point iteration converges slowly—or oscillates—can we use the history of previous residuals to predict a better next iterate without forming a full Jacobian?

Anderson acceleration does exactly that. Instead of accepting the next Picard/fixed-point iterate G(x_k) blindly, it stores a short history of recent iterates and residuals, solves a small least-squares problem, and mixes those historical steps to reduce the current residual.

This article owns one precise computational job: accelerating an already-defined fixed-point iteration x = G(x) by multisecant-style residual mixing. It does not own the underlying financial model, the transformation that created the fixed-point map, general nonlinear root solving, quasi-Newton Jacobian updates, or constrained optimisation itself.

The neighbouring BTT owners remain separate. Broyden quasi-Newton algorithms update an approximate Jacobian/inverse Jacobian for roots; ADMM owns operator splitting for constrained portfolio problems; Sinkhorn owns entropic matrix scaling. Anderson acceleration can sometimes speed such iterative maps, but it does not replace their mathematical objectives.

This is public mathematical and computational education. It is not financial advice and not a guarantee that acceleration makes an unstable model or incorrect fixed point reliable.

1. Start with a fixed-point problem

Suppose the target x* satisfies:

x* = G(x*).

Ordinary Picard iteration is:

x_k+1 = G(x_k).

Define the fixed-point residual:

r_k = G(x_k) − x_k.

At a true fixed point:

r(x*) = 0.

Therefore solving x = G(x) is equivalent to solving the root problem:

r(x) = 0.

2. Why plain fixed-point iteration can be slow

If G is locally contractive with Lipschitz factor ρ < 1, Banach’s fixed-point theorem gives local/global convergence under the theorem’s conditions.

But when ρ is close to 1:

||x_k+1−x*|| ≲ ρ ||x_k−x*||

shrinks slowly.

For example, ρ = 0.98 leaves about 36% of an error after 50 iterations because 0.98⁵⁰ ≈ 0.364.

The map is convergent, but the iteration wastes information by using only the most recent point.

3. Anderson’s idea: use residual history

Choose a memory depth m.

At iteration k, let:

m_k = min(m,k).

Keep the recent residuals:

r_{k−m_k}, …, r_k.

Find coefficients α that make a linear combination of those residuals as small as possible:

min_α ||Σ α_i r_i||₂

subject to:

Σ α_i = 1.

Then mix the mapped iterates:

x_k+1^AA = Σ α_i G(x_i).

This is the clean constrained-mixing view of Anderson acceleration.

4. Why the sum-to-one constraint matters

If every historical point equals the same fixed point x*, then every G(x_i) = x*.

The condition:

Σ α_i = 1

ensures their mixture also equals x*.

The affine constraint preserves fixed points while allowing extrapolation through positive and negative weights.

5. Anderson is not simple averaging

A moving average might set α_i = 1/(m+1).

Anderson chooses α by residual minimisation.

If two recent residuals point in nearly opposite directions, the least-squares problem can combine them to cancel much of the error.

If residuals all point in the same direction, acceleration may offer little gain.

6. Difference-matrix formulation

Many implementations use residual and iterate differences rather than the constrained α formulation.

Define:

Δx_j = x_j+1 − x_j,

Δr_j = r_j+1 − r_j.

Build matrices from the latest m differences:

X_k = [Δx_k−m,…,Δx_k−1],

R_k = [Δr_k−m,…,Δr_k−1].

A Type-II style least-squares step solves:

γ_k = argmin_γ ||r_k − R_kγ||₂.

The coefficient vector then corrects the unaccelerated step using the historical iterate/residual differences.

Equivalent formulations differ by sign and residual convention, so implementation documentation must specify whether r = G(x)−x or x−G(x).

7. Multisecant interpretation

Near a solution:

r(x+Δx) ≈ r(x) + J_rΔx.

The stored pairs (Δx, Δr) contain approximate secant information about the residual Jacobian.

Anderson uses several recent secant directions simultaneously.

This is why it is often described as a multisecant or quasi-Newton-like acceleration even though the algorithm can be implemented without explicitly storing a full Jacobian matrix.

8. Linear problems connect Anderson to GMRES

For a linear fixed-point map, Walker and Ni showed a close/essential equivalence between Anderson acceleration and GMRES before stagnation under the corresponding formulation.

This gives useful intuition:

Anderson builds a nonlinear residual-minimising subspace from recent iteration history.

But the nonlinear case is not simply GMRES with different notation. Convergence depends on local map behaviour and the conditioning of the mixing problem.

9. A simple scalar example

Suppose:

x = cos(x).

Picard iteration:

x_k+1 = cos(x_k)

converges to approximately 0.739085, but only linearly.

With one or two historical residuals, Anderson acceleration can extrapolate toward the intersection faster.

This toy example is not financial. Its role is to isolate the numerical mechanism before using the same solver machinery inside a financial fixed-point map.

10. Where fixed points appear in finance

Financial computation frequently contains recursive maps even when the original problem is written differently.

Examples can include:

risk-parity or equilibrium equations rearranged into fixed-point form;
iterative collateral/funding/valuation equations;
alternating calibration maps;
matrix scaling and scenario-reweighting procedures;
splitting/proximal optimisation iterations;
recursive dynamic-programming or policy-evaluation maps.

Anderson should be attached only after the underlying map G has a clear mathematical owner and a verifiable residual.

11. The first diagnostic is the unaccelerated map

Before acceleration, test plain iteration.

Record:

residual norm ||r_k||;
step norm ||x_k+1−x_k||;
objective/error if available;
constraint violations;
estimated local contraction ratio.

If plain iteration converges smoothly but slowly, Anderson has a clear baseline to improve.

If plain iteration is chaotic or converges to the wrong object, acceleration may make failure faster rather than fix it.

12. Memory depth m

m = 0 recovers ordinary fixed-point iteration.

Larger m uses more history.

Potential benefits:

richer residual subspace;
better cancellation of slow modes;
fewer expensive evaluations of G.

Potential costs:

more memory;
larger least-squares solve;
older, less locally relevant information;
worse conditioning when residual differences are nearly dependent.

More memory is not automatically better.

13. Conditioning of the least-squares problem

If columns of R_k are nearly linearly dependent, the coefficient problem becomes ill-conditioned.

Then small floating-point perturbations can produce very large γ or α coefficients.

Large coefficients create aggressive extrapolation far outside the region represented by recent iterates.

Diagnostic: monitor singular values or the condition number of the history matrix and the norm of the mixing coefficients.

14. QR or SVD instead of normal equations

The least-squares problem should generally be solved using a numerically stable factorisation such as QR or SVD.

Forming normal equations:

R^TR γ = R^Tr

squares the condition number approximately and can amplify instability.

SVD is especially useful when residual-history columns are nearly rank deficient because small singular directions can be diagnosed or truncated explicitly.

15. Regularisation

A Tikhonov-regularised coefficient solve can use:

min ||r_k−R_kγ||² + λ||γ||².

Regularisation reduces extreme coefficients when history is ill-conditioned.

It also weakens the exact residual-minimisation step.

Falsifier: sweep λ. If convergence depends on a very narrow regularisation value, the underlying map/history may be unstable.

16. Damping or relaxation

A damped accelerated update can blend the Anderson proposal with a safer step.

Schematically:

x_k+1 = (1−β)x_safe + βx_AA,

with 0 < β ≤ 1 for conservative damping.

Some formulations incorporate β directly into the Type-I/Type-II update.

Damping can enlarge the region of practical convergence, but too much damping destroys the acceleration benefit.

17. Safeguards

A production solver should not accept every accelerated proposal unconditionally.

Common safeguards include rejecting the AA step when:

residual norm increases too much;
coefficient norm exceeds a threshold;
least-squares condition number is too large;
constraints become violated;
objective deteriorates materially;
the proposal leaves an admissible parameter region.

On rejection, fall back to the ordinary fixed-point step and possibly reset history.

18. Restarting history

Old residual directions can become misleading after:

a regime/parameter update;
a nonlinear region change;
a clipping/projection event;
near-stagnation;
a rejected accelerated step.

Restarting clears the stored residual subspace and rebuilds local information.

Diagnostic: record restart frequency. Constant restarts indicate that the chosen depth or map may not support stable acceleration.

19. Stagnation can break the history solve

If:

x_k+1 ≈ x_k

while residual remains nonzero, then Δx and Δr can approach zero.

The history matrix loses rank.

Potra and Engler show that on linear problems Anderson’s behaviour is closely linked to GMRES before stagnation, and stagnation can lead to convergence to an incorrect solution under certain formulations.

Falsifier: never use “small step” alone as a stopping rule. Require a small residual in the original fixed-point equation.

20. Residual norm is the primary stopping condition

Useful criteria include:

||G(x_k)−x_k|| ≤ atol + rtol·scale.

Also verify:

financial constraints;
pricing/calibration residuals;
objective consistency;
independent root checks where practical.

An accelerated solver that takes tiny steps can still be far from the true fixed point.

21. Anderson can improve linear convergence

Modern convergence analysis provides theoretical support that Anderson acceleration can improve the first-order rate of linearly convergent fixed-point iterations under assumptions including local contractivity and controlled coefficients.

Evans, Pollock, Rebholz and Xiao show a gain mechanism tied to the residual-minimisation stage.

The theorem is not a licence to ignore coefficient growth or noncontractive regions.

22. Anderson may not help a quadratically convergent method

If a Newton method is already in its quadratic convergence region, the error can shrink roughly like:

e_k+1 ≈ C e_k².

Adding Anderson mixing can interfere with that fast local structure.

Research shows acceleration is most naturally useful for linearly convergent fixed-point maps, not as a universal wrapper around every solver.

23. Anderson versus Broyden

Broyden updates an explicit approximate Jacobian or inverse Jacobian satisfying a secant condition.

Anderson typically stores a limited recent history and solves a residual mixing problem.

Both use secant information, and the mathematical relationship is close, but their state, update algebra and operational diagnostics differ.

If a full/structured Jacobian approximation is useful elsewhere, Broyden may be the better owner. If only a black-box map G is available, Anderson can be attractive.

24. Anderson versus Newton

Newton solves:

J_r(x_k) Δx = −r(x_k).

It requires a Jacobian or Jacobian-vector machinery.

Anderson is derivative-free in its basic implementation and learns local directions from residual history.

Newton can be much faster near a well-conditioned root; Anderson can be easier to attach to an existing expensive fixed-point map.

25. Anderson and ADMM

ADMM generates a fixed-point-like operator on primal/dual variables.

Research exists on Anderson-accelerated ADMM, but the safeguard must respect ADMM’s primal and dual residual structure.

An accelerated iterate with a smaller generic Euclidean residual can still worsen feasibility.

Falsifier: monitor the original ADMM primal/dual residuals, not only the Anderson residual.

26. Anderson and Sinkhorn

Sinkhorn scaling alternates row and column normalisations to a fixed point.

Acceleration techniques can reduce slow late-stage convergence in some matrix-scaling regimes.

But positivity and marginal constraints are essential. Any accelerated formulation must preserve or safely project back to the feasible structure.

27. Noisy mappings

Suppose G(x) itself is estimated by Monte Carlo.

Then:

Ĝ(x) = G(x) + ε.

Residual differences contain simulation noise.

Anderson can fit that noise with large coefficients, especially near convergence when the true residual is small.

Falsifier: replicate the same x with independent random seeds. If residual variability is comparable to the convergence tolerance, deterministic acceleration assumptions are not adequate.

28. Common-random-number stabilisation

For Monte Carlo fixed-point maps, one possible variance-control tactic is to use common random numbers across nearby iterates so that differences in G(x) reflect x changes more than random-number changes.

But common random numbers create dependence and do not remove simulation bias.

Use them as a controlled numerical device and validate with independent-seed final checks.

29. Projections and clipping create nonsmooth maps

A financial iteration may project weights to bounds or clip probabilities/parameters to admissible ranges.

This makes G piecewise smooth or nonsmooth.

Anderson can still work empirically, but smooth local convergence theory may not apply.

Falsifier: record active-set changes and compare with unaccelerated/proximal solvers. Frequent projection switching can destabilise residual extrapolation.

30. Inputs and outputs

Inputs can include:

fixed-point map G;
initial state x₀;
memory depth m;
damping/relaxation β;
regularisation λ;
residual convention;
least-squares solver;
safeguard thresholds;
restart policy;
absolute/relative tolerance;
maximum iterations.

Outputs can include:

final fixed point;
residual norm path;
iteration count;
map-evaluation count;
mixing coefficients;
history condition numbers/singular values;
restart/rejection counts;
constraint diagnostics;
baseline Picard comparison;
independent verification residual.

31. Evidence polarity

Evidence for confidence includes:

same fixed point as a trusted unaccelerated/root-solver benchmark;
fewer expensive G evaluations at equal tolerance;
residual decreases stably;
mixing coefficients remain controlled;
history matrix is reasonably conditioned;
results are robust to modest depth changes;
safeguard rejects rare pathological steps rather than most steps;
financial constraints/objectives remain satisfied.

Evidence against confidence includes:

residual grows after many AA steps;
coefficient norms explode;
history matrix becomes nearly rank deficient;
different depths converge to different fixed points;
frequent restarts are required;
step norm is small but residual is not;
acceleration worsens a Newton-like quadratic regime;
Monte Carlo noise dominates residual differences.

32. Counterexample: a divergent base map

Let:

G(x) = 1.2x + b.

The linear fixed-point iteration has spectral factor 1.2 and diverges.

Anderson can sometimes stabilise noncontractive maps, but there is no general guarantee that it will rescue this iteration.

Falsifier: do not treat temporary residual decrease as proof. Compare with a proper root solver and test robustness to initial conditions.

33. Counterexample: nearly identical residual history

If r_k−2, r_k−1, r_k are almost collinear, the least-squares problem has little independent directional information.

Large positive/negative coefficients can cancel the residual numerically while producing an extreme iterate.

Falsifier: monitor SVD singular values and cap/reject coefficient norms.

34. Counterexample: acceleration reaches a wrong clipped solution

Suppose every iteration clips a model parameter into [0,1].

An extrapolated Anderson proposal lands outside the range and is projected to 1 repeatedly.

Step changes become tiny, but:

G(1)−1 ≠ 0.

Falsifier: residual in the original equation must be small. Boundary sticking is not convergence.

35. Counterexample: overlarge memory

Increasing m from 5 to 50 can reduce iteration count but make every least-squares solve expensive and ill-conditioned.

Total wall-clock time rises.

Falsifier: optimise on total map evaluations and runtime, not iteration count alone.

36. Counterexample: noisy Monte Carlo map near tolerance

The true residual is 10⁻⁶, but simulation noise has standard deviation 10⁻⁴.

Anderson is fitting noise.

Falsifier: tighten Monte Carlo precision or stop at a tolerance consistent with statistical error. Numerical convergence below estimator noise has no meaning.

37. Minimal safeguarded algorithm

Compute g_k = G(x_k) and r_k = g_k−x_k.
If ||r_k|| meets tolerance, stop.
Add (x_k,r_k) to a bounded history.
Build residual-difference matrix.
Check rank/condition number.
Solve the small least-squares problem using QR/SVD plus regularisation if required.
Construct the AA proposal.
Apply damping and admissibility checks.
Evaluate/estimate proposal residual or a safeguard proxy.
Accept if sufficiently safe; otherwise use g_k and optionally restart history.
Repeat.

38. Weak links

fixed-point map itself is badly chosen;
residual sign/convention inconsistent in code;
normal equations used on ill-conditioned history;
memory too large;
no regularisation or safeguard;
small step mistaken for convergence;
constraints ignored;
Monte Carlo noise fitted as curvature;
old history retained across model/regime changes;
acceleration judged by iteration count instead of total cost.

39. What would falsify confidence?

Withdraw confidence if AA converges to a different root than independent solvers; if residuals or financial constraints are worse than the base iteration; if coefficient norms/condition numbers repeatedly explode; if acceleration benefit disappears under modest depth changes; if noisy-map replications produce materially different results; or if a Broyden/Newton method achieves lower total cost with stronger verification.

40. Verification and update triggers

Preserve the exact map G, residual convention, state scaling, depth, damping, regularisation, least-squares method, safeguard/restart rules, tolerance, initial point and complete convergence trace.

Revalidate when:

the underlying financial model changes;
G is reformulated;
state dimension/scaling changes;
constraints/projections change;
Monte Carlo precision changes;
the mapping becomes less contractive;
history conditioning deteriorates;
hardware/solver libraries change numerical behaviour.

41. Primary and high-quality references

Donald G. Anderson, Iterative Procedures for Nonlinear Integral Equations, Journal of the ACM, 1965.
Homer F. Walker and Peng Ni, Anderson Acceleration for Fixed-Point Iterations, SIAM Journal on Numerical Analysis, 2011.
Alex Toth and C. T. Kelley, Convergence Analysis for Anderson Acceleration, SIAM Journal on Numerical Analysis, 2015.
Claire Evans, Sara Pollock, Leo Rebholz and Mengying Xiao, A Proof That Anderson Acceleration Improves the Convergence Rate in Linearly Converging Fixed-Point Methods (But Not in Those Converging Quadratically), SIAM Journal on Numerical Analysis, 2020.
Florian Potra and Hans Engler, work on the behaviour of Anderson acceleration on linear problems.

Educational boundary: Anderson acceleration is a wrapper around a fixed-point map. It can make a good map much faster, but residual-history extrapolation cannot certify that the underlying financial equation, model or fixed point is the right one.

Small Group Tutorials

How Anderson-Acceleration Algorithms Speed Financial Fixed-Point Solvers: Residual Histories, Least-Squares Mixing, Damping, Conditioning, Safeguards and Divergence