Banach spaces control convergence. Hilbert spaces add geometry: angles, orthogonality, projection, coordinates and best approximation in infinite dimensions.
Once an inner product is available, many finite-dimensional ideas survive in a precise infinite-dimensional form. Closed subspaces have nearest points. Continuous linear functionals can be represented by vectors. Orthonormal families produce Fourier coefficients. Least-squares approximation becomes an orthogonal projection theorem rather than a collection of matrix tricks.
This guide owns R27.02. It develops Hilbert-space structure and prepares the operator and harmonic-analysis routes in R27.03 and R27.04.
Prerequisites: Normed and Banach Spaces, linear algebra and integration.
Reading route: inner products → induced norms → Cauchy–Schwarz → orthogonality → Hilbert completeness → closed subspaces → projection theorem → orthogonal complements → Gram–Schmidt → orthonormal systems → Bessel and Parseval → completeness of bases → Riesz representation → least squares → weak convergence → practice and solutions.
1. An inner product measures both length and angle
On a real or complex vector space H, an inner product ⟨x,y⟩ is linear in one argument, conjugate-linear in the other, conjugate symmetric, and positive definite.
It induces the norm
||x||=√⟨x,x⟩.
Not every norm comes from an inner product.
2. Cauchy–Schwarz controls inner products by norms
For every x,y in an inner-product space,
|⟨x,y⟩|≤||x||||y||.
Equality holds exactly when x and y are linearly dependent, with the usual zero-vector caveat.
Cauchy–Schwarz immediately gives the triangle inequality for the induced norm.
3. The parallelogram identity detects inner-product norms
Every inner-product norm satisfies
||x+y||²+||x−y||²=2||x||²+2||y||².
Conversely, a norm satisfying the parallelogram identity arises from an inner product through a polarization identity.
This gives a structural test: ℓ² is Hilbert; ℓ¹ and ℓ∞ are Banach but their usual norms are not Hilbert norms.
4. Orthogonality means inner product zero
Vectors x and y are orthogonal, written x⊥y, when
⟨x,y⟩=0.
Then the Pythagorean identity holds:
||x+y||²=||x||²+||y||².
This is the geometric engine behind least squares and Fourier expansion.
5. A Hilbert space is a complete inner-product space
A Hilbert space is an inner-product space complete under its induced norm.
Completeness ensures that orthogonal approximation processes and infinite expansions converge to elements still inside the space.
6. Euclidean spaces are the finite-dimensional model
Rⁿ with ⟨x,y⟩=Σx_jy_j and Cⁿ with ⟨x,y⟩=Σx_j overline{y_j} are Hilbert spaces.
Every finite-dimensional inner-product space is automatically complete.
The infinite-dimensional theory preserves much of this geometry, but basis expansions and compactness require extra care.
7. ℓ² is the canonical sequence Hilbert space
Define
ℓ²={x=(x_n):Σ|x_n|²<∞}
with inner product
⟨x,y⟩=Σx_n overline{y_n}.
Cauchy–Schwarz guarantees convergence of the series defining the inner product, and ℓ² is complete.
8. L² is the canonical function Hilbert space
On a measure space, define
⟨f,g⟩=∫ f overline{g}.
Then ||f||₂²=∫|f|².
Functions equal almost everywhere represent the same L² element. Riesz–Fischer completeness makes L² a Hilbert space.
9. Closed subspaces of Hilbert spaces are Hilbert spaces
If M⊂H is a closed linear subspace and H is Hilbert, then M is complete in the inherited norm.
Closedness is essential because a nonclosed subspace may contain Cauchy sequences whose limits lie in H but outside M.
10. Orthogonal complements capture what a subspace cannot represent
For a subset M⊂H, define
M⊥={x∈H:⟨x,m⟩=0 for all m∈M}.
M⊥ is always a closed linear subspace.
If M is a closed subspace, then
H=M⊕M⊥.
11. The Projection Theorem gives the unique nearest point
Let M be a closed subspace of Hilbert space H. For every x∈H there exists a unique m∈M such that
||x−m||=inf_{y∈M}||x−y||.
Moreover, x−m∈M⊥.
The vector m is the orthogonal projection P_Mx.
12. Orthogonality characterises best approximation
For m∈M, m=P_Mx exactly when
⟨x−m,y⟩=0 for every y∈M.
This converts a minimisation problem into a system of orthogonality equations.
13. Worked projection onto one vector
If u≠0 and M=span{u}, then
P_Mx=⟨x,u⟩/⟨u,u⟩ · u
with the usual convention adjusted to the chosen complex-inner-product linearity.
The residual x−P_Mx is orthogonal to u.
14. Projection operators are bounded and idempotent
For a closed subspace M, the orthogonal projection P satisfies
- P²=P;
- Range(P)=M;
- Ker(P)=M⊥;
- ||P||=1 when M is nonzero;
- P is self-adjoint.
Not every bounded projection is orthogonal; oblique projections can have norm greater than 1.
15. Gram–Schmidt creates orthonormal vectors from independent ones
Given linearly independent v₁,v₂,…, define recursively
w_k=v_k−Σ_{j<k}⟨v_k,e_j⟩e_j, e_k=w_k/||w_k||.
The resulting e_j are orthonormal and span the same successive finite-dimensional subspaces.
In infinite dimensions, completeness of the resulting orthonormal system is a separate question.
16. Orthonormal systems give Fourier coefficients
For an orthonormal family {e_n}, the coefficient of x in direction e_n is
c_n=⟨x,e_n⟩.
The finite partial sum Σ_{n=1}^N c_ne_n is the orthogonal projection of x onto span{e₁,…,e_N}.
17. Bessel’s inequality controls coefficient energy
For any orthonormal sequence {e_n},
Σ|⟨x,e_n⟩|²≤||x||².
The proof is the Pythagorean theorem applied to projection residuals.
Bessel says coefficient energy cannot exceed the energy of x.
18. Parseval equality characterises complete orthonormal systems
If {e_n} is an orthonormal basis in the Hilbert-space sense, then
||x||²=Σ|⟨x,e_n⟩|²
and
x=Σ⟨x,e_n⟩e_n
with convergence in Hilbert-space norm.
This is infinite-dimensional Pythagoras.
19. Hilbert-space basis does not mean Hamel basis
An orthonormal basis usually means a complete orthonormal system whose closed linear span is H.
Vectors are represented by convergent infinite series, not finite combinations.
A Hamel basis requires finite combinations and is typically enormous in infinite-dimensional Banach spaces.
20. Separability gives countable orthonormal bases
A Hilbert space is separable if it contains a countable dense subset.
Every separable Hilbert space has a countable orthonormal basis.
Every infinite-dimensional separable Hilbert space is isometrically isomorphic to ℓ² after choosing such a basis.
21. Riesz Representation turns functionals into vectors
If H is Hilbert and φ∈H*, then there exists a unique y∈H such that
φ(x)=⟨x,y⟩
for every x, with conjugation convention adjusted to the chosen inner-product linearity.
Moreover ||φ||=||y||.
This theorem identifies the continuous dual of a Hilbert space with the space itself, anti-linearly or linearly depending on convention.
22. Least squares is orthogonal projection onto a model subspace
Suppose observations form b in a Hilbert space and admissible model outputs form a closed subspace M.
The best least-squares approximation is P_Mb.
The residual b−P_Mb is orthogonal to every admissible model direction.
Finite-dimensional normal equations are simply this orthogonality condition written in coordinates.
23. Worked least-squares geometry
Let A:R^n→R^m and seek x minimising ||Ax−b||₂.
The vector Ax lies in Range(A). Best approximation requires residual b−Ax orthogonal to Range(A), so
Aᵀ(b−Ax)=0.
Thus the normal equations are
AᵀAx=Aᵀb.
Uniqueness of x requires additional rank conditions even though the best projected vector Ax is unique.
24. Orthogonal decomposition solves constrained minimisation cleanly
If H=M⊕M⊥ and x=m+n, then for any y∈M,
||x−y||²=||m−y||²+||n||².
The minimum occurs uniquely at y=m.
This proof is both geometric and exact; no coordinate differentiation is needed.
25. Weak convergence tests vectors through inner products
A sequence x_n converges weakly to x in H if
⟨x_n,y⟩→⟨x,y⟩ for every y∈H.
Norm convergence implies weak convergence, but not conversely in infinite dimensions.
For example, the standard unit vectors e_n in ℓ² converge weakly to 0 but have ||e_n||=1 for all n.
26. Weak convergence is useful because bounded sets behave better
Hilbert spaces are reflexive, and bounded sequences admit weakly convergent subsequences under standard sequential compactness results in separable/reflexive settings.
This weaker topology is a central existence tool in variational methods and PDEs.
27. Adjoint operators are defined by the inner product
If T:H→H is bounded, there exists a unique bounded operator T* satisfying
⟨Tx,y⟩=⟨x,T*y⟩.
Self-adjoint, unitary and normal operators are defined through this relation. R27.03 develops their spectra and spectral theorems.
28. Geometry can fail outside Hilbert norms
Banach spaces such as ℓ¹ or L¹ lack a compatible inner product producing their usual norms.
Nearest-point projections need not be linear or unique in general Banach spaces. Riesz representation takes a different form or may fail in the Hilbert-space sense.
Hilbert-space geometry is powerful precisely because it is special.
29. Common theorem-boundary errors
- Every Hilbert space is Banach, but not every Banach space is Hilbert.
- A norm comes from an inner product only when it satisfies the parallelogram identity.
- Projection onto a subspace requires the subspace to be closed for the standard Hilbert projection theorem.
- An orthonormal system need not be complete.
- Bessel inequality becomes Parseval equality only for a complete orthonormal basis.
- Weak convergence does not imply norm convergence.
- The least-squares fitted vector can be unique even when coefficient vectors are not.
30. A dependable Hilbert-space workflow
State the inner product and verify that the intended norm is the induced norm. Check completeness before calling the space Hilbert.
For approximation, identify the closed model subspace and impose orthogonality of the residual. For expansions, verify orthonormality and then separately verify completeness.
For duality questions, invoke Riesz only in a genuine Hilbert setting. For weak limits, distinguish scalar coefficient convergence from norm convergence.
31. Independent practice: twenty questions
- State the inner-product axioms.
- How does an inner product induce a norm?
- State Cauchy–Schwarz.
- State the parallelogram identity.
- Define a Hilbert space.
- Why is ℓ² a Hilbert space?
- Write the L² inner product.
- Define M⊥.
- State the Hilbert Projection Theorem for a closed subspace.
- What condition characterises P_Mx?
- Project x onto span{u}.
- List two algebraic properties of an orthogonal projection.
- What does Gram–Schmidt produce?
- Define Fourier coefficients relative to an orthonormal family.
- State Bessel’s inequality.
- State Parseval’s identity.
- What does Riesz Representation say?
- How do normal equations arise from projection?
- Give a weakly convergent but not norm-convergent sequence in ℓ².
- Define the adjoint of a bounded operator.
32. Worked solutions and checks
1. Sesquilinearity/linearity convention, conjugate symmetry and positive definiteness.
2. ||x||=√⟨x,x⟩.
3. |⟨x,y⟩|≤||x||||y||.
4. ||x+y||²+||x−y||²=2||x||²+2||y||².
5. A complete inner-product space.
6. Its standard inner product is well-defined by Cauchy–Schwarz and the induced norm is complete.
7. ⟨f,g⟩=∫f overline{g}.
8. Vectors orthogonal to every element of M.
9. Every x has a unique nearest point m∈M, with x−m∈M⊥.
10. x−P_Mx is orthogonal to M.
11. P(x)=⟨x,u⟩u/⟨u,u⟩.
12. Examples: P²=P, Range(P)=M, Ker(P)=M⊥, P*=P.
13. An orthonormal family spanning the same successive finite-dimensional subspaces.
14. c_n=⟨x,e_n⟩.
15. Σ|⟨x,e_n⟩|²≤||x||².
16. For a complete orthonormal basis, ||x||²=Σ|⟨x,e_n⟩|².
17. Every bounded linear functional on H is inner product with a unique vector.
18. The least-squares residual must be orthogonal to Range(A), giving Aᵀ(Ax−b)=0.
19. The standard unit vectors e_n converge weakly to 0 but retain norm 1.
20. T* is uniquely defined by ⟨Tx,y⟩=⟨x,T*y⟩.
33. Where this guide hands off
R27.03 studies bounded and selected unbounded operators as mathematical objects: resolvents, spectra, compact operators, self-adjoint and normal structure, and spectral theorems that generalise diagonalisation beyond finite matrices.
Sources and further study
See MIT OpenCourseWare 18.102 Introduction to Functional Analysis for Hilbert spaces, orthonormal bases, Riesz representation and operator theory.