Quantum information becomes measurable once we can quantify uncertainty, mixedness and correlation. The central tool is von Neumann entropy, built directly from the eigenvalues of a density matrix.
Batch 02 has progressively widened the mathematical lens. Guide 5 introduced density matrices. Guide 6 introduced channels. Guide 7 introduced generalised measurements and distinguishability. This final guide asks how much uncertainty a state contains, how strongly two systems are correlated and how those quantities behave under quantum processing.
Density matrix → eigenvalues → entropy. Joint state → reduced states → mutual information. Pure bipartite state → reduced entropy → entanglement entropy.
1. From Shannon entropy to von Neumann entropy
For a classical probability distribution {p_i}, Shannon entropy is
H(p)=-Σ_i p_i log₂ p_i.
The quantum analogue replaces the probability vector with a density matrix:
S(ρ)=-Tr(ρ log₂ρ).
If ρ has eigenvalues λ_i, then
S(ρ)=-Σ_i λ_i log₂λ_i.
Thus von Neumann entropy is simply the Shannon entropy of the density matrix’s eigenvalue distribution. We use base-2 logarithms in this guide, so entropy is measured in bits.
2. Pure states have zero entropy
A pure density matrix has spectrum {1,0,0,...}. Therefore
S(ρ)=-(1 log₂1)=0.
The zero eigenvalues contribute zero by continuity. A pure state therefore has zero von Neumann entropy even though measurements in some bases can still produce random outcomes.
This distinction matters. State entropy measures mixedness of the density operator, not the uncertainty of every possible measurement.
3. The maximally mixed state has maximum entropy
For a d-dimensional maximally mixed state
ρ=I/d,
all d eigenvalues equal 1/d. Hence
S(I/d)=log₂d.
For a qubit, the maximum is one bit. For two qubits considered jointly, the four-dimensional maximally mixed state has entropy two bits.
Worked example: a diagonal qubit state
Let ρ=diag(3/4,1/4). Then
S(ρ)=-(3/4)log₂(3/4)-(1/4)log₂(1/4).
This is the binary entropy h₂(1/4), approximately 0.811 bits.
4. Entropy depends only on eigenvalues
Unitary conjugation ρ→UρU† changes the basis representation but not the eigenvalues. Therefore
S(UρU†)=S(ρ).
Closed-system unitary dynamics preserve von Neumann entropy. A pure state remains pure. A mixed state retains the same spectrum even while its matrix entries change.
5. Purity and entropy measure related but different things
Guide 5 introduced purity
γ=Tr(ρ²)=Σ_i λ_i².
Purity is algebraically simpler than von Neumann entropy. Both are functions of the eigenvalues and both detect pure versus mixed states, but they are not the same measure.
- Pure state:
γ=1,S=0. - Maximally mixed d-state:
γ=1/d,S=log₂d.
For higher-dimensional states, two density matrices can have the same purity while having different entropy. One scalar summary does not reconstruct the full spectrum.
6. Rényi entropies connect purity to a larger family
For order α>0, α≠1, the quantum Rényi entropy is
S_α(ρ)=(1/(1-α))log₂Tr(ρ^α).
At α=2,
S₂(ρ)=-log₂Tr(ρ²).
So purity directly determines the second Rényi entropy. As α→1, the Rényi entropy approaches von Neumann entropy under standard conditions.
7. Joint and reduced entropies
For a bipartite state ρ_AB, define
S(A)=S(ρ_A), whereρ_A=Tr_Bρ_AB;S(B)=S(ρ_B), whereρ_B=Tr_Aρ_AB;S(AB)=S(ρ_AB).
These three quantities reveal how uncertainty is distributed between the whole system and its parts.
8. Product states add their entropies
If ρ_AB=ρ_A⊗ρ_B, then
S(AB)=S(A)+S(B).
This mirrors classical independent distributions. There are no A-B correlations in a product state.
9. Subadditivity
For every bipartite quantum state,
S(AB)≤S(A)+S(B).
Equality holds for product states. When the inequality is strict, the deficit reflects correlations between A and B.
10. Araki–Lieb inequality
Quantum entropy also satisfies
|S(A)-S(B)|≤S(AB).
Together with subadditivity, this places the joint entropy between the absolute difference and the sum of the marginal entropies.
11. Pure bipartite states have equal reduced entropies
If ρ_AB=|ψ⟩⟨ψ| is pure, then S(AB)=0. The Schmidt decomposition implies that the non-zero eigenvalues of ρ_A and ρ_B are identical. Therefore
S(A)=S(B).
This shared reduced entropy is the entanglement entropy of the bipartite pure state.
Worked example: Bell-state entropy
For |Φ+⟩=(|00⟩+|11⟩)/√2, the joint state is pure, so S(AB)=0. Each reduced state is I/2, so
S(A)=S(B)=1 bit.
The whole is perfectly known as a pure state while either part alone is maximally uncertain.
In quantum mechanics, uncertainty of the parts can come from entanglement even when the whole is pure.
12. Quantum mutual information
Define the quantum mutual information by
I(A:B)=S(A)+S(B)-S(AB).
It is always non-negative and measures the total correlations—classical and quantum—between A and B.
For a product state, I(A:B)=0. Conversely, zero quantum mutual information occurs exactly for product states.
Bell-state mutual information
For the Bell state, S(A)=1, S(B)=1 and S(AB)=0. Therefore
I(A:B)=2 bits.
This value can exceed either local entropy because quantum mutual information counts the total shared correlation structure between the two subsystems.
13. Classical correlated mixture versus Bell state
Consider the classical-looking correlated state
ρ_class=(1/2)|00⟩⟨00|+(1/2)|11⟩⟨11|.
Its reduced states are again I/2, so S(A)=S(B)=1. But the joint state has two eigenvalues 1/2,1/2, so S(AB)=1. Hence
I(A:B)=1 bit.
The Bell state had mutual information two bits. Thus identical computational-basis correlations do not imply identical total correlation structure.
14. Quantum conditional entropy
Define
S(A|B)=S(AB)-S(B).
Classical conditional entropy is never negative. Quantum conditional entropy can be.
For a Bell state, S(AB)=0 and S(B)=1, giving
S(A|B)=-1.
Negative conditional entropy is a genuinely quantum phenomenon associated with sufficiently strong entanglement and has operational meaning in quantum information protocols. It should not be interpreted as “negative ignorance” in an ordinary classical sense.
15. Relative entropy
The quantum relative entropy of ρ with respect to σ is
D(ρ||σ)=Tr[ρ(log₂ρ-log₂σ)]
when the support of ρ is contained in the support of σ; otherwise it is taken to be infinite.
Relative entropy is not a metric: it is not symmetric and does not satisfy a triangle inequality. But it is a central measure of distinguishability and statistical mismatch.
16. Mutual information as relative entropy
Quantum mutual information can be rewritten as
I(A:B)=D(ρ_AB || ρ_A⊗ρ_B).
This gives a powerful interpretation: mutual information measures how distinguishable the actual joint state is from the product state built from its marginals. If there are no correlations, the two states are identical and the relative entropy is zero.
17. Data-processing inequality
For any quantum channel 𝓔, quantum relative entropy obeys
D(𝓔(ρ)||𝓔(σ))≤D(ρ||σ).
This is another form of the data-processing principle encountered in Guide 7. Physical processing cannot make two candidate states more distinguishable according to this information measure.
Applying local channels also cannot increase quantum mutual information between two parties when the processing discards information rather than adding new shared resources.
18. Strong subadditivity
For a tripartite state ABC, von Neumann entropy satisfies the fundamental inequality
S(ABC)+S(B)≤S(AB)+S(BC).
Equivalent formulations include non-negativity of quantum conditional mutual information:
I(A:C|B)=S(AB)+S(BC)-S(B)-S(ABC)≥0.
Strong subadditivity is one of the central structural results of quantum information theory. It controls how correlations can be distributed across larger systems.
19. Entanglement entropy from Schmidt coefficients
For a bipartite pure state in Schmidt form
|ψ⟩=Σ_i √λ_i |u_i⟩|v_i⟩,
the reduced density matrices have eigenvalues {λ_i}. Therefore the entanglement entropy is
S_ent=-Σ_i λ_i log₂λ_i.
A product pure state has one Schmidt coefficient equal to one and entropy zero. A maximally entangled pair of d-dimensional systems has d equal Schmidt coefficients and entropy log₂d.
20. Worked example: partially entangled two-qubit state
Let
|ψ⟩=√p|00⟩+√(1-p)|11⟩
for 0≤p≤1.
The reduced density matrix is diag(p,1-p), so the entanglement entropy is
S_ent=h₂(p).
At p=0 or 1, the state is a product and entropy is zero. At p=1/2, the state is maximally entangled and entropy is one bit.
21. Entropy is not automatically “disorder”
The word “disorder” is sometimes used informally, but it can obscure the mathematics. Von Neumann entropy is a spectral function of the density operator. Its operational meanings depend on the information-processing task under discussion.
A pure Bell state has zero joint entropy yet highly non-trivial correlations. Each subsystem has maximum entropy. Calling one state “more disordered” without specifying which system and which operational question can therefore become misleading.
22. Measurement entropy is not state entropy
A pure state can produce a random measurement distribution. For example, |+⟩ has zero von Neumann entropy but a Z-basis measurement gives one bit of Shannon entropy because the two outcomes are equally likely.
The state entropy and a chosen measurement’s outcome entropy answer different questions. The first is basis-independent; the second depends on the measurement.
23. Noise, entropy and an important caution
Noise often increases entropy, but not every quantum channel increases entropy for every input. Unital channels cannot decrease von Neumann entropy in finite dimensions, while non-unital channels such as amplitude damping can reduce entropy by driving states toward a pure fixed state.
Therefore “noise always increases entropy” is too broad. The behaviour depends on the channel and the state.
24. Common misconception: high local entropy means the global system is highly mixed
The Bell state is the decisive counterexample. The global state is pure with entropy zero while each local state is maximally mixed with entropy one bit. Local entropy can be generated by entanglement rather than by global classical ignorance.
25. Common misconception: mutual information measures only entanglement
No. Quantum mutual information measures total correlations. A separable classically correlated state can have positive mutual information. Entanglement measures require more specialised definitions.
26. Common misconception: negative conditional entropy means negative ordinary uncertainty
Quantum conditional entropy is defined algebraically as S(AB)-S(B) and can become negative because the joint and marginal entropy structure differs fundamentally from the classical case. Its interpretation belongs to quantum information tasks, not ordinary negative probability or negative ignorance.
27. Worked synthesis problem
Compare two two-qubit states:
ρ_1=|Φ+⟩⟨Φ+|
and
ρ_2=(1/2)|00⟩⟨00|+(1/2)|11⟩⟨11|.
Step 1: Marginals. Both have ρ_A=ρ_B=I/2, so S(A)=S(B)=1.
Step 2: Joint entropy of ρ₁. It is pure, so S(AB)=0.
Step 3: Joint entropy of ρ₂. Its non-zero eigenvalues are 1/2,1/2, so S(AB)=1.
Step 4: Mutual information. For ρ₁, I=2 bits. For ρ₂, I=1 bit.
Step 5: Conditional entropy. For ρ₁, S(A|B)=-1. For ρ₂, S(A|B)=0.
Interpretation. The two states share the same simple computational-basis correlations and identical marginals, yet their entropy structure reveals a fundamental difference between coherent entanglement and classical correlation.
28. Practice set
- Define von Neumann entropy.
- What is the entropy of a pure state?
- What is the entropy of
I/2using base-2 logarithms? - Compute the purity of
I/2. - State subadditivity.
- Define quantum mutual information.
- What is the mutual information of a product state?
- For a pure bipartite state, how are
S(A)andS(B)related? - Define quantum conditional entropy.
- Give one reason quantum conditional entropy can differ qualitatively from classical conditional entropy.
Answers
S(ρ)=-Tr(ρlog₂ρ).- Zero.
- One bit.
1/2.S(AB)≤S(A)+S(B).I(A:B)=S(A)+S(B)-S(AB).- Zero.
- They are equal.
S(A|B)=S(AB)-S(B).- It can be negative for entangled states, which has no direct classical analogue.
29. Batch 02 completes the open-system and information layer
The eight-guide Quantum Mathematics estate now forms two connected stages.
Batch 01 built the state-and-dynamics layer: complex amplitudes, vectors, operators, eigenvalues, tensor products, entanglement and Schrödinger evolution.
Batch 02 built the information-and-open-system layer: density matrices, reduced states, channels, noise, generalised measurements, distinguishability and entropy.
Represent → transform → compose → reduce → process → measure → compare → quantify information.
30. Series navigation
- Guide 1: Complex Numbers, State Vectors and Probability Amplitudes
- Guide 2: Matrices, Operators, Eigenvalues and Measurement
- Guide 3: Tensor Products, Qubits and Entanglement
- Guide 4: Unitary Evolution, Schrödinger’s Equation and Quantum Dynamics
- Guide 5: Density Matrices, Mixed States, Trace and Partial Trace
- Guide 6: Quantum Channels, Kraus Operators, Noise and Decoherence
- Guide 7: Generalised Measurements, POVMs, State Discrimination and Fidelity
- Guide 8: Quantum Entropy, Purity, Mutual Information and Correlations
- Return to the BTT Mathematics Learning Hub
Educational note: entropy and information measures are task-dependent mathematical tools. Their operational meanings become most useful when attached to a clearly specified communication, compression, thermodynamic or estimation problem.