A · Entropy across four futures
H(Z) = −∑z p(z) log2p(z)
Entropy = 2.00 bits
Module II · Final exploration
Exploration 8 follows uncertainty from its source, through a model pathway, to a named scientific target.
Module II · Exploration 8
How Uncertainty Travels
Uncertainty is a journey, not an error bar
An uncertain input does not simply carry the same error bar into the future. Dynamics can amplify it, damp it, reshape it, combine it with new uncertainty, or turn it into uncertainty about a different question.
Follow the uncertainty ↓Sourceuncertain present state↓Dynamicsmodel carries an ensemble forward↓Targetfuture distribution
source → dynamical pathway → target consequence
Watch first · short video
From Chaos to Probability
Start with the move from uncertain trajectories to distributions that can travel through a dynamical system.
Watch video ↗Applied Mathematics in Geosciences · Episode 9
Then explore it yourself ↓
How does uncertainty move through a linear model?
Begin with one uncertain number. Watch old uncertainty propagate, then allow new uncertainty to enter during the forecast.
TargetxT at T = 10
QuestionWhat happens to uncertainty before it reaches xT?ChangeP0, a, Q, and TWatchWhether the ensemble spreads or contracts
xk+1 = a xk + wk, wk ∼ 𝒩(0,Q)
xkcurrent stateadynamical persistencewknew uncertaintyQforcing variance
Ensemble through time
Uncertainty at the target
Mean-0.03Variance0.07Std. dev.0.27P(xT>0)48%
Inherited from x₀0.055
Injected by forcing0.000
Pk+1 = a2Pk + Q
PT = a2TP0 + Q ∑j=0T−1a2j
ConclusionUncertainty has dynamics. The same initial uncertainty can become smaller or larger depending on the model pathway through which it travels.
What if the model parameter is uncertain?
Give each ensemble member a slightly different ai. The rule that propagates every later uncertainty now varies from member to member. Parameter uncertainty is therefore different from adding another random kick.
Parameter uncertainty asks which value belongs inside one model form. Structural uncertainty asks whether the model form itself is adequate.
The same ensemble can become something variance cannot describe
A two-regime model shows how nonlinear dynamics can bend or split a target distribution and make uncertainty sources interact.
TargetP(xT > 0)
Regime A0 · threshold / regime boundaryRegime B
xk+1 = xk + Δt(r xk − xk3) + wk
Ensemble through time
Distribution shape and regime probability
Mean-0.91Variance0.08Std. dev.0.29P(xT>0)2%
Variance0.08Probability of Regime B2%Shapemostly one regime
Initial-only crossings 1%
Forcing-only crossings 1%
Both together 2%
ConclusionVariance measures spread around the mean. It does not tell us whether probability lies in one broad regime or is split between distinct futures. Near a threshold, one source can change the effect of another.
same variance ⇏ same uncertainty | nonlinearity can destroy simple additivity
Uncertainty is not only variance
Variance measures numerical spread. Information theory asks how broadly probability is distributed and how much one quantity tells us about another.
B · Gaussian variance and entropy
Variance 0.64 · differential entropy 1.20
h(Z) = ½ log(2πeσ2)
Within a one-dimensional Gaussian family, variance and entropy give the same ordering. This equivalence is special.
What does an observation tell us about the target?
We measure the present state imperfectly and ask how much that measurement tells us about the future target Z = xT.
TargetxT at T = 8
Present-state observation
Y = x0 + η, η ∼ 𝒩(0,R)
Before measuring Y, several target futures are plausible. After learning Y, some uncertainty may disappear.
Future target
Before observation 0.36After observation 0.26
Variance reduction 28% · Mutual information 0.17
Show mathematical statement
Var(Z) − 𝔼[Var(Z | Y)] ≥ 0
I(Z;Y) = H(Z) − H(Z | Y)
Mutual information asks how much knowing the observation reduces uncertainty about the target, on average.
Add a second observation
Information if used aloneI(Z;Y2) = 0.16Already supplied by Sensor 10.16Additional informationI(Z;Y2 | Y1) = 0.00
Prior p(Z)After Sensor 1After Sensors 1 + 2
ConclusionAn observation is valuable when it remains connected to the chosen target. A second sensor is valuable only to the extent that it adds information beyond what is already known.
Dependence can be nonlinear—and a modeled pathway can be false
Information measures can see beyond linear correlation, but they still inherit the probability model used to calculate them.
Correlation 0.82 · Mutual information 0.56
Correlation detects linear association and retains its sign. Mutual information detects broader dependence but has no sign. Neither establishes causation.What if the model invents an information pathway?
What the model predictsModel-predicted information: high
What synthetic truth doesTrue relationship: weak
An observation can appear highly informative because the model supplies the pathway connecting it to the target. If that pathway is false, conditioning can produce greater confidence and a worse answer.
predicted information value ≠ guaranteed real-world value
In a seasonal forecast, x0 might represent uncertainty in today’s ocean state, wk future weather forcing, an uncertain parameter coupling strength, and structural uncertainty a missing or incorrect feedback. This one-dimensional model does not reproduce ENSO; it isolates the mathematics needed to reason about how uncertainties reach a target.
Five equations connect the journey
Each card points back to one visual experiment rather than introducing a separate formalism.
xk+1=a xk+wk
Pk+1=a2Pk+Q
Existing uncertainty is propagated; new uncertainty is injected.
Var(Z)=cTPc
Uncertainty is target-dependent.
Z(i)=f(Θ(i))
Ensemble propagation follows nonlinear transformations and multiple regimes.
H(Z)=−∑p(z)log p(z)
I(Z;Y)=H(Z)−H(Z|Y)
I(Z;Y2|Y1)=H(Z|Y1)−H(Z|Y1,Y2)
The second observation is valued after the first is known.
Four ideas
The goal is not to make every distribution narrow. It is to trace uncertainty and decide which conclusions deserve trust.
Begin with the target
There is no model-wide scalar called ‘the uncertainty.’ It belongs to a variable, event, lead time, and question.
Uncertainty has dynamics
Initial uncertainty can be amplified or forgotten, while new uncertainty enters during evolution.
Nonlinearity changes more than variance
A distribution can bend, skew, or split between regimes.
Observation value is conditional and model-dependent
A measurement matters only when it adds credible information about the chosen target beyond what is known.
Uncertainty moves
Uncertainty is not a static error bar. It begins in particular sources, travels through particular dynamical pathways, changes shape, and eventually reaches a target.
Trace uncertainty from source to pathway to target.
Then ask which evidence actually changes what can be known about that target.What this experiment leaves out
The lab uses deliberately small one-dimensional linear and nonlinear models. They isolate the mathematics of uncertainty propagation and information; they are not quantitative Earth-system forecasts.
- The variance decomposition assumes independent inputs in the linear example and is explanatory, not a unique sensitivity decomposition.
- Observation value is conditional on the model pathways. A misspecified model can make an observation look more useful than it is in the real system.
- Histogram entropy, mutual information, and KL divergence depend on binning and finite-ensemble sampling.
Sources, method, and synthetic-model note
- R. C. Smith, Uncertainty Quantification: Theory, Implementation, and Applications, 2014.
- A. Saltelli and coauthors, Global Sensitivity Analysis: The Primer, 2008.
- T. M. Cover and J. A. Thomas, Elements of Information Theory, second edition, 2006.
- O. P. Le Maître and O. M. Knio, Spectral Methods for Uncertainty Quantification, 2010.
All ensembles and observations are reproducible synthetic teaching data generated in the browser with controlled deterministic seeds. Information values are illustrative model-based calculations.