Known at this point
A large volcanic perturbation occurred.
Not yet known
The full temperature response.
From Pattern
to Trust
How a model earns bounded confidence by surviving questions it was allowed to fail.
A model can reproduce the past and still fail when the world changes. The harder question is not whether a model looks convincing, but what evidence its success actually supports, and where that support should end.
A revealing test contains possible failure modes before the result is known.
The model could have failed inamplitudetimingdurationspatial patternphysical sequence
Read the Pinatubo story
In June 1991, Mount Pinatubo erupted in the Philippines and sent volcanic aerosols into the stratosphere. The eruption abruptly changed a physically identifiable part of the climate system by altering the movement of sunlight and heat radiation through the atmosphere.
Before the full surface-temperature response was known, a global climate model was used to state what should follow: temporary global cooling, strongest during 1992, followed by recovery as the aerosol burden declined.
Later observations broadly supported the global cooling response while also revealing regional patterns that the model did not reproduce adequately. The value of the test was therefore not one successful number. The model had been given an opportunity to fail in amplitude, timing, duration, spatial pattern, and physical sequence.
A revealing test keeps both success and failure visible.
Watch first · short video
Why Simulations Must Respect Physics
See why reproducing patterns is not enough, and why physical tests place boundaries around model trust.
Watch video ↗Applied Mathematics in Geosciences · Episode 12
Related: Why Science Uses Methods It Doesn’t Fully Understand ↗Then explore it yourself ↓
What exactly has succeeded?
One correct prediction can hide several different scientific questions. Before judging a model, first identify which question its result answers.
What tends to follow what?
A predictive relationship can be useful even before its physical role is fully understood.
What changes after a specified change?
State what is changed, what is compared, what is held fixed, and what may adjust.
How does the change travel through the system?
Test intermediate variables, timing, spatial pathways, feedbacks, and background-state dependence.
Trust is not a fourth scientific result of the same kind. It is a bounded judgment about whether the available evidence is sufficient for a particular use.
Setup
The same statement can appear to support several kinds of claim.
Question
Is it prediction, response, mechanism, or trust for a stated use?
Try
Classify each statement; feedback appears immediately.
Which question is being asked?
“Weaker trade winds are often followed by warmer eastern-Pacific SST.”
“This model is trustworthy for ENSO.”
Not supported — target and domain are unspecified.“This model is supported for eastern-Pacific SST prediction over the tested forcing range at three-month lead time.”
Supported more precisely by the available evidence.What do you want the model to do?
CURRENT PURPOSEPredict eastern-Pacific SST three months ahead.
The hidden ocean behind a prediction
Suppose weaker-than-normal trade winds are often followed by warmer eastern-Pacific SST. The relationship may be useful, but observed wind can carry two kinds of information at once.
Wind stress changes currents, thermocline depth, upwelling, and the redistribution of ocean heat. Changing wind can help produce later SST change.
Wind may also reveal a subsurface heat reservoir already developing. Hidden heat can influence both the observed wind and later SST.
The wind can alter the ocean, and it can reveal what the ocean was already doing.
Setup
Observed wind can act as both a lever and a clue about hidden ocean heat.
Question
Does observing a wind anomaly answer the same question as setting it?
Try
Switch modes, then hold wind fixed while changing the hidden heat state.
Interactive ocean cross-section
Observe or set the wind
Schematic teaching ocean · not a calibrated ENSO forecast
Predicted later SST=wind pathway+information about hidden ocean heat
Open the mathematical structure
H = hidden subsurface-heat anomaly · W = wind anomaly · T = later SST anomaly
H = εH
W = aH + εW
T = bW + cH + εTObserved wind:
E[T | W = w] = bw + c E[H | W = w]
Set wind:
E[T | W set to w] = bwThe observed relationship contains both the wind pathway and information carried by wind about hidden heat. Setting wind asks a different question.
The predictive question asks: “What usually follows when this wind pattern is observed?”
The response question asks: “What follows when the wind is changed from a specified ocean state, with specified feedbacks?”
Both questions are meaningful. They require different evidence.Do not test only the endpoint
A mechanism is valuable not because it sounds physical, but because it creates additional expectations. Every intermediate step gives evidence another opportunity to reveal that the explanation is incomplete.
Setup
Several pathways can end at a similar SST value.
Question
Did the response travel with the right sequence, location, sign, and time scale?
Try
Choose a pathway variant and advance one step at a time.
1Wind weakens→
2Thermocline deepens in the east→
3Subsurface heat moves east→
4Upwelling cools less→
5Eastern SST warms
SequenceThe ordering follows a plausible wind–thermocline–upwelling–SST sequence.
A model can reach approximately the right final value while responding too early, too late, in the wrong place, or through the wrong pathway.
Testing intermediate variables helps determine whether the model is likely to remain credible when forcing, background state, or intended use changes.
Similar history does not determine future response
Two models can reproduce the same familiar record while containing different response strengths and adjustment time scales. Build trust one test at a time.
Setup
Two models fit the same familiar history but retain different response assumptions.
Question
Which tests separate response strength, adjustment time, pathway, and domain?
Try
Move through Fit → Change → Observe → Boundary → Break.
Fit the familiar range
Model A and Model B nearly overlap across the familiar historical interval.
What has the historical fit failed to distinguish?response amplitudeadjustment timeinternal pathway
Show model
dx/dt = −ax + bu
Response strength: how far the system eventually moves. Adjustment time: how quickly it responds.
Trust comes from tests with different weaknesses
Different tests target different ways in which a model claim can fail. Their value comes not from their number alone, but from their ability to expose different errors.
Setup
Every test has a different failure mode and degree of independence.
Question
Which combination supports the stated use without double-counting evidence?
Try
Inspect the existing evidence nodes and their immediate interpretation.
Bounded trustEvidence is narrow
Do predictions continue to work on outcomes unavailable during model construction?
This line of evidence has not yet been tested.No single line of evidence proves a model true. Partly independent tests constrain different weaknesses.
Complete the sentence
Asking whether a model is trustworthy is too broad. The question becomes meaningful only after its target, range, scale, and purpose are stated.
Setup
A model name alone cannot carry a scientific trust claim.
Question
What target, range, scale, and use are supported?
Try
Build a bounded sentence from the available evidence.
This model is supported foroveratfor
This model is supported for eastern-Pacific SST over the tested forcing range, at three-month lead time, for short-range prediction. Its behavior outside the familiar range has not yet been established.
The qualifications do not weaken the claim. They identify what the evidence actually supports.
From pattern to responsible confidence
A predictive pattern is not yet a response claim. An observed variable may influence the future and also reveal a hidden state.
A mechanism earns weight by exposing a pathway to failure. Sequence, location, sign, timing, intermediate variables, and state dependence all create tests.
A strong test must be informative and relevant. It should use evidence whose role is separated as far as possible from model construction and should probe the conditions in which the model will actually be used.
Trust is bounded and built from converging evidence. It belongs to a target, range, scale, and purpose. It must change when new successes or failures arrive.
A model earns trust by surviving questions it was allowed to fail.
Mathematics cannot remove uncertainty or provide a view from outside the planet. It can turn partial representations into explicit claims, design tests that distinguish them, and make confidence rise or fall with the evidence. That is how a pattern becomes something we can responsibly trust.
Sources and further reading
- J. Hansen, A. Lacis, R. Ruedy, and M. Sato, “Potential climate impact of Mount Pinatubo eruption,” Geophysical Research Letters, 1992.
- D. E. Parker, H. Wilson, P. D. Jones, J. R. Christy, and C. K. Folland, “The impact of Mount Pinatubo on world-wide temperatures,” International Journal of Climatology, 1996.
- National Academies of Sciences, Engineering, and Medicine, Attribution of Extreme Weather Events in the Context of Climate Change, 2016.
- R. Knutti, “Should we believe model predictions of future climate change?” Philosophical Transactions of the Royal Society A, 2008.
- R. Knutti, “The end of model democracy?” Climatic Change, 2010.
- N. Oreskes, K. Shrader-Frechette, and K. Belitz, “Verification, validation, and confirmation of numerical models in the Earth sciences,” Science, 1994.