Lab · Foundation 03

Covariance and
Correlation

How can two quantities vary together, and how can we measure that relationship without confusing scale, shape, or cause?

Many scientific questions involve more than one quantity. Temperature at two locations may rise and fall together. Wind and pressure may change in opposite directions. A model variable may appear closely related to an observation.

Covariance and correlation give us a language for describing these relationships. But they do not tell us exactly the same thing, and neither one by itself tells us why a relationship exists.

Start exploring ↓

01 Center the data → 02 Measure joint variation → 03 Normalize and interpret

Moderate positive association

What does it mean to “move together”?

Start with paired observations (x_i, y_i). What matters is where a point lies relative to the average of each variable. Same-sign deviations make a positive centered product; opposite-sign deviations make a negative one.

What you are seeing

Covariance begins by centering both variables. Each observation then contributes according to the product of its two deviations from the mean.

Why it matters

A single point does not determine covariance. It summarizes whether same-sign or opposite-sign deviations dominate across the dataset.

Try this

Move one point far from both means, then move it across a mean line. Watch its magnitude and sign change.

Add the contributions together

Covariance is the average product of two centered variables. It asks whether departures from the two means tend to have the same sign or opposite signs.

Cov(X, Y) = E[(X − μ_X)(Y − μ_Y)]

s_xy = 1/(n−1) Σ(x_i − x̄)(y_i − ȳ)

Positive contributions+ 55.11

Negative contributions− 0.00

Net sum55.11

Sample covariance6.12

Why (n−1)?

Estimating both means from the same sample uses one degree of freedom. Dividing by n−1 gives the usual unbiased sample covariance under standard sampling assumptions.

Positive covariance

Large values tend to occur with large values, and small values with small values.

Negative covariance

Large values tend to occur with small values of the other variable.

Near zero

Positive and negative centered products largely cancel. This does not rule out a nonlinear relationship.

Same relationship, different units

Covariance contains useful information, but its numerical value depends on the units used to measure the variables.

Original variables

s_x = 2.73 · s_y = 2.28 · covariance = 6.12 · r = 0.98

Rescaled X*

s_x* = 2.73 · covariance = 6.12 · r = 0.98

Scale a × 1.0Shift b 0

Cov(aX + b, Y) = a Cov(X, Y) (a > 0)

Changing an offset does not change covariance. Multiplying the scale changes covariance, even though the visible relationship has not fundamentally changed.

Remove the units

Pearson correlation is normalized covariance. It measures the direction and strength of a linear association.

r = s_xy / (s_xs_y) −1 ≤ r ≤ 1

Direction Noise 1.2Sample size 30

A relationship can be strong even when r is small

Pearson correlation looks for a linear pattern. Nature is not required to organize itself along a straight line.

Pearson r = 0.99

Zero correlation is not the same as independence.

A Pearson correlation near zero says there is little linear association. A nonlinear dependence can still be strong.

Correlation can be sensitive to unusual observations

Because covariance and correlation depend on distances from the mean, observations far from the center can contribute strongly.

Moving together does not tell us why

A scatterplot can reveal co-variation. It cannot, by itself, identify the mechanism that produced it.

Z↙↘XYshared driver

Observed X–Y correlation: 0.88

A large correlation is evidence of association, not a complete causal explanation. Direction, common drivers, selection effects, and other mechanisms require additional information or assumptions.

Covariance becomes a matrix

Earth-system models do not contain only two variables. The pairwise covariance idea extends naturally to a covariance matrix.

Illustrative coupled-system data · synthetic

X₁ temperature-like · X₂ wind-like · X₃ ocean-memory-like

Selected entry C₁₂: covariance 2.17

Foundation 04 · Principal Component Analysis
PCA uses covariance structure to identify directions along which a multivariable system varies most strongly. Coming next

Foundation 05 · The One-Dimensional Kalman Update
In data assimilation, covariance describes uncertainty and how information about one variable can update another. Coming next

Why this matters for a planet full of interacting variables

Spatial coherenceMeasurements at nearby locations often vary together. Covariance helps describe how information is shared across space.

Coupled variablesAtmospheric, oceanic, and land variables can co-vary. Correlation is a first description; physical interpretation requires more.

UncertaintyModel errors and observational uncertainty can also co-vary. Covariance is central to uncertainty quantification, data assimilation, and prediction.

In Earth science, covariance is not merely a descriptive statistic. It is also a mathematical representation of how variability and uncertainty are organized across a system.

What should you remember?

Covariance measures joint variationIt combines centered deviations to determine whether two variables tend to depart from their means in the same or opposite directions.
Covariance carries scaleIts magnitude depends on the units of the variables.
Correlation is normalized covariancePearson correlation removes individual scales and lies between −1 and +1.
Correlation describes linear associationA small Pearson correlation does not rule out a strong nonlinear relationship.
Association is not mechanismCorrelation alone cannot establish causality, and individual observations can strongly influence the result.

Covariance tells us how variations are connected. Correlation puts that connection on a common scale. The challenge is not only to calculate them, but to understand what they reveal, and what they do not.

Plain-language glossary

Mean: The average value of a variable.
Deviation: The difference between one observation and the mean.
Variance: The average squared size of deviations from the mean.
Standard deviation: A measure of the typical scale of variation.
Covariance: A signed measure of how two centered variables vary together.
Correlation: A normalized measure of linear association between two variables.
Pearson correlation: The standard covariance-based correlation coefficient used on this page.
Outlier / unusual observation: An observation far from the main body of data that should be investigated rather than automatically discarded.
Covariance matrix: A matrix containing variances and pairwise covariances of a multivariable system.

Sources, method, and teaching-data note

Definitions and visual teaching choices follow standard sample covariance and Pearson-correlation treatments, including the NIST/SEMATECH e-Handbook of Statistical Methods. All datasets here are deterministic or reproducibly generated synthetic teaching data, not observational Earth-system records.

← Return to Foundations