Statistical Toolkit · 12

Importance Sampling: Reweighting Samples for Integration

Putting random samples where an integral matters most

Standard Monte Carlo can waste samples where the integrand is nearly zero. Importance sampling draws from a better proposal and corrects the change with weights.

watchone idea

→

manipulateone example

→

leave withone intuition

The essential idea: sample from a convenient proposal, then reweight so the desired integral remains unchanged.

Watch the concept

One Concept · One Example

Importance Sampling: Reweighting Samples for Integration

Presented by Charlotte Moser

Watch on YouTube ↗

What to notice

The proposal must cover every important target region.
Weights correct the deliberate sampling bias.
A well-matched proposal lowers estimator variance.

The idea in 30 seconds

Change the samples, preserve the answer

Reweighting identity

Insert a proposal density q into the integral, then treat the ratio p/q as part of the sampled quantity.

E_{p} [f (X)] = E_{q} [f (X) \frac{p (X)}{q (X)}]

Numerical integration

For an unnormalized integrand, divide directly by the proposal density at each sampled point.

\int_{0}^{\infty} h (x) d x = E_{q} [\frac{h (X)}{q (X)}]

Explore

Compare uniform and importance sampling

Estimate the integral of exp(-x^1.5), then repeat many 100-sample experiments to compare the full estimator PDFs, means, and variances from the slides.

integrand exp(-x^1.5)proposal q(x)=exp(-x)

Uniform [0,5]0.921754estimated SE 0.06425

Importance sampling0.907877estimated SE 0.01453

SLIDES EXPERIMENT · DISTRIBUTION OF ESTIMATES

Repeat many experiments with only N = 100 samples each

Each bar chart is an empirical PDF built from K independent estimates. A good estimator should be centered near the truth and tightly concentrated.

number of repeated experiments KK = 1,000

Standard Monte Carlo

Mean 0.906081Variance 1.97e-2

Importance sampling

Mean 0.901984Variance 1.13e-3

At S = 5, importance sampling uses the same 100 samples per experiment but typically produces a much narrower PDF.

KEY TAKEAWAY

Importance sampling can reduce Monte Carlo variability dramatically by concentrating samples where their weighted contributions matter.