Statistical Toolkit · 13

Importance Sampling: Rare Events and Tail Probabilities

Estimating events ordinary sampling almost never sees

Direct sampling is inefficient for extreme tails because most samples contribute zero. A shifted proposal deliberately visits the rare-event region and weights each visit correctly.

watchone idea

→

manipulateone example

→

leave withone intuition

The essential idea: make the rare event common under the proposal, then undo the bias with importance weights.

Watch the concept

One Concept · One Example

Importance Sampling: Rare Events and Tail Probabilities

Presented by Charlotte Moser

Watch on YouTube ↗

What to notice

A zero direct estimate does not mean zero true probability.
The indicator selects samples beyond the threshold.
Proposal placement controls weight stability.

The idea in 30 seconds

Move computational effort into the tail

Rare target event

For a standard Gaussian, exceeding four standard deviations has probability only about three in one hundred thousand.

P (Z > 4) = 3.167 \times 10^{- 5}

Shifted proposal

Samples from N(m,1) reach the tail frequently and receive weight p(x)/q(x).

\hat{P} = \frac{1}{N} \sum_{i = 1}^{N} 1_{x_{i} > 4} \frac{p (x_{i})}{q (x_{i})}

Explore

Can 100 samples see a four-sigma event?

Compare one run, then repeat K 100-sample experiments to reproduce the slides comparison of estimator PDFs, means, variances, and the many zero direct-MC results.

Direct Monte Carlo0.000e+00 tail hits in 100

Importance sampling4.175e-579 proposal tail hits

SLIDES EXPERIMENT · RARE-EVENT ESTIMATOR PDF

Repeat K experiments, using only N = 100 samples each time

Direct Monte Carlo usually reports zero because no four-sigma event appears. Importance sampling produces a continuous cloud of small, correctly weighted estimates around the truth.

number of repeated experiments KK = 1,000

Standard Monte Carlo

Mean 6.000e-5Variance 5.970e-7Zero estimates 994 / 1,000

Importance sampling

Mean 3.121e-5Variance 6.645e-11Truth 3.169e-5

The slides comparison becomes visible: many direct-MC experiments equal exactly zero, while importance sampling remains centered near 3.167 × 10⁻⁵ with much smaller variance.

KEY TAKEAWAY

Careful reweighting can estimate extremely small probabilities with far fewer samples than direct Monte Carlo.