Machine Learning Toolkit · 10

PCA and Linear Regression

Two notions of the best line for denoising

PCA and linear regression can both replace a noisy cloud with a line, but they optimize different distances and answer different questions.

watchone idea

→

manipulateone example

→

leave withone intuition

The essential idea: regression minimizes vertical error; PCA minimizes orthogonal reconstruction error.

Watch the concept

One Concept · One Example

Presented by Charlotte Moser

The idea in 30 seconds

Vertical residuals are appropriate when x is treated as known and y is predicted.

\min_{θ} \sum_{i}^{n} (^{y_{i} - {\hat{y}}_{i}) 2}

Orthogonal residuals are appropriate when all coordinates are noisy measurements.

\min_{v} \sum_{i}^{n} ‖^{x_{i} - {\hat{x}}_{i} ‖ 2}

Explore

Increase noise in both coordinates. See when the regression and PCA directions visibly separate.

KEY TAKEAWAY

Choose regression for directional prediction and PCA for symmetric geometric denoising.