Dimensionality Reduction with PCA

Squash many correlated features into a handful of new axes that keep most of the information. Learn how PCA finds directions of maximum variance and how to read the explained variance ratio.

Learn Dimensionality Reduction with PCA in our free AI & Machine Learning course — a beginner-friendly interactive lesson with worked examples, a practice…

Part of the free AI & Machine Learning course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

To photograph a 3D sculpture as a single 2D photo, you choose the angle that shows the most detail — you'd never shoot it edge-on, where everything collapses into a thin line. PCA does the same with data: it finds the viewing angle (direction) along which the points spread out the most, then flattens onto it while losing as little as possible.

That best angle is the first principal component . The next-best angle, at right angles to it, is the second, and so on.

PCA rests on one idea: the directions where your data spreads out the most carry the most information. A direction where all the points sit on top of each other tells you nothing; a direction where they fan out tells you a lot. That spread is the variance .

So PCA's job is to find new axes — principal components — ordered by how much variance each one captures. PC1 captures the most, PC2 the next-most (and is orthogonal to PC1), and so on. Each component is a linear combination (weighted blend) of your original features.

Let's make it concrete. We project the data onto candidate directions and measure the spread of the resulting 1D shadow. The direction with the most variance is closest to the true first component.

Each component captures a fraction of the data's total variance — its explained variance ratio . These fractions are sorted high to low and they add up to 1.0 across all components. To decide how many components to keep, add them from the top until you hit a target like 90% or 95%.

For example, if PC1 keeps 0.85 and PC2 keeps 0.10, then two components keep 0.85 + 0.10 = 0.95 — 95% of the variance with half the axes you might have started with.

Here's PCA with scikit-learn's PCA , after scaling. Study it (it isn't runnable in the in-browser sandbox). Notice fit_transform turns 4 features into 2 components, and explained_variance_ratio_ tells you how much each one kept.

The scaler comes before PCA in the pipeline — skipping it lets the largest-range feature hijack the first component.

Reach for PCA when you have many features, especially correlated ones . It speeds up training, reduces noise, fights the "curse of dimensionality", and lets you plot high-dimensional data in 2D or 3D.

Fill in the blanks so the helper projects the data onto a direction and returns the variance of the shadow. The expected output is in the comments.

Add explained-variance ratios until you reach a target. Fill in the blank so the function returns the number of components needed for 90% variance.

Given each component's explained variance, print the running cumulative total and how many components reach 95%. Only a comment outline is provided.

These are the most common PCA mistakes. Watch for them.

The largest-range feature dominates the first component for no real reason.

Fitting PCA on all data before splitting leaks test information into training — a form of data leakage.

✅ Fix: fit on train only, transform the test set:

❌ Using PCA when you need interpretable features

Components blend many features, so you can no longer explain a decision in plain terms.

✅ Fix: keep the original features (or use feature selection) when explanation matters:

You now understand how PCA finds the directions of greatest variance , what principal components are, how to read the explained variance ratio , and why you must scale first .

🚀 Up next: Gradient Boosting — chaining weak learners into one of the most accurate models for tabular data.

Practice quiz

• Browse all free courses
• Cheat sheets & quizzes
• Coding dictionary