Unsupervised Learning

Reviewed & published by Brayan K

Find hidden groups in data that has no labels. By the end you'll run one K-Means step by hand, cluster with scikit-learn, choose K with the elbow and silhouette methods, reach for DBSCAN and hierarchical clustering, and know where association rules and anomaly detection fit in.

Part of the free AI & Machine Learning course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

What You'll Learn in This Lesson

• What unsupervised learning is and why it needs no labels
• Run one K-Means step (assign + update) in plain Python
• Cluster data in two lines with scikit-learn's KMeans
• Choose K with the elbow method and silhouette score
• Pick DBSCAN or hierarchical clustering when K-Means fails
• Where association rules and anomaly detection fit in

🌍 Real-World Analogy: sorting without labels

Imagine someone hands you a shoebox of 1,000 photos with no folders, no tags, nothing. Nobody tells you the categories. Yet within minutes you make piles: beaches, faces, food, pets. You never knew the labels in advance — you discovered them by noticing which photos look alike.

That is unsupervised learning. Supervised learning is like sorting photos that already have a label on the back; unsupervised learning is sorting blank-backed photos into piles you invent yourself. The algorithm only sees the raw data and groups similar things together — which is exactly what clustering means.

📊 The four jobs of unsupervised learning

• • Clustering — group similar points (K-Means, DBSCAN, hierarchical)
• • Dimensionality reduction — compress many features into a few (PCA, t-SNE)
• • Anomaly detection — flag the rare, odd points (Isolation Forest)
• • Association rules — find what co-occurs (Apriori: "nappies → beer")

This lesson focuses on clustering, then tours the other three at the end.

1 K-Means, Step by Step (the whole idea)

K-Means is the most popular clustering algorithm, and its whole engine is two steps repeated in a loop. You pick K (how many groups you want), drop K centroids (a centroid is just the centre point of a cluster), then:

• Assign: every point joins its nearest centroid.
• Update: move each centroid to the mean (average) of the points that joined it.

Repeat assign → update until the centroids stop moving. That's it — no labels anywhere. The code below does one full iteration on six 1-D points in plain Python so you can see exactly what happens. Read every comment, then run it.

# K-MEANS BY HAND — one step, plain Python (no libraries)
# Goal: group points into 2 clusters when NOBODY told us the groups.

# Six points on a line: three small, three large.
points = [1.0, 1.5, 2.0, 8.0, 8.5, 9.0]

# Step 0: pick 2 starting centres (centroids). We just guess.
c1 = 2.0   # centre of cluster 0
c2 = 7.0   # centre of cluster 1

# Step 1: ASSIGN — each point joins the NEAREST centroid.
labels = []
for p in points:
    d1 = abs(p - c1)        # distance to centre 1
    d2 = abs(p - c2)        # distance to centre 2
    labels.append(0 if d1 <= d2 else 1)   # 0 = closer to c1, 1 = closer to c2

print("Points: ", points)
print("Labels: ", labels)        # which cluster each point joined

# Step 2: UPDATE — move each centre to the MEAN of its points.
group0 = [p for p, lab in zip(points, labels) if lab == 0]
group1 = [p for p, lab in zip(points, labels) if lab == 1]
c1 = sum(group0) / len(group0)
c2 = sum(group1) / len(group1)

print("Group 0:", group0, "-> new centre", round(c1, 2))
print("Group 1:", group1, "-> new centre", round(c2, 2))

# Expected output:
# Points:  [1.0, 1.5, 2.0, 8.0, 8.5, 9.0]
# Labels:  [0, 0, 0, 1, 1, 1]
# Group 0: [1.0, 1.5, 2.0] -> new centre 1.5
# Group 1: [8.0, 8.5, 9.0] -> new centre 8.5
#
# That is ONE iteration. Repeat assign -> update until the centres stop moving.

2 The Real Tool: scikit-learn's KMeans

You'll never hand-code the loop in real work — scikit-learn does it in two lines and runs the whole assign/update cycle for you. The same six points, now clustered by KMeans:

# THE REAL TOOL — scikit-learn does the assign/update loop for you.
from sklearn.cluster import KMeans
import numpy as np

# Same six 1-D points as before, reshaped into rows (KMeans wants 2-D input).
X = np.array([1.0, 1.5, 2.0, 8.0, 8.5, 9.0]).reshape(-1, 1)

# n_clusters = K (how many groups). n_init=10 reruns with different starts
# and keeps the best — this avoids a bad random start. random_state makes it
# reproducible so you get the same answer every run.
model = KMeans(n_clusters=2, n_init=10, random_state=0)
labels = model.fit_predict(X)          # one call: fit AND get cluster labels

print("Labels:   ", labels.tolist())                 # cluster id per point
print("Centres:  ", model.cluster_centers_.ravel())  # learned centres
print("Inertia:  ", round(model.inertia_, 2))         # total within-cluster spread

# Expected output (cluster ids 0/1 may swap — the grouping is what matters):
# Labels:   [1, 1, 1, 0, 0, 0]
# Centres:  [8.5 1.5]
# Inertia:  2.5

fit_predict(X) trains the model and hands back the cluster id of every point in one call. inertia_ is the total spread within clusters — smaller is tighter — and you'll use it for the elbow method in Section 5.

Your Turn 1: finish the assign step

One blank to fill. Each point must join whichever centre is nearer — you already wrote the line for c1, so mirror it for c2.

# 🎯 YOUR TURN — finish the ASSIGN step of K-Means.
# Each point should join whichever centre is nearer.

points = [2.0, 3.0, 10.0, 11.0]
c1 = 1.0     # centre of cluster 0
c2 = 9.0     # centre of cluster 1

labels = []
for p in points:
    d1 = abs(p - c1)
    d2 = ___          # 👉 distance from p to c2 (copy the line above, swap c1 for c2)
    nearest = 0 if d1 <= d2 else 1
    labels.append(nearest)

print("Labels:", labels)

# ✅ Expected output:
# Labels: [0, 0, 1, 1]

Your Turn 2: finish the update step

Now move each centre to the mean of its points. The c1 line is done — write the matching one for c2.

# 🎯 YOUR TURN — finish the UPDATE step.
# Move each centre to the MEAN (average) of the points assigned to it.

points = [2.0, 3.0, 10.0, 11.0]
labels = [0, 0, 1, 1]    # from the assign step

group0 = [p for p, lab in zip(points, labels) if lab == 0]
group1 = [p for p, lab in zip(points, labels) if lab == 1]

c1 = sum(group0) / len(group0)
c2 = ___                  # 👉 the mean of group1 (mirror the c1 line above)

print("New c1:", c1)
print("New c2:", c2)

# ✅ Expected output:
# New c1: 2.5
# New c2: 10.5

3 DBSCAN: Clusters of Any Shape + Outliers

K-Means has two weaknesses: you must pick K, and it only finds round blobs. DBSCAN (Density-Based Spatial Clustering) fixes both. It grows clusters wherever points are densely packed, finds any shape, decides the number of clusters itself, and labels lonely points as noise (id -1) — free anomaly detection.

Two knobs control it: eps (how close counts as "neighbour") and min_samples (how many neighbours make a dense core).

# DBSCAN — clusters by DENSITY. Finds any shape AND labels outliers as noise.
from sklearn.cluster import DBSCAN
import numpy as np

# A tight blob, a second tight blob, and one lonely outlier far away.
X = np.array([
    [1.0, 1.0], [1.2, 0.9], [0.9, 1.1],     # blob A
    [8.0, 8.0], [8.1, 7.9], [7.9, 8.2],     # blob B
    [50.0, 50.0],                           # outlier
])

# eps = neighbourhood radius. min_samples = points needed to form a dense core.
model = DBSCAN(eps=1.0, min_samples=2)
labels = model.fit_predict(X)

print("Labels:", labels.tolist())   # -1 means NOISE (an outlier)
n_clusters = len(set(labels) - {-1})
print("Clusters found:", n_clusters)
print("Noise points:  ", list(labels).count(-1))

# Expected output:
# Labels: [0, 0, 0, 1, 1, 1, -1]
# Clusters found: 2
# Noise points:   1
#
# Notice: you never told DBSCAN how many clusters to find. It decided.

4 Hierarchical Clustering: a Tree of Groups

Hierarchical (agglomerative) clustering starts with every point as its own tiny cluster, then repeatedly merges the two closest clusters until one remains. The record of merges is a tree called a dendrogram. The trick: you don't choose K up front — you cut the tree at any height afterwards. Cut high for few big clusters, cut low for many small ones.

# HIERARCHICAL CLUSTERING — start with every point alone, merge the closest.
# Builds a tree (dendrogram); cut it at any height to choose K.
from scipy.cluster.hierarchy import linkage, fcluster
import numpy as np

# Five points: two near 1, two near 8, one in between.
X = np.array([[1.0], [1.5], [8.0], [8.5], [4.5]])

# 'ward' merges the pair that least increases total spread (a good default).
Z = linkage(X, method="ward")

# Each row of Z = "merged cluster A and B at this distance, new size = N".
print("Merge steps (idxA, idxB, distance, size):")
for a, b, dist, size in Z:
    print(f"  merge {int(a)} + {int(b)}  at dist {dist:.2f}  -> size {int(size)}")

# Cut the tree into 2 clusters.
clusters = fcluster(Z, t=2, criterion="maxclust")
print("Two-cluster labels:", clusters.tolist())

# Expected output:
# Merge steps (idxA, idxB, distance, size):
#   merge 0 + 1  at dist 0.50  -> size 2
#   merge 2 + 3  at dist 0.50  -> size 2
#   merge 4 + 5  at dist 4.42  -> size 3
#   merge 6 + 7  at dist 8.06  -> size 5
# Two-cluster labels: [2, 2, 1, 1, 1]

Each row of the merge table shows which two clusters joined and at what distance. The big jump in distance on the last merges is your cue for where to cut — that's the dendrogram telling you the natural number of groups.

5 Choosing K: Elbow + Silhouette

K-Means won't pick K for you, so try a range and compare two numbers. The elbow method plots inertia against K — it always falls, but you want the "elbow" where it stops falling fast. The silhouette score (from -1 to 1) measures how tight and separated the clusters are; pick the K with the highest score. When both agree, you're confident.

# HOW MANY CLUSTERS? Compare K with the ELBOW and SILHOUETTE methods.
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
import numpy as np

# Three clearly separated blobs in 1-D -> the "true" answer is K = 3.
X = np.array([1, 1.2, 0.8, 5, 5.1, 4.9, 9, 9.2, 8.8]).reshape(-1, 1)

print("K  inertia  silhouette")
for k in range(2, 6):
    km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(X)
    sil = silhouette_score(X, km.labels_)   # 1.0 = perfect, near 0 = overlapping
    print(f"{k}  {km.inertia_:7.2f}  {sil:.3f}")

# Expected output:
# K  inertia  silhouette
# 2     8.34  0.715
# 3     0.28  0.838   <- elbow: inertia drops sharply then flattens; silhouette peaks
# 4     0.17  0.604
# 5     0.10  0.561
#
# ELBOW: inertia falls steeply up to K=3, then barely moves -> 3 is the "elbow".
# SILHOUETTE: highest score is at K=3 -> both methods agree on 3 clusters.

🧭 Beyond Clustering: Association & Anomaly Detection

Clustering is the headline act, but two other unsupervised jobs power real products — and neither needs labels:

Find items that show up together in baskets — "customers who buy nappies also buy beer." The Apriori algorithm mines these rules from transaction logs to drive recommendations and shelf layout.

Flag the rare, weird points — fraud, failing sensors, intrusions. Isolation Forest scores how easily each point is "isolated"; DBSCAN's -1 noise label is a simpler version of the same idea.

You'll meet dimensionality reduction (PCA, t-SNE) — squeezing many features into a few for plotting and speed — in the very next lesson.

6 Common Errors (And How to Fix Them)

These four mistakes trip up nearly every beginner. Spotting them early saves hours.

❌ Wrong K — guessing the number of clusters

Pick K=2 when the data really has 5 groups and K-Means happily merges distinct groups into mush. There's no error message — just bad clusters.

✅ Fix: never hard-code K blindly. Run the elbow + silhouette loop from Section 5 first, or switch to DBSCAN/hierarchical, which decide K for you.

❌ Unscaled features — one column dominates

Cluster on [income (0–100000), age (18–90)] and income's huge range drowns out age entirely — distance becomes "income distance".

from sklearn.preprocessing import StandardScaler
X_scaled = StandardScaler().fit_transform(X)   # mean 0, std 1 per feature
labels = KMeans(n_clusters=3, n_init=10).fit_predict(X_scaled)

✅ Fix: always scale features before any distance-based clustering.

❌ Assuming spherical clusters — K-Means on the wrong shape

K-Means assumes round, similarly-sized blobs. On crescents, rings, or stretched bands it slices straight through the real structure and the clusters look nonsensical.

✅ Fix: if your clusters aren't roughly spherical, use DBSCAN (density, any shape) or a Gaussian Mixture Model instead.

❌ No random init control — unstable, irreproducible results

K-Means starts from random centroids, so a single run can land in a bad local optimum and give different clusters every time.

# ❌ one start, different answer each run:
KMeans(n_clusters=3)

# ✅ many starts (keeps the best) + reproducible:
KMeans(n_clusters=3, n_init=10, random_state=42)

✅ Fix: set n_init=10 and a fixed random_state.

📋 Quick Reference

Mini-Challenge: segment your customers

No blanks this time — just a comment outline. Build the array, fit KMeans, and print the labels. The low spenders should share one group, the high spenders the other.

# 🎯 MINI-CHALLENGE: segment customers with scikit-learn KMeans
# Data: [annual_spend, visits_per_month] for 6 customers.
#
# 1. from sklearn.cluster import KMeans  and  import numpy as np
# 2. Build X with np.array([...]) using the rows below.
# 3. Fit KMeans with n_clusters=2, n_init=10, random_state=0.
# 4. Use fit_predict(X) to get the labels, then print them.
#
# Rows:
#   [100, 1], [120, 2], [110, 1],   <- low spenders
#   [900, 9], [950, 8], [880, 10]   <- high spenders
#
# ✅ Expected: two groups — the three low rows share one label,
#    the three high rows share the other (the 0/1 ids may swap).

# your code here

🎉 Lesson Complete!

You can now find hidden structure in unlabelled data. You ran a K-Means step by hand (assign + update), clustered in two lines with scikit-learn, chose K with the elbow and silhouette methods, reached for DBSCAN and hierarchical clustering when blobs aren't enough, and saw where association rules and anomaly detection fit.

Practice quiz

Continue this course

• Previous: Dimensionality Reduction with PCA
• Next: Dimensionality Reduction — Beyond PCA — t-SNE, UMAP and choosing how many dimensions to keep
• Quick reference: AI & Machine Learning cheat sheet

Frequently asked questions

What is the difference between supervised and unsupervised learning?

Supervised learning trains on labelled examples — every row comes with the correct answer (this email is spam, this house sold for £300k), and the model learns to reproduce those answers. Unsupervised learning gets no labels at all. It looks at the raw features and discovers structure on its own — which points sit close together, which look unusual. Use unsupervised learning when you do not know the groups in advance, or when labelling would be too expensive.

How do I choose K, the number of clusters, for K-Means?

K-Means cannot choose K for you, so you try several and compare. The elbow method plots inertia (total within-cluster spread) against K: it always falls as K rises, but you look for the 'elbow' where it stops falling sharply — that K captures the structure without over-splitting. The silhouette score (-1 to 1, higher is better) measures how tight and well-separated the clusters are; pick the K with the highest score. When both methods agree, you can be fairly confident. If you want clusters chosen automatically, use DBSCAN instead.

Why must I scale my features before clustering?

K-Means and most clustering algorithms measure distance, and distance is dominated by whichever feature has the biggest numeric range. If income runs 0–100000 and age runs 18–90, income alone decides every cluster and age is ignored. Standardise first — for example with StandardScaler, which rescales each feature to mean 0 and standard deviation 1 — so every feature contributes fairly. Skipping this is the single most common clustering mistake.

When should I use DBSCAN instead of K-Means?

Use DBSCAN when your clusters are not round blobs (think crescents, rings, or long curved bands), when you do not know how many clusters there are, or when you need outliers flagged. DBSCAN groups by density and labels low-density points as noise (-1), so it finds arbitrary shapes and detects anomalies for free. K-Means is faster and simpler, but it assumes roughly spherical, similarly-sized clusters and forces every point into a group — including outliers.

What else can unsupervised learning do besides clustering?

Two big jobs. Association rule mining finds items that co-occur — the classic 'customers who buy nappies also buy beer' from shopping baskets (the Apriori algorithm) — which powers recommendations and store layout. Anomaly detection finds the rare, unusual points: fraudulent transactions, failing machines, network intrusions. Algorithms like Isolation Forest score how 'odd' each point is, and DBSCAN's noise label is a simple form of the same idea. Both work without any labels.

Related lessons

• Classification