Clustering

🎯 Goal

After reading this chapter:

• You will understand what unsupervised learning and clustering are
• You will know the difference between K-Means, DBSCAN, and Hierarchical algorithms
• You will know methods to find the optimal number of clusters
• You will be able to apply it to real business projects like customer segmentation

What to learn

• K-Means — the simplest and most widely used
• K-Means++ — better initialization
• MiniBatchKMeans — for large datasets
• DBSCAN — density-based, arbitrary shapes
• Hierarchical Clustering — agglomerative, dendrogram
• Gaussian Mixture Models (GMM) — soft clustering
• Mean Shift, OPTICS — alternatives
• Choosing number of clusters — Elbow method, Silhouette score
• Visualization — to 2D with PCA, t-SNE, UMAP

Libraries

pip install scikit-learn umap-learn yellowbrick

• scikit-learn — main algorithms
• umap-learn — dimensionality reduction (faster and more accurate than t-SNE)
• yellowbrick — ML visualizations (Elbow, Silhouette)

Important topics

When is clustering needed?

• Customer segmentation — grouping customers (for marketing)
• Anomaly detection — which point doesn’t fit any group
• Document grouping — finding similar texts
• Image compression — clustering colors
• Feature engineering — making cluster ID a new feature

K-Means algorithm

1. K ta tasodifiy markaz (centroid) tanlash
2. Har nuqtani eng yaqin centroidga assign qilish
3. Centroidlarni o'rta arifmetik bilan yangilash
4. Konvergentsiyaga qadar 2-3 qadamlarini takrorlash

Limitations:

• K must be known in advance
• Only spherical clusters
• Sensitive to outliers
• Feature scaling is important

DBSCAN — alternative

Unlike K-Means:

• K is not needed (automatic)
• Clusters of arbitrary shape
• Automatically detects outliers (noise label -1)
• 2 parameters: eps (radius) and min_samples

DBSCAN'da nuqta turlari:
- Core: eps radiusida >= min_samples ta nuqta
- Border: core'ga yaqin lekin o'zi core emas
- Noise: hech bir cluster'ga to'g'ri kelmaydi (outlier)

Finding the optimal K

1. Elbow method:

Compute inertia (within-cluster sum of squares) for each K
→ Find the "elbow" location (by bend)

2. Silhouette score:

Score = (b - a) / max(a, b)
a = average distance to own cluster
b = average distance to nearest other cluster

Range: [-1, 1]
1 = ajoyib clustering
0 = overlapping clusters
< 0 = noto'g'ri assignment

Code examples

K-Means clustering

import numpy as np
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import silhouette_score
import matplotlib.pyplot as plt

# Synthetic data
X, _ = make_blobs(n_samples=500, centers=4, n_features=2, random_state=42)

# Scale (IMPORTANT — for distance-based algorithms)
X_scaled = StandardScaler().fit_transform(X)

# K-Means
kmeans = KMeans(n_clusters=4, n_init=10, random_state=42)
labels = kmeans.fit_predict(X_scaled)

# Silhouette
score = silhouette_score(X_scaled, labels)
print(f"Silhouette Score:{score:.3f}")  # 0.8+ — good

# Visualization
fig, ax = plt.subplots(figsize=(8, 6))
ax.scatter(X[:, 0], X[:, 1], c=labels, cmap="viridis", s=30)
ax.scatter(*kmeans.cluster_centers_.T, c="red", s=200, marker="X", label="Centroids")
ax.legend()
plt.show()

Elbow method

from yellowbrick.cluster import KElbowVisualizer

model = KMeans(n_init=10, random_state=42)
visualizer = KElbowVisualizer(model, k=(2, 11), metric="distortion")
visualizer.fit(X_scaled)
visualizer.show()
# Automatically detects the "elbow point"

Silhouette analysis

from sklearn.metrics import silhouette_score

scores = {}
for k in range(2, 11):
    km = KMeans(n_clusters=k, n_init=10, random_state=42)
    labels = km.fit_predict(X_scaled)
    scores[k] = silhouette_score(X_scaled, labels)

best_k = max(scores, key=scores.get)
print(f"Best k:{best_k}(silhouette ={scores[best_k]:.3f})")

DBSCAN

from sklearn.cluster import DBSCAN

dbscan = DBSCAN(eps=0.3, min_samples=10)
labels = dbscan.fit_predict(X_scaled)

n_clusters = len(set(labels)) - (1 if -1 in labels else 0)
n_noise = list(labels).count(-1)
print(f"Clusters:{n_clusters}, Noise points:{n_noise}")

Hierarchical Clustering + Dendrogram

from scipy.cluster.hierarchy import dendrogram, linkage, fcluster
import matplotlib.pyplot as plt

linkage_matrix = linkage(X_scaled, method="ward")

fig, ax = plt.subplots(figsize=(12, 5))
dendrogram(linkage_matrix, truncate_mode="lastp", p=20, leaf_font_size=10, ax=ax)
ax.set_title("Hierarchical Clustering Dendrogram")
plt.show()

# Cut tree at threshold
labels = fcluster(linkage_matrix, t=4, criterion="maxclust")

Customer Segmentation — Real example (RFM)

import pandas as pd
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

# Synthetic customer data
df = pd.DataFrame({
    "customer_id": range(1000),
    "recency_days": np.random.exponential(30, 1000),
    "frequency": np.random.poisson(5, 1000),
    "monetary": np.random.exponential(500, 1000),
})

# RFM scaling
X = df[["recency_days", "frequency", "monetary"]].copy()
X["recency_days"] = -X["recency_days"]  # less is better → invert
X_scaled = StandardScaler().fit_transform(X)

# Clustering
km = KMeans(n_clusters=4, n_init=10, random_state=42)
df["segment"] = km.fit_predict(X_scaled)

# Segment summaries
segment_summary = df.groupby("segment")[["recency_days", "frequency", "monetary"]].mean()
print(segment_summary)
# Business naming: # Champions: low recency, high freq, high monetary # At Risk: high recency, low freq, low monetary # etc.

Backend integration

Customer Segmentation API

from fastapi import FastAPI
from pydantic import BaseModel
import joblib
import numpy as np

app = FastAPI()
model_bundle = joblib.load("models/customer_segments.joblib")
# {"kmeans": kmeans, "scaler": scaler, "segment_names": [...]}

class CustomerRFM(BaseModel):
    recency_days: int
    frequency: int
    monetary: float

class SegmentResponse(BaseModel):
    segment_id: int
    segment_name: str
    marketing_action: str

SEGMENT_ACTIONS = {
    0: "Champions — VIP offer",
    1: "At Risk — win-back campaign",
    2: "New — onboarding email",
    3: "Loyal — referral program",
}

@app.post("/segment", response_model=SegmentResponse)
def get_segment(customer: CustomerRFM):
    X = np.array([[-customer.recency_days, customer.frequency, customer.monetary]])
    X_scaled = model_bundle["scaler"].transform(X)
    seg_id = int(model_bundle["kmeans"].predict(X_scaled)[0])
    return SegmentResponse(
        segment_id=seg_id,
        segment_name=model_bundle["segment_names"][seg_id],
        marketing_action=SEGMENT_ACTIONS[seg_id],
    )

Anomaly Detection (DBSCAN)

@app.post("/check-anomaly")
def detect_anomaly(transaction: TransactionData):
    X = np.array([[transaction.amount, transaction.time_of_day, ...]])
    X_scaled = scaler.transform(X)
    
    # DBSCAN re-fit on recent data + new point
    cluster = dbscan.fit_predict(np.vstack([recent_data, X_scaled]))[-1]
    
    is_anomaly = cluster == -1
    return {"is_anomaly": is_anomaly, "cluster": int(cluster)}

Resources

• Scikit-learn Clustering — scikit-learn.org/stable/modules/clustering.html
• StatQuest — K-Means, Hierarchical Clustering(YouTube)
• “K-Means visualization” — naftaliharris.com/blog/visualizing-k-means-clustering/
• UMAP docs — t-SNE alternative
• “Customer Segmentation in Python” — Towards Data Science

🏋️ Exercises

🟢 Easy

1. Create 3 clusters with make_blobs, classify with K-Means, and visualize.
2. Find the optimal K with the Elbow method (K=2..10).
3. Change eps in DBSCAN (0.1, 0.3, 0.5, 1.0) and observe the result.

🟡 Medium

1. Load the Wholesale Customerdataset (UCI), find customer segments with K-Means, and interpret each segment from a business perspective.
2. Visualize high-dimensional data in 2D using t-SNE / UMAP.
3. Silhouette analysis: Plot silhouette plots for different k values.

🔴 Hard

1. Segmentation API: Production-ready FastAPI service — returns real-time segment when customer RFM data arrives, model is retrained every week (Airflow or cron).
2. Image color quantization: Load an image file and recreate it with 16 dominant colors using K-Means (image compression).

Capstone

notebooks/month-02/03_clustering.ipynb:

• Mall Customer SegmentationKaggle dataset
• EDA → feature selection (Age, Income, Spending Score)
• Compare K-Means, DBSCAN, Hierarchical
• Finding the optimal K
• Name each cluster in business terms (Premium, Budget, Young Spenders, etc.)
• Write marketing recommendations

✅ Checklist

• I know the difference between Supervised vs Unsupervised
• I understand how the K-Means algorithm works
• I know how to find optimal K with Elbow and Silhouette
• I know when to use K-Means vs DBSCAN
• I know why feature scaling is important for clustering
• I can apply clustering to a real business project like customer segmentation

Moving on to Feature Engineering.