Introduction to ML

🎯 Goal

After reading this chapter:

• You will understand what Machine Learning is and when to use it
• You will know the difference between Supervised, Unsupervised, and Reinforcement learning
• You will know why Training, Validation, and Test sets are needed
• You will recognize Overfitting and Underfitting problems
• You will write one complete ML pipeline (idea → data → model → evaluation)

What to learn

• What ML is and when it’s needed(why you shouldn’t avoid it)
• ML task types — supervised vs unsupervised vs reinforcement
• Supervised tasks — regression vs classification
• Train / Validation / Test — why we split into 3
• Overfitting and Underfitting — bias-variance tradeoff
• Cross-validation — k-fold strategy
• Scikit-learn API design — fit, predict, transform, fit_transform
• Pipeline and ColumnTransformer

Libraries

pip install scikit-learn pandas numpy matplotlib seaborn joblib

Main version: scikit-learn 1.4+.

Important topics

When ML is NOT needed?

As a backend dev, if simple if/else rules work — don’t use ML. Situations where ML is needed:

✅ Too many and changing rules (spam filter) ✅ Complex patterns (image recognition, language translation) ✅ Personalization (different for each user) ✅ Prediction (future sales, disease risk)

❌ There’s a clear formula (area = π * r²) ❌ Little data (10 examples — not ML, write rules) ❌ Critical safety (uncontrolled ML — dangerous) ❌ Explainability required and ML is black-box

ML task types

1. Supervised Learning (input → output mavjud)
   ├── Regression: continuous output
   │   └── Misol: uy narxi, harorat, foydalanuvchi LTV
   └── Classification: discrete classes
       ├── Binary: spam/not-spam, churn/retain
       └── Multi-class: rasm turi, kasallik turi

2. Unsupervised Learning (faqat input)
   ├── Clustering: o'xshashlarni guruhlash
   ├── Dimensionality reduction: PCA, t-SNE
   └── Anomaly detection: g'ayrioddiy nuqtalar

3. Reinforcement Learning (agent + reward)
   └── O'yinlar, robotics, recommendation systems

Train / Validation / Test split

**Why?**To evaluate how the model will perform on “future” data.

Hammasi (100%)
├── Training set (60-70%)    — model o'rganadi
├── Validation set (15-20%)  — hyperparameter tuning
└── Test set (15-20%)        — yakuniy baholash (faqat 1 marta!)

**Important rule:**The model must not see the test set during training. Otherwise — data leakage and incorrect results.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

Overfitting vs Underfitting

Underfitting          Just right            Overfitting
   .                     .                     .
  / \                   / \                   /\/\
 /   \                 /   \                 /    \
/     \               /     \               /      \
Model juda             Model muvozanatda      Model trainni
oddiy                                         yodlab olgan

Solutions:

• Underfitting: more complex model, more features
• Overfitting: regularization, more data, simpler model, cross-validation

Cross-validation

A single train/test split is sometimes unfair. Solution — k-fold cross-validation:

5-fold CV:
Fold 1: Test=[1], Train=[2,3,4,5]
Fold 2: Test=[2], Train=[1,3,4,5]
Fold 3: Test=[3], Train=[1,2,4,5]
Fold 4: Test=[4], Train=[1,2,3,5]
Fold 5: Test=[5], Train=[1,2,3,4]
→ 5 ta accuracy → mean ± std

Code examples

Complete pipeline example (Iris classification)

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.metrics import accuracy_score, classification_report
import joblib

# 1. Load data
iris = load_iris(as_frame=True)
X, y = iris.data, iris.target

# 2. Train/test split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# 3. Create pipeline (preprocessing + model in one place)
pipeline = Pipeline([
    ("scaler", StandardScaler()),
    ("classifier", LogisticRegression(max_iter=1000, random_state=42)),
])

# 4. Train
pipeline.fit(X_train, y_train)

# 5. Predict and evaluate
y_pred = pipeline.predict(X_test)
print(f"Accuracy:{accuracy_score(y_test, y_pred):.3f}")
print(classification_report(y_test, y_pred, target_names=iris.target_names))

# 6. Save
joblib.dump(pipeline, "iris_model.joblib")

Cross-validation

from sklearn.model_selection import cross_val_score

scores = cross_val_score(pipeline, X, y, cv=5, scoring="accuracy")
print(f"CV accuracy:{scores.mean():.3f}±{scores.std():.3f}")
print(f"Each fold:{scores}")

ColumnTransformer (mixed types)

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder

# Separate preprocessing for numeric and categorical columns
preprocessor = ColumnTransformer([
    ("num", StandardScaler(), ["age", "salary"]),
    ("cat", OneHotEncoder(handle_unknown="ignore"), ["city", "department"]),
])

full_pipeline = Pipeline([
    ("preprocess", preprocessor),
    ("classifier", LogisticRegression()),
])

full_pipeline.fit(X_train, y_train)

Backend integration

Serving an ML model in FastAPI (minimal)

# main.py
from fastapi import FastAPI
from pydantic import BaseModel
import joblib
import numpy as np

app = FastAPI()
model = joblib.load("iris_model.joblib")

CLASS_NAMES = ["setosa", "versicolor", "virginica"]

class IrisInput(BaseModel):
    sepal_length: float
    sepal_width: float
    petal_length: float
    petal_width: float

class IrisPrediction(BaseModel):
    class_name: str
    confidence: float

@app.post("/predict", response_model=IrisPrediction)
def predict(data: IrisInput):
    X = np.array([[data.sepal_length, data.sepal_width, 
                   data.petal_length, data.petal_width]])
    pred = model.predict(X)[0]
    proba = model.predict_proba(X)[0]
    return IrisPrediction(
        class_name=CLASS_NAMES[pred],
        confidence=float(proba.max()),
    )

@app.get("/health")
def health():
    return {"status": "ok", "model_version": "v1"}

uvicorn main:app --reload
# POST http://localhost:8000/predict
# {"sepal_length": 5.1, "sepal_width": 3.5, "petal_length": 1.4, "petal_width": 0.2}

Best practices for ML serving

from contextlib import asynccontextmanager
from fastapi import FastAPI

# Load model only once (lifespan)
@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.model = joblib.load("iris_model.joblib")
    app.state.model_version = "v1.2.0"
    yield
    # cleanup if needed

app = FastAPI(lifespan=lifespan)

Resources

• Scikit-learn User Guide — scikit-learn.org/stable/user_guide.html
• “Hands-On Machine Learning” — Aurélien Géron (3rd edition) — MUST READbook
• Andrew Ng — Machine Learning Specialization(Coursera) — free auditing
• StatQuest — best YouTube channel explaining ML algorithms
• Kaggle Learn — Intro to Machine Learning (free mini-course)

🏋️ Exercises

🟢 Easy

1. Load sklearn.datasets.load_wine(), classify with LogisticRegression, output accuracy.
2. Change random_state in train_test_split multiple times and observe the differences.
3. Compare 3-fold and 10-fold CV with cross_val_score.

🟡 Medium

1. Pipeline example: Create a Pipeline for the Titanic dataset — SimpleImputer + OneHotEncoder + StandardScaler + LogisticRegression.
2. Stratification: See the difference between using and not using stratify=y on an imbalanced dataset.
3. Overfitting demo: Add PolynomialFeatures(degree=20) to a single-feature regression and visually demonstrate overfitting.

🔴 Hard

1. FastAPI ML service: Containerize the Iris classifier with Docker, add CI/CD with GitHub Actions, create a healthcheck endpoint. This work will be the foundation for the 6-month MLOps project.
2. Custom Estimator: Create your own custom transformer that inherits from BaseEstimator and TransformerMixin — make it usable inside a Pipeline.

Capstone

notebooks/month-02/00_ml_intro.ipynb:

• Load the California Housingdataset (sklearn.datasets.fetch_california_housing)
• Train/test split, train LinearRegression
• Evaluate with cross-validation (RMSE)
• Write it as a Pipeline (StandardScaler + LinearRegression)
• Create a FastAPI endpoint and test it with curl

✅ Checklist

• I know the difference between Supervised vs Unsupervised
• I can distinguish Regression and Classification tasks
• I understand why Train/Validation/Test splitting is needed
• I can recognize Overfitting and Underfitting when I see them
• I can write Cross-validation in code
• I use Scikit-learn Pipeline and ColumnTransformer
• I know how to save a model and serve it with FastAPI
• I understand the significance of random_state=42

Moving on to Regression.