Math foundations

🎯 Goal

After reading this chapter:

• You will understand concepts like vector, matrix, gradient encountered in ML code
• You’ll be able to see from a mathematical perspective why algorithms work the way they do
• Terms like loss function, gradient descent won’t be a “black box” to you

**Note:**math for ML isn’t a full university course. You just need intuition and understanding the meaning of basic operations. You don’t need to study deep theorems.

What to learn

1. Linear Algebra

• Scalar, Vector, Matrix, Tensor — in ML data has this shape
• Vector operations — addition, multiplication, dot product
• Matrix operations — transpose, multiplication, inverse, determinant
• Identity matrix, Diagonal matrix — special matrices
• Eigenvalues and Eigenvectors — for PCA and SVD

2. Calculus

• Function — input → output
• Derivative — how quickly a function changes
• Partial derivative — for multi-variable functions
• Gradient — vector of all partial derivatives
• Chain rule — foundation of neural networks (backpropagation)
• Optimization — finding minimum/maximum

3. Statistics and Probability

• Mean, Median, Mode — measures of central tendency
• Variance, Standard Deviation — spread
• Normal distribution (Gaussian) — the most important distribution in ML
• Probability distributions — Bernoulli, Binomial, Poisson, Uniform
• Bayes Theorem — conditional probability
• Correlationvs Causation — relation vs cause
• Hypothesis testing — for A/B tests

Libraries

pip install numpy scipy sympy matplotlib

• NumPy — vector/matrix computations
• SciPy — advanced math functions, statistics
• SymPy — symbolic math (working with formulas)

Important topics

Vector and Matrix in ML

Any data in ML takes the form of a tensor:

• Scalar(0-d tensor) — a single number: 5
• Vector(1-d tensor) — list of numbers: [1, 2, 3] (e.g., one student’s grades in 3 subjects)
• Matrix(2-d tensor) — table: [[1,2,3], [4,5,6]] (e.g., 2 students × 3 subjects)
• Tensor(3+ d) — e.g., image: [height, width, channels]

What is a gradient and why do we need it?

Imagine you’re on a mountain and need to descend to the lowest point. Gradient tells you: “which direction is the steepest climb” — you take a step in the opposite direction. That’s the essence of the Gradient Descent algorithm.

In ML:

• Mountain = loss function(error level)
• Descent = training
• Goal = minimize loss

Why is Normal distribution important?

Many real-world measurements (people’s heights, product prices, IQ) have a normal distribution. This follows from the Central Limit Theorem. ML algorithms are often tuned to this distribution as well.

Code examples

Vector and matrix with NumPy

import numpy as np

# Vector
v = np.array([1, 2, 3])

# Matrix
A = np.array([[1, 2], [3, 4]])

# Dot product
u = np.array([4, 5, 6])
result = np.dot(v, u)  # 1*4 + 2*5 + 3*6 = 32

# Matrix multiplication
B = np.array([[5, 6], [7, 8]])
C = A @ B  # or np.matmul(A, B)

# Transpose
A_T = A.T

Computing gradient (simple example)

import numpy as np

# derivative of f(x) = x^2: f'(x) = 2x
def f(x):
    return x ** 2

def gradient_f(x):
    return 2 * x

# Gradient descent — finding minimum
x = 10.0  # starting point
learning_rate = 0.1

for i in range(20):
    grad = gradient_f(x)
    x = x - learning_rate * grad  # step in opposite direction
    print(f"step{i}: x ={x:.4f}, f(x) ={f(x):.4f}")

# Result: x → approaches 0 (minimum of f(x) = x^2)

Statistical measures

import numpy as np

data = np.array([2, 4, 4, 4, 5, 5, 7, 9])

print(f"Mean:{np.mean(data)}")      # 5.0
print(f"Median:{np.median(data)}")    # 4.5
print(f"Std:{np.std(data):.2f}")   # 2.00
print(f"Var:{np.var(data):.2f}")   # 4.00

# Random number from normal distribution
sample = np.random.normal(loc=0, scale=1, size=1000)
print(f"Sample mean:{sample.mean():.3f}")  # close to ~0
print(f"Sample std:{sample.std():.3f}")   # close to ~1

Backend integration

As a backend dev, you’ll need math in these places:

1. Analytics endpoints — Django /api/stats/ route — for computing mean, median, percentile you can use NumPy (faster than Python’s built-in statistics module)
2. A/B testing backend — checking if the difference between two versions is statistically significant (scipy.stats.ttest_ind)
3. Anomaly detection — finding outliers via z-score or IQR
4. Rate limiting and load forecasting — forecasting request load via Poisson distribution

# Example of a statistics endpoint in FastAPI
from fastapi import FastAPI
import numpy as np
from scipy import stats

app = FastAPI()

@app.post("/api/stats/")
def calculate_stats(values: list[float]):
    arr = np.array(values)
    return {
        "mean": float(arr.mean()),
        "median": float(np.median(arr)),
        "std": float(arr.std()),
        "p95": float(np.percentile(arr, 95)),
        "outliers_zscore": [
            float(v) for v in arr if abs((v - arr.mean()) / arr.std()) > 3
        ],
    }

Resources

Free

• 3Blue1Brown — “Essence of Linear Algebra”(YouTube playlist) — visual explanation, MUST WATCH(link)
• 3Blue1Brown — “Essence of Calculus”(YouTube playlist) — for calculus
• Khan Academy — Linear Algebra(link)
• StatQuest with Josh Starmer(YouTube) — simplifying statistics concepts
• “Mathematics for Machine Learning” — Deisenroth, Faisal, Ong (free PDF: mml-book.com)

Paid (optional)

• Coursera — Mathematics for Machine Learning Specialization(Imperial College London)

🏋️ Exercises

🟢 Easy

1. Create 5 random numbers with NumPy, find their mean, median, std.
2. Compute the dot product of vectors [1, 2, 3] and [4, 5, 6] by hand, then verify with NumPy.
3. Create a 3x3 identity matrix.

🟡 Medium

1. Find the minimum of f(x) = (x-3)^2 + 5 via gradient descent (try different learning rates: 0.01, 0.1, 1.0).
2. Create a dataset of 1000 random normal numbers and plot a histogram (with matplotlib).
3. Using scipy.stats, run a t-test for results of two groups and interpret the p-value.

🔴 Hard

1. Write a FastAPI endpoint: user sends a list [float], respond with mean, std, outliers (z-score > 3), normality test (Shapiro-Wilk) results. Make it fully type-safe with Pydantic models.

Capstone (final exercise)

In the file notebooks/month-01/00_math_warmup.ipynb do the following:

1. Create a 100×100 random matrix with NumPy
2. Find its eigenvalues and eigenvectors (np.linalg.eig)
3. Decompose the matrix using SVD (np.linalg.svd)
4. Visualize the singular values

✅ Checklist

• I understand the difference between vector and matrix
• I know what a dot product is and when it’s used
• What is a gradient — I can explain in one sentence
• I’ve written the gradient descent algorithm in code
• I know the difference between mean, median, std
• I understand what normal distribution is and why it’s important
• I can give an example of Bayes theorem
• I know how to perform matrix operations in NumPy

If you’re ready, move on to the NumPy chapter.