Working with OpenCV

🎯 Goal

After reading this chapter:

You will know OpenCV’s main operations
You will be able to do classical image processing (filtering, edge detection, contour)
You will be able to write real-time video processing
You will be able to do preprocessing steps before ML models

What to learn

Loading/Saving — imread, imwrite, VideoCapture
Color spaces — RGB, HSV, Grayscale, Lab
Geometric transformations — resize, rotate, crop, warp
Filtering — blur, Gaussian, median, bilateral
Edge detection — Sobel, Canny
Thresholding — binary, adaptive, Otsu
Morphological ops — erosion, dilation, opening, closing
Contours — finding, drawing, properties
Histograms — equalization, matching
Feature detection — Harris corners, SIFT, ORB
Image stitching, perspective correction

Libraries

pip install opencv-python opencv-contrib-python
# opencv-contrib-python — extra modules (SIFT, etc.)

Code examples

Loading and Saving

import cv2

# Image
img = cv2.imread("photo.jpg")               # BGR
img_rgb = cv2.imread("photo.jpg", cv2.IMREAD_COLOR)  # BGR default
gray = cv2.imread("photo.jpg", cv2.IMREAD_GRAYSCALE)
cv2.imwrite("output.jpg", img)

# Video
cap = cv2.VideoCapture("video.mp4")        # or 0 (webcam)
while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break
    # process frame
    cv2.imshow("Video", frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break
cap.release()
cv2.destroyAllWindows()

Geometric transformations

# Resize
small = cv2.resize(img, (640, 480))
large = cv2.resize(img, None, fx=2, fy=2, interpolation=cv2.INTER_CUBIC)

# Rotate
h, w = img.shape[:2]
center = (w // 2, h // 2)
M = cv2.getRotationMatrix2D(center, angle=30, scale=1.0)
rotated = cv2.warpAffine(img, M, (w, h))

# Affine transformation (3 points)
pts1 = np.float32([[50,50],[200,50],[50,200]])
pts2 = np.float32([[10,100],[200,50],[100,250]])
M = cv2.getAffineTransform(pts1, pts2)
warped = cv2.warpAffine(img, M, (w, h))

# Perspective transformation (4 points) — e.g., "flattening" a document
pts1 = np.float32([[56,65],[368,52],[28,387],[389,390]])
pts2 = np.float32([[0,0],[300,0],[0,300],[300,300]])
M = cv2.getPerspectiveTransform(pts1, pts2)
warped = cv2.warpPerspective(img, M, (300, 300))

Filtering

# Gaussian blur — reduces noise
blurred = cv2.GaussianBlur(img, (5, 5), sigmaX=1.0)

# Median blur — for salt-and-pepper noise
median = cv2.medianBlur(img, 5)

# Bilateral — blur while preserving edges
bilateral = cv2.bilateralFilter(img, 9, 75, 75)

# Custom kernel
import numpy as np
sharpen_kernel = np.array([[0, -1, 0], [-1, 5, -1], [0, -1, 0]])
sharpened = cv2.filter2D(img, -1, sharpen_kernel)

Edge Detection

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Canny — the most popular
edges = cv2.Canny(gray, threshold1=100, threshold2=200)

# Sobel — gradient
sobel_x = cv2.Sobel(gray, cv2.CV_64F, 1, 0, ksize=3)
sobel_y = cv2.Sobel(gray, cv2.CV_64F, 0, 1, ksize=3)
sobel_magnitude = np.sqrt(sobel_x**2 + sobel_y**2)

# Laplacian
laplacian = cv2.Laplacian(gray, cv2.CV_64F)

Thresholding

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Binary threshold
_, binary = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY)

# Adaptive (per-region)
adaptive = cv2.adaptiveThreshold(
    gray, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 11, 2
)

# Otsu — automatic optimal threshold
_, otsu = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)

Morphological operations

kernel = np.ones((5, 5), np.uint8)

# Erosion — shrinks (white dots)
eroded = cv2.erode(binary, kernel, iterations=1)

# Dilation — enlarges
dilated = cv2.dilate(binary, kernel, iterations=1)

# Opening = erosion + dilation (removes small noise)
opening = cv2.morphologyEx(binary, cv2.MORPH_OPEN, kernel)

# Closing = dilation + erosion (fills small holes)
closing = cv2.morphologyEx(binary, cv2.MORPH_CLOSE, kernel)

Contours

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, binary = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY)

contours, hierarchy = cv2.findContours(
    binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE
)

# Draw
img_with_contours = img.copy()
cv2.drawContours(img_with_contours, contours, -1, (0, 255, 0), 2)

# Bounding box for each contour
for contour in contours:
    x, y, w, h = cv2.boundingRect(contour)
    cv2.rectangle(img_with_contours, (x, y), (x+w, y+h), (255, 0, 0), 2)
    
    # Area
    area = cv2.contourArea(contour)
    
    # Perimeter
    perimeter = cv2.arcLength(contour, closed=True)
    
    # Approximated polygon
    epsilon = 0.02 * perimeter
    approx = cv2.approxPolyDP(contour, epsilon, closed=True)
    # len(approx) — number of corners (4 → rectangle)

Histogram and Equalization

import matplotlib.pyplot as plt

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Histogram
hist = cv2.calcHist([gray], [0], None, [256], [0, 256])
plt.plot(hist)
plt.show()

# Histogram equalization — improve contrast
equalized = cv2.equalizeHist(gray)

# CLAHE — adaptive histogram equalization (better)
clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
clahe_img = clahe.apply(gray)

Face Detection (classical — Haar cascade)

# Pretrained Haar cascade
face_cascade = cv2.CascadeClassifier(
    cv2.data.haarcascades + "haarcascade_frontalface_default.xml"
)
eye_cascade = cv2.CascadeClassifier(
    cv2.data.haarcascades + "haarcascade_eye.xml"
)

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, scaleFactor=1.1, minNeighbors=5)

for (x, y, w, h) in faces:
    cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), 2)
    roi = gray[y:y+h, x:x+w]
    eyes = eye_cascade.detectMultiScale(roi)
    for (ex, ey, ew, eh) in eyes:
        cv2.rectangle(img[y:y+h, x:x+w], (ex, ey), (ex+ew, ey+eh), (0, 255, 0), 2)

**Note:**For modern face detection, MediaPipeor DeepFaceare many times better.

Real-time processing from webcam

cap = cv2.VideoCapture(0)  # 0 = default webcam

while True:
    ret, frame = cap.read()
    if not ret:
        break
    
    # Grayscale
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    
    # Edges
    edges = cv2.Canny(gray, 100, 200)
    
    cv2.imshow("Original", frame)
    cv2.imshow("Edges", edges)
    
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

cap.release()
cv2.destroyAllWindows()

Backend integration

Image preprocessing service

from fastapi import FastAPI, UploadFile
from fastapi.responses import Response
import cv2
import numpy as np

app = FastAPI()

@app.post("/preprocess/document")
async def preprocess_document(file: UploadFile):
    """Prepare a document image for OCR."""
    contents = await file.read()
    arr = np.frombuffer(contents, np.uint8)
    img = cv2.imdecode(arr, cv2.IMREAD_COLOR)
    
    # 1. Grayscale
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    
    # 2. Denoise
    denoised = cv2.fastNlMeansDenoising(gray, h=10)
    
    # 3. Adaptive threshold
    thresh = cv2.adaptiveThreshold(
        denoised, 255,
        cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY,
        11, 2
    )
    
    # 4. Morphology
    kernel = np.ones((1, 1), np.uint8)
    cleaned = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)
    
    # Encode and return
    _, buf = cv2.imencode(".png", cleaned)
    return Response(content=buf.tobytes(), media_type="image/png")

Background removal (simple)

@app.post("/remove-background")
async def remove_background(file: UploadFile):
    contents = await file.read()
    arr = np.frombuffer(contents, np.uint8)
    img = cv2.imdecode(arr, cv2.IMREAD_COLOR)
    
    # GrabCut algorithm
    mask = np.zeros(img.shape[:2], np.uint8)
    bgd_model = np.zeros((1, 65), np.float64)
    fgd_model = np.zeros((1, 65), np.float64)
    
    rect = (10, 10, img.shape[1] - 10, img.shape[0] - 10)
    cv2.grabCut(img, mask, rect, bgd_model, fgd_model, 5, cv2.GC_INIT_WITH_RECT)
    
    mask2 = np.where((mask == 2) | (mask == 0), 0, 1).astype("uint8")
    result = img * mask2[:, :, np.newaxis]
    
    _, buf = cv2.imencode(".png", result)
    return Response(content=buf.tobytes(), media_type="image/png")

Modern alternative:****rembglibrary (U-Net based) gives many times better results.

Document Scanner-like Perspective Correction

def order_points(pts):
    rect = np.zeros((4, 2), dtype="float32")
    s = pts.sum(axis=1)
    rect[0] = pts[np.argmin(s)]
    rect[2] = pts[np.argmax(s)]
    diff = np.diff(pts, axis=1)
    rect[1] = pts[np.argmin(diff)]
    rect[3] = pts[np.argmax(diff)]
    return rect

def four_point_transform(image, pts):
    rect = order_points(pts)
    (tl, tr, br, bl) = rect
    widthA = np.linalg.norm(br - bl)
    widthB = np.linalg.norm(tr - tl)
    maxWidth = max(int(widthA), int(widthB))
    heightA = np.linalg.norm(tr - br)
    heightB = np.linalg.norm(tl - bl)
    maxHeight = max(int(heightA), int(heightB))
    
    dst = np.array([
        [0, 0],
        [maxWidth - 1, 0],
        [maxWidth - 1, maxHeight - 1],
        [0, maxHeight - 1]], dtype="float32")
    
    M = cv2.getPerspectiveTransform(rect, dst)
    warped = cv2.warpPerspective(image, M, (maxWidth, maxHeight))
    return warped

Resources

OpenCV docs — docs.opencv.org
PyImageSearch tutorials — hundreds of practical examples
“Learning OpenCV” — Adrian Kaehler
“Practical Python and OpenCV” — Adrian Rosebrock
OpenCV samples — opencv/samples on GitHub

🏋️ Exercises

🟢 Easy

Convert an image to Grayscale, HSV, LAB color spaces and plot.
Draw image contours with Canny edge detection.
Binarize a document image with adaptive threshold.

🟡 Medium

Document Scanner: “Flatten” a document from phone camera — contour detection + perspective transform.
Color picker: Upload an image, find the dominant 5 colors with K-Means.
Real-time face detection: with Haar cascade on webcam.

🔴 Hard

OCR pipeline preprocessor: FastAPI service — prepare document image for OCR (denoise, deskew, perspective correction).
Image deduplication: Find similar images with feature hashing (pHash, dHash).
Sport analytics: Track moving objects in video (background subtraction + tracking).

Capstone

notebooks/month-04/02_opencv_pipeline.ipynb:

Custom dataset (20 document images from phone)
Complete pipeline: detect → perspective correct → enhance → ready for OCR
Endpoint in FastAPI
Streamlit demo

✅ Checklist

I know the difference between OpenCV and PIL (BGR vs RGB)
Basic filtering (Gaussian, median, bilateral)
Edge detection (Canny)
Thresholding (adaptive, Otsu)
Finding and analyzing contours
Morphological operations
Perspective transformation
Real-time video processing
I wrote an image preprocessing endpoint

Moving on to YOLO and Object Detection.

Keyboard shortcuts

Backend to ML: 6 Oylik Roadmap