Introduction to Computer Vision

🎯 Goal

After reading this chapter:

• You will know the 5 main types of Computer Vision tasks
• You will be able to choose suitable pretrained models for each task
• You will know the concepts needed for working with images/video
• You will be able to plan a domain-appropriate CV pipeline

What to learn

• CV task types — classification, detection, segmentation, OCR, pose, generation
• Image fundamentals — pixel, channels, color spaces (RGB, BGR, HSV, Grayscale)
• Image formats — JPEG, PNG, WebP, TIFF
• Pretrained ecosystem for CV — torchvision, timm, MMDetection, Detectron2, Ultralytics
• Edge cases — rotation, occlusion, lighting, scale

The 5 main types of CV tasks

1. Image Classification

• One image → one label (or multiple labels, multi-label)
• Models: ResNet, EfficientNet, ViT, ConvNeXt
• Examples: spam image, disease type, product category

2. Object Detection

• One image → multiple bounding boxes + labels + confidence
• Models: YOLO, Faster R-CNN, DETR
• Examples: counting vehicles, security threats

3. Semantic / Instance / Panoptic Segmentation

• Pixel-level classification
• Models: U-Net, Mask R-CNN, SAM (Segment Anything Model)
• Examples: medical imaging, satellite analysis

4. OCR (Optical Character Recognition)

• Image → text
• Models: Tesseract, EasyOCR, PaddleOCR, TrOCR
• Examples: ID cards, documents, receipts

5. Pose / Keypoint Estimation

• Finding human body or object keypoints
• Models: MediaPipe, OpenPose, MMPose
• Examples: sports analytics, AR filters

6. Generative (bonus)

• Creating/modifying images
• Models: Stable Diffusion, DALL-E, ControlNet
• Examples: marketing assets, design tools

Image fundamentals

Pixels and Channels

RGB rasm (3 channel):
shape = (height, width, 3)
har pixel: [R, G, B] qiymatlari, har biri [0..255] (uint8) yoki [0..1] (float)

Grayscale (1 channel):
shape = (height, width)
har pixel: [0..255] (yorqinlik darajasi)

OpenCV o'qiganda BGR (not RGB)!
PIL/torchvision RGB ishlatadi

Color spaces

Image formats — when which?

Main libraries

pip install opencv-python pillow numpy matplotlib
pip install torch torchvision timm
pip install ultralytics                  # YOLO
pip install easyocr paddleocr            # OCR
pip install mediapipe                    # Pose, hand tracking
pip install albumentations               # Augmentation

Code examples

Image loading and inspection

import cv2
import numpy as np
from PIL import Image
import matplotlib.pyplot as plt

# OpenCV (BGR)
img_cv = cv2.imread("photo.jpg")
print(img_cv.shape)         # (H, W, 3)
print(img_cv.dtype)         # uint8
img_rgb = cv2.cvtColor(img_cv, cv2.COLOR_BGR2RGB)

# PIL (RGB)
img_pil = Image.open("photo.jpg")
print(img_pil.size)         # (W, H) — note: order is different!

# matplotlib (expects RGB)
plt.imshow(img_rgb)
plt.axis("off")
plt.show()

Basic operations on images

# Resize
resized = cv2.resize(img_cv, (224, 224))

# Crop
cropped = img_cv[100:400, 200:500]  # [y1:y2, x1:x2]

# Rotation
h, w = img_cv.shape[:2]
M = cv2.getRotationMatrix2D((w/2, h/2), angle=45, scale=1.0)
rotated = cv2.warpAffine(img_cv, M, (w, h))

# Flip
flipped = cv2.flip(img_cv, 1)  # 1=horizontal, 0=vertical, -1=both

# Color conversion
gray = cv2.cvtColor(img_cv, cv2.COLOR_BGR2GRAY)
hsv = cv2.cvtColor(img_cv, cv2.COLOR_BGR2HSV)

Choosing a CV pipeline — decision tree

Your task?
│
├── "What is this image?"
│   → Image Classification (ResNet/EfficientNet/ViT)
│
├── "Where and what is in the image?"
│   → Object Detection (YOLO, Faster R-CNN)
│
├── "Which object does each pixel belong to?"
│   → Segmentation (U-Net, SAM)
│
├── "What text is written in this image?"
│   → OCR (Tesseract, EasyOCR, PaddleOCR)
│
├── "Find keypoints on a person"
│   → Pose Estimation (MediaPipe, OpenPose)
│
└── "Create/modify an image"
    → Generative (Stable Diffusion)

Backend integration — common patterns

1. Image upload endpoint

from fastapi import FastAPI, UploadFile
from PIL import Image
import io

app = FastAPI()

@app.post("/process-image")
async def process_image(file: UploadFile):
    # Validation
    if not file.content_type.startswith("image/"):
        return {"error": "Not an image"}
    
    # Read
    contents = await file.read()
    img = Image.open(io.BytesIO(contents)).convert("RGB")
    
    # Validate size
    if img.size[0] > 4000 or img.size[1] > 4000:
        return {"error": "Image too large"}
    
    # Process (CV pipeline) # ...
    
    return {"status": "ok", "size": img.size}

2. Loading an image from URL

import httpx
from PIL import Image
import io

@app.post("/process-url")
async def process_url(url: str):
    async with httpx.AsyncClient(timeout=10) as client:
        response = await client.get(url)
    
    img = Image.open(io.BytesIO(response.content)).convert("RGB")
    # ...

3. Stream/Video processing

import cv2

def process_video(video_path: str, output_path: str):
    cap = cv2.VideoCapture(video_path)
    fps = cap.get(cv2.CAP_PROP_FPS)
    w = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    h = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
    
    out = cv2.VideoWriter(output_path, cv2.VideoWriter_fourcc(*"mp4v"), fps, (w, h))
    
    while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            break
        
        # Apply model
        processed = some_model_inference(frame)
        out.write(processed)
    
    cap.release()
    out.release()

4. Async processing (Celery)

@celery_app.task
def process_image_async(image_path: str):
    img = cv2.imread(image_path)
    
    # Heavy processing
    result = run_yolo(img)
    
    # Save result
    output_path = image_path.replace(".jpg", "_processed.jpg")
    cv2.imwrite(output_path, result)
    
    return {"output": output_path}

@app.post("/process-async")
async def process_async(file: UploadFile):
    # Save uploaded file
    path = f"/tmp/{uuid.uuid4()}.jpg"
    with open(path, "wb") as f:
        f.write(await file.read())
    
    # Queue task
    task = process_image_async.delay(path)
    return {"task_id": task.id}

Resources

• PyImageSearch — pyimagesearch.com — the best CV blog
• OpenCV docs — docs.opencv.org
• CS231n(Stanford) — CV theory
• Roboflow — datasets and training (no-code)
• MMDetection / Detectron2 — production-grade detection frameworks
• HuggingFace Vision — pretrained vision models

🏋️ Exercises

🟢 Easy

1. Load an image (OpenCV and PIL), output the shape and format.
2. Convert RGB → Grayscale, RGB → HSV and visualize.
3. Resize the image to 224x224 and save.

🟡 Medium

1. Image gallery API: Upload image in FastAPI, create thumbnail (200x200), extract EXIF metadata.
2. Color analysis: Find dominant colors from an image with K-Means (from Month 2).
3. Pretrained classifier: Top-5 predictions for an image with a torchvision model.

🔴 Hard

1. CV Pipeline Service: FastAPI + Celery + Redis. Endpoints:

• Upload image
• Resize / convert format
• Apply pretrained model (classification/detection)
• Async with webhook callback

2. Real-time webcam: FastAPI WebSocket + browser webcam → YOLO on server → return bounding box JSON.

Capstone

notebooks/month-04/01_cv_intro.ipynb:

• Load a custom dataset (200+ images) or get from Kaggle
• Apply 5 different CV tasks to one dataset:
• Classification (pretrained)
• Detection (YOLO)
• Segmentation (SAM)
• OCR (on images with text)
• Pose (on images with people)

✅ Checklist

• I know 5+ main CV tasks
• I know the difference between image formats and color spaces
• I know the difference between OpenCV and PIL
• I know when to choose which pretrained model
• I can plan an async image processing pipeline

Moving on to Working with OpenCV.