YOLO and Object Detection

🎯 Goal

After reading this chapter:

• You will be able to distinguish Object Detection from Image Classification
• You will know the YOLO ecosystem (v5, v8, v11)
• You will run inference with pretrained YOLO
• You will be able to fine-tune YOLO on your own dataset
• You will deploy an object detection service to production
• You will also get familiar with Segmentation and OCR

What to learn

• Detectionvs Classification vs Segmentation
• Bounding box — coordinates, IoU (Intersection over Union)
• Anchor boxes, anchor-free detection
• NMS (Non-Maximum Suppression)
• YOLO architectureevolution (v1 → v11)
• mAP (mean Average Precision) — detection metric
• Annotation formats — YOLO, COCO, Pascal VOC
• Ultralytics ecosystem — YOLOv8/v11 (PyTorch)
• Segmentation — instance (Mask R-CNN), semantic (DeepLab), SAM
• OCR — Tesseract, EasyOCR, PaddleOCR, TrOCR

Libraries

pip install ultralytics                    # YOLO
pip install supervision                    # Detection helpers
pip install easyocr                        # OCR (multi-language)
pip install paddleocr paddlepaddle         # PaddleOCR (best for many languages)
pip install segment-anything-py            # SAM (Meta)

Important topics

Detection metric — mAP

• **IoU (Intersection over Union):**degree of overlap between predicted and ground truth boxes
• IoU > 0.5typically a “true positive”
• AP (Average Precision)= area of precision-recall curve for one class
• mAP= average across all classes
• mAP@0.5:0.95= average over IoU thresholds 0.5..0.95 (COCO standard)

YOLO evolution

**Recommendation:**YOLOv8 or YOLOv11 — best choice for production (Ultralytics).

Annotation formats

YOLO format(simplest):

# image1.txt — har qator: class_id x_center y_center width height (normallashtirilgan 0..1)
0 0.5 0.5 0.3 0.4
2 0.7 0.3 0.1 0.2

COCO format(JSON):

{
  "images": [{"id": 1, "file_name": "image1.jpg", "width": 800, "height": 600}],
  "annotations": [
    {"image_id": 1, "category_id": 0, "bbox": [100, 200, 50, 80], "area": 4000}
  ],
  "categories": [{"id": 0, "name": "person"}]
}

Code examples

YOLOv8 — inference

from ultralytics import YOLO

# Pretrained model (COCO dataset — 80 classes)
model = YOLO("yolov8n.pt")  # n=nano, s=small, m=medium, l=large, x=xlarge

# Inference
results = model("path/to/image.jpg")

for result in results:
    boxes = result.boxes
    for box in boxes:
        x1, y1, x2, y2 = box.xyxy[0].tolist()
        conf = box.conf[0].item()
        cls = int(box.cls[0].item())
        cls_name = model.names[cls]
        print(f"{cls_name}: ({x1:.0f},{y1:.0f})-({x2:.0f},{y2:.0f}), conf={conf:.2f}")

# Visualization
result.show()
result.save("output.jpg")

Batch / video / webcam

# Batch images
results = model(["img1.jpg", "img2.jpg", "img3.jpg"])

# Video file
results = model("video.mp4", save=True, project="runs", name="detection")

# Webcam (real-time)
results = model(source=0, show=True)

# URL
results = model("https://ultralytics.com/images/bus.jpg")

Training on a custom dataset

1. Dataset preparation (YOLO format)

my_dataset/
├── data.yaml
├── images/
│   ├── train/
│   │   ├── img001.jpg
│   │   └── ...
│   ├── val/
│   └── test/
└── labels/
    ├── train/
    │   ├── img001.txt
    │   └── ...
    ├── val/
    └── test/

data.yaml:

path: ./my_dataset
train: images/train
val: images/val
test: images/test

nc: 3  # number of classes names: ['cat', 'dog', 'bird']

2. Training

from ultralytics import YOLO

model = YOLO("yolov8n.pt")  # start from transfer learning

results = model.train(
    data="my_dataset/data.yaml",
    epochs=100,
    imgsz=640,
    batch=16,
    device=0,  # GPU index, "cpu" or "mps"
    patience=20,  # early stopping
    project="runs/train",
    name="my_experiment",
)

# Best model
best_model = YOLO("runs/train/my_experiment/weights/best.pt")

3. Validation

metrics = best_model.val(data="my_dataset/data.yaml")
print(metrics.box.map)       # mAP@0.5:0.95
print(metrics.box.map50)     # mAP@0.5
print(metrics.box.map75)     # mAP@0.75

YOLO Segmentation

# Segmentation model (suffix `-seg`)
model = YOLO("yolov8n-seg.pt")
results = model("image.jpg")

for r in results:
    masks = r.masks  # segmentation masks
    if masks is not None:
        for mask in masks:
            mask_array = mask.data[0].cpu().numpy()  # (H, W) binary

SAM — Segment Anything Model (Meta)

from segment_anything import sam_model_registry, SamPredictor
import cv2

# Pretrained SAM
sam = sam_model_registry["vit_b"](checkpoint="sam_vit_b_01ec64.pth")
predictor = SamPredictor(sam)

# Image
image = cv2.imread("image.jpg")
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
predictor.set_image(image_rgb)

# Point or box prompt
masks, scores, _ = predictor.predict(
    point_coords=np.array([[500, 375]]),
    point_labels=np.array([1]),  # 1=foreground, 0=background
    multimask_output=True,
)
# masks shape: (3, H, W) — 3 variants

OCR — Tesseract

import pytesseract
import cv2

img = cv2.imread("text.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Default English
text = pytesseract.image_to_string(gray)

# Multi-language ("uzb" for Uzbek Latin)
text = pytesseract.image_to_string(gray, lang="uzb+eng+Russian")

# Configuration
custom_config = r"--oem 3 --psm 6"
text = pytesseract.image_to_string(gray, config=custom_config)

# With bounding boxes
data = pytesseract.image_to_data(gray, output_type=pytesseract.Output.DICT)

OCR — EasyOCR (modern)

import easyocr

# Multiple languages (uzbek not included, but Latin script may work)
reader = easyocr.Reader(['en', 'ru'])

result = reader.readtext("text.jpg")
for (bbox, text, prob) in result:
    print(f"Text:{text}, Confidence:{prob:.2f}")

OCR — PaddleOCR (best multi-language)

from paddleocr import PaddleOCR

ocr = PaddleOCR(use_angle_cls=True, lang="ru")  # "ru" or "en" for uzbek text

result = ocr.ocr("text.jpg")
for line in result[0]:
    bbox, (text, conf) = line
    print(text, conf)

Backend integration

Detection API (FastAPI + YOLO)

from fastapi import FastAPI, UploadFile
from fastapi.responses import JSONResponse, Response
from contextlib import asynccontextmanager
from ultralytics import YOLO
import cv2
import numpy as np

@asynccontextmanager
async def lifespan(app):
    app.state.model = YOLO("yolov8n.pt")
    yield

app = FastAPI(lifespan=lifespan)

@app.post("/detect")
async def detect(file: UploadFile, conf_threshold: float = 0.25):
    contents = await file.read()
    arr = np.frombuffer(contents, np.uint8)
    img = cv2.imdecode(arr, cv2.IMREAD_COLOR)
    
    results = app.state.model(img, conf=conf_threshold)
    detections = []
    
    for result in results:
        for box in result.boxes:
            x1, y1, x2, y2 = box.xyxy[0].tolist()
            detections.append({
                "class": app.state.model.names[int(box.cls[0])],
                "confidence": float(box.conf[0]),
                "bbox": [int(x1), int(y1), int(x2), int(y2)],
            })
    
    return {"detections": detections, "count": len(detections)}


@app.post("/detect-image")
async def detect_image(file: UploadFile):
    """Returns an annotated image."""
    contents = await file.read()
    arr = np.frombuffer(contents, np.uint8)
    img = cv2.imdecode(arr, cv2.IMREAD_COLOR)
    
    results = app.state.model(img)
    annotated = results[0].plot()
    
    _, buf = cv2.imencode(".jpg", annotated)
    return Response(content=buf.tobytes(), media_type="image/jpeg")

Async video processing (Celery)

from celery import Celery

celery_app = Celery("detection", broker="redis://localhost:6379")

@celery_app.task(bind=True)
def detect_video(self, video_path: str):
    model = YOLO("yolov8n.pt")
    cap = cv2.VideoCapture(video_path)
    
    total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
    all_detections = []
    
    for frame_idx in range(total_frames):
        ret, frame = cap.read()
        if not ret:
            break
        
        results = model(frame, verbose=False)
        frame_detections = []
        for box in results[0].boxes:
            frame_detections.append({
                "class": model.names[int(box.cls[0])],
                "confidence": float(box.conf[0]),
                "bbox": box.xyxy[0].tolist(),
            })
        all_detections.append({"frame": frame_idx, "detections": frame_detections})
        
        # Progress
        if frame_idx % 30 == 0:
            self.update_state(state="PROGRESS", 
                              meta={"current": frame_idx, "total": total_frames})
    
    cap.release()
    return all_detections

OCR + Detection combo (ID card scanner)

@app.post("/scan-id-card")
async def scan_id_card(file: UploadFile):
    contents = await file.read()
    arr = np.frombuffer(contents, np.uint8)
    img = cv2.imdecode(arr, cv2.IMREAD_COLOR)
    
    # 1. Detect ID card via custom YOLO model
    detections = id_card_detector(img)
    
    # 2. Crop ID card
    x1, y1, x2, y2 = detections[0]["bbox"]
    id_crop = img[y1:y2, x1:x2]
    
    # 3. Preprocess
    gray = cv2.cvtColor(id_crop, cv2.COLOR_BGR2GRAY)
    enhanced = cv2.adaptiveThreshold(gray, 255, 
        cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 11, 2)
    
    # 4. OCR
    text = pytesseract.image_to_string(enhanced, lang="uzb+eng")
    
    # 5. Parse fields (regex or ML)
    fields = parse_id_text(text)
    
    return fields

Resources

• Ultralytics docs — docs.ultralytics.com
• Roboflow Universe — datasets + pretrained models
• Supervision library — supervision.roboflow.com (detection helpers)
• Detectron2 — Facebook research detection framework
• MMDetection — OpenMMLab toolbox
• PaddleOCR docs — best multi-language OCR
• Segment Anything (SAM) — Meta research

🏋️ Exercises

🟢 Easy

1. Run inference on an image and video with pretrained YOLOv8n.
2. Change confidence threshold (0.1, 0.3, 0.7) and observe results.
3. Read a simple text image with Tesseract.

🟡 Medium

1. Custom YOLO: Label 100 images (1-2 classes) with Roboflow or Label Studio, fine-tune YOLO (with Colab GPU).
2. OCR comparison: Compare results of Tesseract, EasyOCR, PaddleOCR on the same image.
3. People counter: Count people in video in real-time.

🔴 Hard

1. Production CV pipeline: FastAPI + YOLO + Celery + Redis + Docker. Real-time video stream processing via WebSocket.
2. Custom OCR pipeline: Document page → text region detection (YOLO) → OCR (PaddleOCR) → JSON structured output.
3. SAM + YOLO combo: YOLO bounding box → segmentation mask with SAM → object-by-object analysis.

Capstone

notebooks/month-04/03_yolo_detection.ipynb:

• **Project:**Your own dataset (50-100 phone photos) — e.g., local signs (road signs, shop signs, fruits)
• Annotation in Roboflow
• YOLOv8 fine-tune (Colab GPU)
• mAP 80%+
• Deploy FastAPI service in Docker

✅ Checklist

• I know the difference between Detection and Classification
• I understand IoU and mAP metrics
• I can run YOLO inference (image, video, webcam)
• I know how to fine-tune YOLO on a custom dataset
• Familiar with Segmentation (YOLO-seg, SAM)
• OCR (any of the 3 libraries)
• I know how to create a detection service in FastAPI
• Async video processing (Celery)

Moving on to NLP basics.