YOLO и Object Detection

🎯 Цель

После прочтения этой главы:

• Можете отличать задачу Object Detection от Image Classification
• Знаете экосистему YOLO (v5, v8, v11)
• Делаете inference с pretrained YOLO
• Можете fine-tune YOLO для своего dataset
• Deploy object detection сервис в production
• Знакомы с Segmentation и OCR

Что нужно изучить

• Detectionvs Classification vs Segmentation
• Bounding box — coordinates, IoU (Intersection over Union)
• Anchor boxes, anchor-free detection
• NMS (Non-Maximum Suppression)
• Эволюция архитектуры YOLO(v1 → v11)
• mAP (mean Average Precision) — detection metric
• Annotation formats — YOLO, COCO, Pascal VOC
• Экосистема Ultralytics — YOLOv8/v11 (PyTorch)
• Segmentation — instance (Mask R-CNN), semantic (DeepLab), SAM
• OCR — Tesseract, EasyOCR, PaddleOCR, TrOCR

Библиотеки

pip install ultralytics                    # YOLO
pip install supervision                    # Detection helpers
pip install easyocr                        # OCR (multi-language)
pip install paddleocr paddlepaddle         # PaddleOCR (best for many languages)
pip install segment-anything-py            # SAM (Meta)

Важные темы

Detection metric — mAP

• **IoU (Intersection over Union):**степень перекрытия predicted и ground truth box
• IoU > 0.5обычно «true positive»
• AP (Average Precision)= area под precision-recall curve для одного class
• mAP= среднее по всем class
• mAP@0.5:0.95= среднее по IoU threshold от 0.5..0.95 (COCO standard)

Эволюция YOLO

**Совет:**YOLOv8 или YOLOv11 — лучший выбор для production (Ultralytics).

Annotation форматы

YOLO format(самый простой):

# image1.txt — har qator: class_id x_center y_center width height (normallashtirilgan 0..1)
0 0.5 0.5 0.3 0.4
2 0.7 0.3 0.1 0.2

COCO format(JSON):

{
  "images": [{"id": 1, "file_name": "image1.jpg", "width": 800, "height": 600}],
  "annotations": [
    {"image_id": 1, "category_id": 0, "bbox": [100, 200, 50, 80], "area": 4000}
  ],
  "categories": [{"id": 0, "name": "person"}]
}

Примеры кода

YOLOv8 — inference

from ultralytics import YOLO

# Pretrained model (COCO dataset — 80 class)
model = YOLO("yolov8n.pt")  # n=nano, s=small, m=medium, l=large, x=xlarge

# Inference
results = model("path/to/image.jpg")

for result in results:
    boxes = result.boxes
    for box in boxes:
        x1, y1, x2, y2 = box.xyxy[0].tolist()
        conf = box.conf[0].item()
        cls = int(box.cls[0].item())
        cls_name = model.names[cls]
        print(f"{cls_name}: ({x1:.0f}, {y1:.0f})-({x2:.0f}, {y2:.0f}), conf={conf:.2f}")

# Визуализация
result.show()
result.save("output.jpg")

Batch / video / webcam

# Batch images
results = model(["img1.jpg", "img2.jpg", "img3.jpg"])

# Video file
results = model("video.mp4", save=True, project="runs", name="detection")

# Webcam (real-time)
results = model(source=0, show=True)

# URL
results = model("https://ultralytics.com/images/bus.jpg")

Training для custom dataset

1. Подготовка dataset (YOLO format)

my_dataset/
├── data.yaml
├── images/
│   ├── train/
│   │   ├── img001.jpg
│   │   └── ...
│   ├── val/
│   └── test/
└── labels/
    ├── train/
    │   ├── img001.txt
    │   └── ...
    ├── val/
    └── test/

data.yaml:

path: ./my_dataset
train: images/train
val: images/val
test: images/test

nc: 3  # number of classes
names: ['cat', 'dog', 'bird']

2. Training

from ultralytics import YOLO

model = YOLO("yolov8n.pt")  # начать с transfer learning

results = model.train(
    data="my_dataset/data.yaml",
    epochs=100,
    imgsz=640,
    batch=16,
    device=0,  # GPU index, "cpu" или "mps"
    patience=20,  # early stopping
    project="runs/train",
    name="my_experiment",
)

# Best model
best_model = YOLO("runs/train/my_experiment/weights/best.pt")

3. Validation

metrics = best_model.val(data="my_dataset/data.yaml")
print(metrics.box.map)       # mAP@0.5:0.95
print(metrics.box.map50)     # mAP@0.5
print(metrics.box.map75)     # mAP@0.75

YOLO Segmentation

# Модель segmentation (suffix `-seg`)
model = YOLO("yolov8n-seg.pt")
results = model("image.jpg")

for r in results:
    masks = r.masks  # segmentation masks
    if masks is not None:
        for mask in masks:
            mask_array = mask.data[0].cpu().numpy()  # (H, W) binary

SAM — Segment Anything Model (Meta)

from segment_anything import sam_model_registry, SamPredictor
import cv2

# Pretrained SAM
sam = sam_model_registry["vit_b"](checkpoint="sam_vit_b_01ec64.pth")
predictor = SamPredictor(sam)

# Image
image = cv2.imread("image.jpg")
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
predictor.set_image(image_rgb)

# Point или box prompt
masks, scores, _ = predictor.predict(
    point_coords=np.array([[500, 375]]),
    point_labels=np.array([1]),  # 1=foreground, 0=background
    multimask_output=True,
)
# masks shape: (3, H, W) — 3 варианта

OCR — Tesseract

import pytesseract
import cv2

img = cv2.imread("text.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Default English
text = pytesseract.image_to_string(gray)

# Multi-language (для узбекской латиницы "uzb")
text = pytesseract.image_to_string(gray, lang="uzb+eng+rus")

# Конфигурация
custom_config = r"--oem 3 --psm 6"
text = pytesseract.image_to_string(gray, config=custom_config)

# С bounding box
data = pytesseract.image_to_data(gray, output_type=pytesseract.Output.DICT)

OCR — EasyOCR (современный)

import easyocr

# Несколько языков (узбекский не добавлен, но латиница может работать)
reader = easyocr.Reader(['en', 'ru'])

result = reader.readtext("text.jpg")
for (bbox, text, prob) in result:
    print(f"Text: {text}, Confidence: {prob:.2f}")

OCR — PaddleOCR (лучший multi-language)

from paddleocr import PaddleOCR

ocr = PaddleOCR(use_angle_cls=True, lang="ru")  # для узбекских надписей "ru" или "en"

result = ocr.ocr("text.jpg")
for line in result[0]:
    bbox, (text, conf) = line
    print(text, conf)

Интеграция с backend

Detection API (FastAPI + YOLO)

from fastapi import FastAPI, UploadFile
from fastapi.responses import JSONResponse, Response
from contextlib import asynccontextmanager
from ultralytics import YOLO
import cv2
import numpy as np

@asynccontextmanager
async def lifespan(app):
    app.state.model = YOLO("yolov8n.pt")
    yield

app = FastAPI(lifespan=lifespan)

@app.post("/detect")
async def detect(file: UploadFile, conf_threshold: float = 0.25):
    contents = await file.read()
    arr = np.frombuffer(contents, np.uint8)
    img = cv2.imdecode(arr, cv2.IMREAD_COLOR)
    
    results = app.state.model(img, conf=conf_threshold)
    detections = []
    
    for result in results:
        for box in result.boxes:
            x1, y1, x2, y2 = box.xyxy[0].tolist()
            detections.append({
                "class": app.state.model.names[int(box.cls[0])],
                "confidence": float(box.conf[0]),
                "bbox": [int(x1), int(y1), int(x2), int(y2)],
            })
    
    return {"detections": detections, "count": len(detections)}


@app.post("/detect-image")
async def detect_image(file: UploadFile):
    """Возвращает annotated изображение."""
    contents = await file.read()
    arr = np.frombuffer(contents, np.uint8)
    img = cv2.imdecode(arr, cv2.IMREAD_COLOR)
    
    results = app.state.model(img)
    annotated = results[0].plot()
    
    _, buf = cv2.imencode(".jpg", annotated)
    return Response(content=buf.tobytes(), media_type="image/jpeg")

Async video processing (Celery)

from celery import Celery

celery_app = Celery("detection", broker="redis://localhost:6379")

@celery_app.task(bind=True)
def detect_video(self, video_path: str):
    model = YOLO("yolov8n.pt")
    cap = cv2.VideoCapture(video_path)
    
    total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
    all_detections = []
    
    for frame_idx in range(total_frames):
        ret, frame = cap.read()
        if not ret:
            break
        
        results = model(frame, verbose=False)
        frame_detections = []
        for box in results[0].boxes:
            frame_detections.append({
                "class": model.names[int(box.cls[0])],
                "confidence": float(box.conf[0]),
                "bbox": box.xyxy[0].tolist(),
            })
        all_detections.append({"frame": frame_idx, "detections": frame_detections})
        
        # Progress
        if frame_idx % 30 == 0:
            self.update_state(state="PROGRESS", 
                              meta={"current": frame_idx, "total": total_frames})
    
    cap.release()
    return all_detections

OCR + Detection combo (ID card scanner)

@app.post("/scan-id-card")
async def scan_id_card(file: UploadFile):
    contents = await file.read()
    arr = np.frombuffer(contents, np.uint8)
    img = cv2.imdecode(arr, cv2.IMREAD_COLOR)
    
    # 1. Detect ID card via custom YOLO model
    detections = id_card_detector(img)
    
    # 2. Crop ID card
    x1, y1, x2, y2 = detections[0]["bbox"]
    id_crop = img[y1:y2, x1:x2]
    
    # 3. Preprocess
    gray = cv2.cvtColor(id_crop, cv2.COLOR_BGR2GRAY)
    enhanced = cv2.adaptiveThreshold(gray, 255, 
        cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 11, 2)
    
    # 4. OCR
    text = pytesseract.image_to_string(enhanced, lang="uzb+eng")
    
    # 5. Parse fields (regex или ML)
    fields = parse_id_text(text)
    
    return fields

Ресурсы

• Ultralytics docs — docs.ultralytics.com
• Roboflow Universe — datasets + pretrained models
• Supervision library — supervision.roboflow.com (detection helpers)
• Detectron2 — Facebook research detection framework
• MMDetection — OpenMMLab toolbox
• PaddleOCR docs — best multi-language OCR
• Segment Anything (SAM) — Meta research

🏋️ Упражнения

🟢 Easy

1. Inference на изображениях и видео через YOLOv8n pretrained.
2. Меняя confidence threshold (0.1, 0.3, 0.7), посмотрите результаты.
3. Прочитайте простое изображение с текстом через Tesseract.

🟡 Medium

1. Custom YOLO: разметьте 100 изображений (1-2 class) в Roboflow или Label Studio, fine-tune YOLO (с Colab GPU).
2. OCR comparison: сравните результаты Tesseract, EasyOCR, PaddleOCR на одном изображении.
3. People counter: real-time подсчёт людей в видео.

🔴 Hard

1. Production CV pipeline: FastAPI + YOLO + Celery + Redis + Docker. Real-time video stream processing через WebSocket.
2. Custom OCR pipeline: страница документа → text region detection (YOLO) → OCR (PaddleOCR) → JSON structured output.
3. SAM + YOLO combo: YOLO bounding box → segmentation mask через SAM → object-by-object analysis.

Capstone

notebooks/month-04/03_yolo_detection.ipynb:

• **Проект:**Собственный dataset (50-100 изображений с телефона) — например, местные знаки (дорожные знаки, вывески магазинов, фрукты)
• Annotation в Roboflow
• YOLOv8 fine-tune (Colab GPU)
• mAP 80%+
• Deploy FastAPI сервиса в Docker

✅ Чек-лист

• Знаю разницу между Detection и Classification
• Понимаю metric IoU и mAP
• Могу делать YOLO inference (image, video, webcam)
• Знаю, как fine-tune YOLO для custom dataset
• Знаком с Segmentation (YOLO-seg, SAM)
• OCR (одна из 3 библиотек)
• Знаю, как создать detection сервис в FastAPI
• Async video processing (Celery)

Переходим к Основы NLP.