Month 5 — Exercises set

🟢 Easy

LLM Fundamentals

1. Compare tokens for English and Uzbek text with tiktoken.
2. 5 models (GPT-4o-mini, Claude Haiku, Gemini Flash, Llama 3.1, Mistral) with the same question.
3. Temperature 0, 0.5, 1.5 — observe answer differences.

Prompt Engineering

1. Zero-shot, few-shot, CoT — for the same task.
2. Structured Pydantic output with Instructor.
3. Prompt + validation for JSON output.

APIs

1. OpenAI streaming chat.
2. Anthropic prompt caching.
3. Function calling — 3 tools.

Vector DB

1. 100 documents in ChromaDB.
2. Qdrant Docker setup.
3. pgvector Postgres extension.

RAG

1. Naive RAG — 10 documents, query.
2. Citation — [Source N] format.
3. Compare chunking strategies.

Agents

1. Pydantic AI agent + 3 tools.
2. CrewAI hello world.
3. LangGraph simple workflow.

Fine-tuning

1. Load pretrained Llama 1B.
2. 50 synthetic datasets (with GPT).
3. Understand LoRA config syntax.

🟡 Medium

Real projects

1. Multi-turn chatbot: store history, manage context window.
2. RAG over Wikipedia: 100 Uzbek Wikipedia articles.
3. PDF Q&A bot: PyPDF + Qdrant + Streamlit.
4. Code review agent: GitHub PR diff → suggestions.
5. Email summarizer: 50 emails → daily digest.

Advanced techniques

1. Multi-query RAG: with query expansion.
2. HyDE: hypothetical embeddings.
3. Hybrid search: dense + BM25.
4. Reranking: with cross-encoder.
5. Multi-agent: CrewAI 3-agent system.

Fine-tuning

1. TinyLlama: QLoRA with Uzbek instruction dataset (Colab).
2. OpenAI fine-tuning: customer support classifier ($1 budget).
3. Synthetic data: 500+ training pairs with GPT-4.

🔴 Hard (Production)

1. Documentation Q&A Bot

Requirements:

• 100+ documents (PDF, markdown, websites) ingestion
• Qdrant + FastAPI + Celery
• Multi-query + reranking
• Citation and source links
• Streamlit UI
• Langfuse observability
• Cost tracking per user

2. AI Customer Support Agent

Requirements:

• Telegram bot (aiogram)
• Multi-turn conversation
• Tools: FAQ search, order lookup, refund process, escalate to human
• LangGraph workflow
• Postgres memory
• Sentiment-based routing
• Admin dashboard

3. RAG Evaluation Framework

Requirements:

• Create test set (100+ Q&A pairs)
• Automated evaluation with RAGAS
• A/B testing framework
• Continuous improvement loop
• Grafana dashboard

4. Domain-specific Fine-tuning Pipeline

Requirements:

• Data collection + cleaning
• Synthetic data augmentation
• QLoRA fine-tuning (Llama 3.1 8B)
• vLLM serving
• Benchmark (vs base model)
• Production rollout strategy

Mini-projects

Mini-project 1: Voice-to-Text Meeting Assistant

• Whisper (audio transcription)
• LLM summarization
• Action items extraction
• Slack integration

Mini-project 2: Code Review Bot

• GitHub webhook
• Diff parsing
• LLM analysis (security, performance)
• Inline PR comments

Mini-project 3: Personal Knowledge Base

• Notion + Obsidian export
• Vector DB ingestion
• “Second brain” chatbot
• Smart search

Mini-project 4: Uzbek Government Documents Chatbot

• lex.uz, data.gov.uz scraping
• Multi-language (uz/ru)
• Citation
• Legal disclaimer

Quiz

LLM

1. Token, context window, temperature, top_p — explain each one.
2. Pretraining, SFT, RLHF — what order?
3. What is hallucination and how to reduce it?
4. Proprietary vs Open Source LLM — selection criteria?
5. How does prompt caching work?

Prompt Engineering

1. Zero-shot, few-shot, CoT — when which?
2. Patterns for structured output (JSON)?
3. Prompt injection — risk and defenses?
4. Self-consistency technique?
5. ReAct pattern intuition?

RAG

1. Difference between RAG and Fine-tuning?
2. Chunking strategies trade-offs?
3. How does the HNSW algorithm work?
4. What is Hybrid search?
5. Why does cross-encoder reranking lead to improvement?

Agents

1. Difference between Agent and LLM call?
2. ReAct pattern — Thought/Action/Observation?
3. When is multi-agent needed?
4. What is MCP (Model Context Protocol)?
5. Agent security — sandbox patterns?

Fine-tuning

1. LoRA mathematical intuition?
2. QLoRA — why 4-bit?
3. Synthetic data generation strategies?
4. RAG vs Fine-tuning — when which?
5. Why is vLLM fast in production?

✅ Month 5 final checklist

• I use LLM APIs (OpenAI, Anthropic)
• I know prompt engineering techniques
• Structured output (Instructor, Pydantic AI)
• Familiar with Vector DB (at least 2)
• I built a full RAG pipeline
• I wrote an AI Agent (tool use)
• I tried a small fine-tuning with LoRA
• I deployed to production (FastAPI + Docker)
• Langfuse / observability
• Capstone project (chatbot/RAG)
• LinkedIn post

Congratulations! Month 6 — MLOps and Production — the most important month for your main goal.