Introduction to MLOps
🎯 Goal
After reading this chapter:
- You know what MLOps is and how it differs from DevOps
- You know the complete picture of the ML lifecycle
- You can assess MLOps maturity levels and what level a company is at
- You know the most important tool ecosystem
What to learn
- MLOps conceptand origin
- ML Lifecycle — data → train → deploy → monitor
- DevOps vs DataOps vs MLOps
- ML Maturity Levels(Google MLOps levels 0-2)
- MLOps challenges — reproducibility, drift, scaling
- Tool landscape — open source vs managed services
- Team structure — Data Engineer, ML Engineer, Data Scientist
MLOps — what and why?
Lifecycle of a classical ML project
Data Scientist in Jupyter notebook:
1. Gets data with Pandas
2. Trains model
3. Saves "model.pkl"
4. Says: "Put it in production"
Backend Engineer:
1. Loads .pkl
2. Adds to FastAPI
3. Deploys
4. All good... for a few weeks
Two months later:
- Model accuracy has dropped (drift!)
- Data scientist keeps sending new models (new format)
- No one can reproduce the original result
- No audit logs
- No A/B test either
- If production makes an error, nobody notices
MLOps solves these problems.
DevOps vs MLOps
DevOps:
Code → Test → Build → Deploy → Monitor
MLOps:
Data → Validate → Train → Test → Register → Deploy → Monitor → Retrain
↑ ↓
└──────────────── Feedback loop ────────────────────────────────────┘
Main differences:
- Dataalso needs versioning (just like code)
- Model — is an artifact, a new one each retraining
- Performanceundergoes degradationover time (drift)
- Reproducibility — getting the same result again is hard (randomness, data changes)
- Testing — accuracy or business metrics
Google MLOps Maturity Levels
Level 0 — Manual
Data scientist: kompyuterda manual
Production: oddiy script, manual deploy
Monitoring: yo'q yoki kam
✅ New projects, MVP, small companies ❌ Not production-grade
Level 1 — ML pipeline automation
Training pipeline avtomatik (Airflow yoki shunga o'xshash)
Data validation, model validation, automated retraining
Hali deployment manual yoki semi-automatic
✅ Medium-sized companies ✅ Most real-world ML projects are at this level
Level 2 — CI/CD pipeline
Hammasi avtomatik:
- Code/data CI: validation, testing
- ML pipeline CD: yangi model avtomatik deploy
- Monitoring orqali retraining trigger
- A/B testing infrastructure
✅ Mature MLOps culture (Google, Netflix, Uber)
MLOps Tool Ecosystem (2024-2026)
Experiment Tracking
- MLflow ⭐⭐⭐⭐⭐ — open source, most popular
- Weights & Biases — managed, excellent UI
- Neptune.ai — managed alternative
- Comet — alternative
Data Versioning
- DVC ⭐⭐⭐⭐⭐ — Git for data
- LakeFS — data warehouse
- Pachyderm — kubernetes-native
- Delta Lake — Databricks ecosystem
Feature Store
- Feast ⭐⭐⭐⭐ — open source
- Tecton — managed (originated from Feast)
- Hopsworks — alternative
Model Serving
- FastAPI+ custom — simple, flexible
- TorchServe — PyTorch native
- TensorFlow Serving — TF native
- BentoML ⭐⭐⭐⭐ — Python-friendly, flexible
- Ray Serve — distributed
- Triton (NVIDIA) — production-grade GPU serving
- vLLM — LLM-specific, very fast
Workflow Orchestration
- Apache Airflow ⭐⭐⭐⭐⭐ — the bible
- Prefect — modern, Pythonic
- Dagster — data-aware
- Kubeflow Pipelines — k8s-native
- Metaflow — from Netflix
Monitoring
- Prometheus + Grafana — infrastructure
- Evidently AI ⭐⭐⭐⭐⭐ — data/model drift
- WhyLabs — managed alternative
- Arize, Fiddler — enterprise
Deployment Platforms
- Kubernetes+ custom — flexibility
- AWS SageMaker — managed
- GCP Vertex AI — managed
- Azure ML — managed
- Databricks — unified analytics
LLMOps (specific to LLM)
- Langfuse ⭐⭐⭐⭐⭐ — open source observability
- LangSmith — LangChain ecosystem
- Helicone — proxy + analytics
- Phoenix(Arize) — open source
ML Lifecycle in detail
1. PROBLEM DEFINITION
- Business problem → ML problem
- Success metrics (online + offline)
2. DATA COLLECTION
- Source identification
- Sampling strategy
- Privacy/compliance
3. DATA PREPARATION
- Cleaning, transformation
- Feature engineering
- Train/val/test split
- Data versioning (DVC)
4. MODEL DEVELOPMENT
- Algorithm selection
- Hyperparameter tuning
- Experiment tracking (MLflow)
- Reproducibility
5. MODEL EVALUATION
- Offline metrics
- Bias/fairness analysis
- Edge cases testing
- Stakeholder review
6. MODEL DEPLOYMENT
- Containerization (Docker)
- Orchestration (K8s)
- Serving framework (FastAPI/BentoML)
- API design
7. MODEL MONITORING
- Performance metrics
- Data drift detection
- Concept drift detection
- Business KPIs
8. CONTINUOUS IMPROVEMENT
- A/B testing
- Shadow deployment
- Champion-challenger
- Automated retraining
Typical MLOps project structure
ml_project/
├── data/ # Raw data (DVC tracked, not git)
│ ├── raw/
│ ├── interim/
│ └── processed/
├── notebooks/ # Exploration
│ └── 01_eda.ipynb
├── src/ # Source code
│ ├── data/
│ │ ├── make_dataset.py
│ │ └── validate.py
│ ├── features/
│ │ └── build_features.py
│ ├── models/
│ │ ├── train.py
│ │ ├── predict.py
│ │ └── evaluate.py
│ └── api/
│ └── main.py # FastAPI
├── tests/
│ ├── test_data.py
│ ├── test_features.py
│ └── test_model.py
├── configs/
│ ├── config.yaml
│ └── model_v1.yaml
├── dvc.yaml # DVC pipeline
├── params.yaml # Hyperparameters
├── Dockerfile
├── docker-compose.yml
├── .github/workflows/
│ ├── ci.yml
│ ├── train.yml
│ └── deploy.yml
├── k8s/ # Kubernetes manifests
│ ├── deployment.yaml
│ └── service.yaml
├── airflow/dags/ # Workflow orchestration
│ └── retrain_dag.py
├── monitoring/
│ ├── prometheus.yml
│ └── grafana_dashboard.json
├── requirements.txt
├── pyproject.toml
├── README.md
└── Makefile # Common commands
Backend dev → MLOps Engineer: skill mapping
You already have:
- ✅ REST API (FastAPI, DRF)
- ✅ Docker, docker-compose
- ✅ PostgreSQL, Redis
- ✅ Celery (async tasks)
- ✅ CI/CD (GitHub Actions / GitLab CI)
- ✅ Linux, basic Kubernetes
- ✅ Monitoring (Prometheus/Grafana)
- ✅ Git workflow
- ✅ Testing (pytest)
New things to learn:
- ML lifecycle thinking
- Experiment tracking (MLflow)
- Data versioning (DVC)
- Model serving frameworks (BentoML)
- Drift detection (Evidently)
- Workflow orchestration (Airflow)
- Feature stores (Feast)
Learning these 6 things in 4 weeks is realistic.
Resources
Books (must)
- “Designing Machine Learning Systems” — Chip Huyen (the best MLOps book)
- “Machine Learning Engineering” — Andriy Burkov
- “Building Machine Learning Pipelines” — Hannes Hapke & Catherine Nelson
- “Practical MLOps” — Noah Gift
Online courses (must)
- MLOps Zoomcamp — DataTalks.Club (github.com/DataTalksClub/mlops-zoomcamp) — MUST DO, free
- Made With ML — Goku Mohandas (free)
- Full Stack Deep Learning — Berkeley course
- DeepLearning.AI MLOps Specialization — Andrew Ng
Blogs
- Chip Huyen’s blog — huyenchip.com/blog
- Eugene Yan’s blog — eugeneyan.com
- Neptune.ai blog — MLOps articles
- Towards Data Science — MLOps section
Communities
- MLOps Community Slack — mlops.community
- DataTalks.Club Slack
- Reddit r/MachineLearning, r/MLOps
🏋️ Exercises
🟢 Easy
- Google the 10 tools in the tool landscape above, write a short description of each.
- Assess what MLOps maturity level your company/project is at.
- Explain the 8 stages of the ML Lifecycle in your own words.
🟡 Medium
- Write an ML integration plan for an existing Django/FastAPI project (where, how, which tools).
- Conversation with ChatGPT or Claude — “20 questions and answers in an MLOps Engineer interview”.
- Analyze 5 “MLOps Engineer” vacancies from job posting sites, what tools are required.
🔴 Hard
- Plan template: Create a complete ML Engineering Document for an ML project (problem statement → success metrics → architecture).
- Tool comparison: BentoML vs TorchServe vs Triton — compare with a POC.
Capstone
notebooks/month-06/01_mlops_intro.ipynb:
- A simple classical ML project (e.g., churn prediction)
- Create the full structure (according to the structure above)
- No tools yet, but we’ll add each in future sections
✅ Checklist
- I know the difference between MLOps and DevOps
- I know the 8 stages of ML Lifecycle
- MLOps Maturity Levels (0, 1, 2)
- I know the main tool landscape
- I know the typical MLOps project structure
- I saw how my existing backend knowledge helps in MLOps
Moving on to MLflow — Experiment tracking.