Month 2 — Exercises set

🟢 Easy

Algorithms

1. load_iris(), load_wine(), load_breast_cancer() — for each one, train 3 different models and compare accuracy.
2. LogisticRegression, KNN, SVM, DecisionTree, RandomForest — evaluate all with cross_val_score.
3. Determine for each model whether feature scaling is needed or not (compare with Pipeline + StandardScaler vs without).

Metrics

1. Compute confusion matrix by hand and verify with sklearn.metrics.confusion_matrix.
2. Compute Precision, Recall, F1 manually using the formulas.
3. Show the difference between predict and predict_proba, change threshold to change accuracy.

Pipeline

1. Create Pipeline([scaler, model]) and run fit_predict.
2. Process numeric and categorical columns separately with ColumnTransformer.
3. Save Pipeline with joblib.dump and reload it.

🟡 Medium

Real datasets

1. Titanic: Get 80%+ accuracy with Pipeline + Random Forest.
2. House Prices: Compare Lasso + Ridge, achieve R² 0.85+.
3. Telco Churn: Combat imbalanced data, achieve F1 0.6+.
4. Wine Quality: Compare regression vs classification approaches.

Feature Engineering

1. NYC Taxi: Create 10+ features from datetime and observe RF accuracy improvement.
2. Text feature engineering: Enrich one categorical column with n-gram.
3. Polynomial features: Experiment with degree=2, observe overfitting.

Hyperparameter Tuning

1. With GridSearchCV XGBoost 3 parameters — how long for 100 trials?
2. Same thing with RandomizedSearchCV — time and quality difference?
3. With Optuna 100 trials — best and fastest!

Ensembles

1. RF vs XGBoost vs LightGBM vs CatBoost — compare on the same dataset (table).
2. Voting Classifier (3 models) — is it better than each one’s individual result?
3. Stacking — create base + meta.

🔴 Hard (Production)

1. Churn Prediction Service

Full requirements:

• Django REST Framework or FastAPI
• PostgreSQL customer table (50+ features)
• /api/v1/predict/churn/{customer_id} — fetch features from DB + prediction
• /api/v1/predict/churn/batch — CSV upload + Celery background
• /api/v1/feedback — return real outcome (for model improvement)
• /api/v1/metrics — Prometheus format
• Docker + docker-compose
• GitHub Actions CI/CD

2. AutoML Service

Upload a dataset and automatically:

• EDA report (ydata-profiling)
• Compare 5+ algorithms
• Save the best model
• Prediction endpoint automatically ready

Inspiration: H2O AutoML, PyCaret.

3. A/B Testing Backend

• Serve two models (v1 and v2)
• Random traffic split (60/40 or configurable)
• Log each prediction to Postgres
• Automatically determine which model is better with a statistical test
• Slack notification: “Model v2 wins!”

4. Real-time Anomaly Detection

• Kafka consumer (transaction stream)
• Online anomaly detection with IsolationForest or DBSCAN
• Send anomalies to a separate Kafka topic
• Grafana dashboard

Mini-projects

Mini-project 1: Spam Classifier

• SMS Spam dataset (UCI)
• TF-IDF + Logistic Regression / Naive Bayes
• FastAPI endpoint
• Streamlit UI

Mini-project 2: Stock Price Direction

• Stock data with yfinance
• Technical indicators (RSI, MACD) feature engineering
• Up/Down classification
• Backtesting

Mini-project 3: Recommendation System (Collaborative Filtering)

• MovieLens dataset
• Surprise library
• User-based and item-based
• API: /recommend/{user_id}

Mini-project 4: Time Series Forecasting

• Prophet or ARIMA
• Daily sales forecasting
• 30-day prediction

Quiz

ML Fundamentals

1. Difference between Supervised and Unsupervised?
2. Explain the Bias-Variance tradeoff with an example.
3. How do you detect overfitting?
4. Why is cross-validation needed?
5. Why 3 in Train/Val/Test split?

Algorithms

1. Why the word “regression” in Logistic Regression’s name? (Hint: log-odds)
2. What does the k parameter affect in KNN?
3. Difference between Random Forest and Gradient Boosting (parallel vs sequential)?
4. Main difference between XGBoost and LightGBM?
5. Why is CatBoost’s categorical handling better?

Metrics

1. Why is accuracy a poor metric for imbalanced classification?
2. When do ROC-AUC and PR-AUC differ?
3. Difference between F1 and F-beta?
4. In regression, when do you use MAE vs MSE?
5. Can R² be negative? Why?

Production

1. Difference between joblib and pickle?
2. How do you containerize an ML model with Docker?
3. What is model drift and how is it detected? (preview, Month 6)
4. Why is ONNX useful?
5. What is statistical significance in A/B testing?

✅ Month 2 final checklist

• Used most of the classical ML algorithms
• Mastered Scikit-learn Pipeline and ColumnTransformer
• Worked with XGBoost/LightGBM (at least 1 competition)
• Did hyperparameter tuning with Optuna
• Interpreted models with SHAP or Feature Importance
• Deployed ML model to production with FastAPI
• Made first Kaggle submission (top 30%)
• Capstone project on GitHub
• LinkedIn post (project + certificate)

Congratulations! Moving on to Month 3 — Deep Learning.