EDA Capstone Project

🎯 Goal

The finishing project of month 1. On a real dataset, perform a **full Exploratory Data Analysis (EDA)**and prepare a professional-level report. This will be the first item in your portfolio.

Project brief

Dataset choice (pick one)

**Recommendation:**for the first time — House Prices or Titanic. They are well-documented with thousands of kernels on Kaggle.

Standard structure of EDA report

1. Project Overview (1 page)

• Purpose: why are we analyzing this data?
• Business question: the main question we’re looking to answer
• Brief about the dataset (source, size, number of columns)

2. Data Loading and Initial Inspection

df = pd.read_csv("data.csv")
print(df.shape)          # how many rows/columns
print(df.dtypes)         # column types
print(df.head())         # first rows
print(df.info())         # memory and nulls
print(df.describe())     # statistical summary

3. Data Quality Check

• Missing values — how many NaN per column, in %
• Duplicates — df.duplicated().sum()
• Data type issues — e.g., date as a string
• Outliers — by IQR or z-score method
• Value distributions — unique values in each categorical column

# Missing values visualization
import missingno as msno
msno.matrix(df)
msno.bar(df)

4. Univariate Analysis

Studying each column individually:

• Numerical: histogram, KDE, box plot
• Categorical: count plot, value_counts
• Date: time series plot

5. Bivariate Analysis

• Relationship between two columns
• Num vs Num: scatter, correlation
• Cat vs Num: box plot, violin plot
• Cat vs Cat: cross-tabulation, stacked bar

6. Multivariate Analysis

• 3+ columns combined
• Pair plot(Seaborn)
• Heatmap(correlation matrix)
• Faceted plots(FacetGrid)

7. Target Variable Deep Dive

If your goal is supervised ML:

• Target distribution
• Class imbalance (classification)
• Feature vs target relationship

8. Feature Engineering Ideas

During EDA, note the following:

• Ideas for new features (e.g., age * income)
• Columns needing transformation (log, sqrt)
• Encoding strategies (categorical → numerical)

9. Key Insights (IN BUSINESS LANGUAGE)

• 5-10 key findings
• Each one sentence, followed by visualization
• Storytelling: “Customers churn with 70% probability if they haven’t called in the last 3 months”

10. Conclusion and Next Steps

• EDA summary
• Recommendations for moving to the model
• Data constraints and risks

Technical requirements

Tools

• Jupyter Notebook or VS Code(.ipynb)
• Pandas — data manipulation
• NumPy — computations
• Matplotlib + Seaborn — visualization
• missingno — missing data visualization
• pandas-profiling or ydata-profiling(automatic EDA report)

pip install pandas numpy matplotlib seaborn missingno ydata-profiling

Code quality

• Split notebook into logical sections (with Markdown headings)
• Explanation before each code block
• Split into functions (def plot_distribution(col))
• Reproducibility: random_state=42 always explicit

Notebook structure (each section in a separate cell)

# 1. IMPORTS
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from pathlib import Path

pd.set_option("display.max_columns", 100)
sns.set_theme(style="whitegrid")

# 2. LOAD DATA
DATA_PATH = Path("../data/house_prices.csv")
df = pd.read_csv(DATA_PATH)

# 3. OVERVIEW
print(f"Shape:{df.shape}")
print(f"Memory:{df.memory_usage(deep=True).sum() / 1e6:.1f}MB")
df.head()

Deliverable

Your GitHub repo should contain:

eda-house-prices/
├── README.md                       # Loyiha tavsifi, qanday ishga tushirish
├── notebooks/
│   └── 01_eda.ipynb               # Asosiy EDA
├── data/
│   ├── raw/                       # Dastlabki CSV (gitignore bilan)
│   └── processed/                 # Tozalangan dataset
├── reports/
│   ├── insights.md                # 5-10 ta key insights (markdown)
│   ├── figures/                   # PNG/PDF chart'lar
│   └── eda_report.html            # ydata-profiling output
├── src/
│   └── plotting.py                # Reusable plot funksiyalari
└── requirements.txt

README.md template

# House Prices EDA

## Maqsad
Ames Housing datasetni tahlil qilib, uy narxiga ta'sir qiluvchi asosiy omillarni aniqlash.

## Asosiy topilmalar
- OverallQual eng kuchli korrelatsiyaga ega (0.79)
- GrLivArea (yashash maydoni) ikkinchi muhim feature (0.71)
- ...

## Qanday ishga tushirish
\`\`\`bash
pip install -r requirements.txt
jupyter notebook notebooks/01_eda.ipynb
\`\`\`

## Texnologiyalar
- Python 3.11
- pandas, numpy, matplotlib, seaborn

Evaluation criteria

For self-evaluation:

Goal: minimum 2 points on each criterion.

References

• Kaggle EDA notebooks(study the best ones):
• Comprehensive Data Exploration with Python
• Titanic EDA + ML
• “Effective Data Storytelling” — Brent Dykes
• ydata-profiling docs — docs.profiling.ydata.ai

Bonus exercises (extra credit)

1. Streamlit dashboard: turn EDA results into an interactive dashboard
2. Automated EDA: create automatic report using ydata-profiling or Sweetviz and compare with manual EDA
3. Geographic visualization(if the dataset has lat/long): create a map with Folium or Plotly

✅ Before submitting the project

• Notebook runs fully without errors
• Each chart has title, xlabel, ylabel, legend
• Each section explained with Markdown
• README clear and complete
• Committed to GitHub (notebook and charts)
• Write a LinkedIn post (this is your first ML project!)

Congratulations — the first big step is done. Complete the additional practice in the Exercises section.

Next: move to Month 2 — Classical ML.