# Train/test splits & data leakage

Code With Purpose · Hinglish · Free starter lesson

## Concept

Supervised learning mein features X aur known targets y se mapping seekhte hain. Classification categories predict karta hai; regression numeric values. Training data se model fit hota hai. Held-out data se unseen examples par performance ka estimate milta hai.

Leakage tab hota hai jab prediction ke time unavailable information training ya preprocessing mein pahunch jaati hai. Split se pehle poore dataset par scaler ya imputer fit karna held-out data ki information leak karta hai. Pipeline preprocessing aur model ko training data par saath fit karta hai.

Random split kuch independent observations ke liye theek hai. Time series ke liye chronological aur repeated entities ke liye grouped split chahiye ho sakta hai. Model choices validation ya cross-validation se karo; final test set evaluation ke liye bachao. Demo mein scikit-learn chahiye. Synthetic score real-world result ka claim nahi hai.

## Example

```python
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
X, y = make_classification(n_samples=200, n_features=6, random_state=7)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=7)
model = make_pipeline(StandardScaler(), LogisticRegression())
model.fit(X_train, y_train)
print(len(X_train), len(X_test))
print(round(model.score(X_test, y_test), 3))
```

## Expected output

```text
150 50
The next line is test accuracy; the value can vary by library version.
```

## Exercise

Churn dataset mein cancellation_date cancellation ke baad record hoti hai. Kya future churn predict karne mein isko feature bana sakte ho? Explain karo aur candidate features se hatao.

## Suggested solution

```
# No: the date is unavailable before the outcome and leaks target information.
# If df is your real dataset:
# X = df.drop(columns=["churn", "cancellation_date"])
# y = df["churn"]
# Audit every other feature's availability at prediction time too.
```

## Takeaway

Prediction ke waqt jo information milegi, sirf wahi use karo.

Reference: https://scikit-learn.org/stable/common_pitfalls.html

https://codewithpurposeai.in/lesson-ml-splits.html?lang=hinglish