# Clean a small dataset with Pandas

Code With Purpose · Hinglish · Free starter lesson

## Concept

DataFrame ek table jaisa object hai jiske columns ke naam hote hain. shape, dtypes aur isna().sum() se shuru karo: kitni rows hain, types kya hain aur missing values kahan hain. Cleaning ka decision data ke meaning se aata hai, sirf error hatane se nahi.

drop_duplicates default mein identical rows hata deta hai. Yeh tabhi sahi hai jab duplicates accidental copies hon; repeated transactions valid bhi ho sakte hain. pd.to_numeric(..., errors="coerce") invalid numeric text ko missing bana deta hai. Dekho kitni values change hui.

Is chhote example mein missing study time ko observed median se fill karenge. Yeh teaching example hai, har dataset ka rule nahi. Predictive modeling mein imputation value sirf training data se nikaalo. Example chalane se pehle environment mein Pandas install karo.

## Example

```python
import pandas as pd
df = pd.DataFrame({"name": ["Asha", "Ravi", "Asha", "Meera"],
                   "hours": ["2", None, "2", "4"]})
clean = df.drop_duplicates().copy()
clean["hours"] = pd.to_numeric(clean["hours"], errors="coerce")
clean["hours"] = clean["hours"].fillna(clean["hours"].median())
print(clean["hours"].tolist())
print(len(clean))
```

## Expected output

```text
[2.0, 3.0, 4.0]
3
```

## Exercise

Missing values fill karne se pehle count karo. Ek flag banao jo record kare ki kaunsi hours values missing thi. Yeh flag useful kyun hai?

## Suggested solution

```
clean = df.drop_duplicates().copy()
clean["hours"] = pd.to_numeric(clean["hours"], errors="coerce")
clean["hours_was_missing"] = clean["hours"].isna()
print(clean["hours_was_missing"].sum())  # 1
clean["hours"] = clean["hours"].fillna(clean["hours"].median())
# The flag preserves the distinction between observed and filled values.
```

## Takeaway

Clean table mein data ka meaning bachna chahiye aur assumptions clear honi chahiye.

Reference: https://pandas.pydata.org/docs/getting_started/intro_tutorials/

https://codewithpurposeai.in/lesson-data-cleaning.html?lang=hinglish