SwadhyAI Notes

All Python notes

Cross Validation ⚔️

Testing Robustness

The problem with Train/Test split: What if the split gets "unlucky" and puts all the hard examples in the test set? Cross Validation solves this by splitting the data into multiple folds and testing on each fold.

✨ K-Fold Cross Validation

The most common method. Here's how it works:

  1. Split data into K equal parts (folds).
  2. Train on K-1 folds, test on the remaining 1 fold.
  3. Repeat K times, using a different fold as the test set each time.
  4. Average the results to get the final score.
python
from sklearn.model_selection import KFold, cross_val_score
from sklearn.tree import DecisionTreeClassifier
from sklearn import datasets

X, y = datasets.load_iris(return_X_y=True)

clf = DecisionTreeClassifier(random_state=42)

# 5-Fold Cross Validation
k_folds = KFold(n_splits=5)
scores = cross_val_score(clf, X, y, cv=k_folds)

print("Scores:", scores)
print("Average Score:", scores.mean())

🔄 Stratified K-Fold

Use this when classes are imbalanced. It ensures each fold has the same proportion of each class.

python
from sklearn.model_selection import StratifiedKFold

sk_folds = StratifiedKFold(n_splits=5)
scores = cross_val_score(clf, X, y, cv=sk_folds)

print("Average Score:", scores.mean())

✨ Pro Tip

Cross Validation is a more reliable estimate of model performance than a single train/test split, especially when you have limited data.