SwadhyAI Notes

All Python notes

Bootstrap Aggregation (Bagging) 🎒

Strength in Numbers

Bagging is an ensemble technique. Instead of relying on one model, it trains multiple models on random subsets of the data and averages their predictions.

✨ How it Works

  • Bootstrap: Create multiple random subsets of the training data (with replacement).
  • Aggregation: Train a model on each subset. Combine predictions (Vote for classification, Average for regression).

💻 Implementation

We can use BaggingClassifier with a base estimator (like Decision Tree).

python
from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

data = load_iris()
X_train, X_test, y_train, y_test = train_test_split(data.data, data.target, test_size=0.25)

# Create a Bagging Classifier with 10 Decision Trees
clf = BaggingClassifier(estimator=DecisionTreeClassifier(), n_estimators=10, random_state=42)

clf.fit(X_train, y_train)
print("Accuracy:", clf.score(X_test, y_test))

💡 Random Forest

The most famous Bagging algorithm is Random Forest, which uses many Decision Trees. It is one of the most powerful and versatile ML algorithms.