Train/Test 🧪
Evaluating Models
How do we know if our model is good? We can't test it on the same data we used to train it, because it already knows the answers! The solution is to split the data into two sets: Training and Testing.
🎓 Analogy: The Final Exam
- Training Set: The homework and practice problems you do during the semester. You see the answers and learn from them.
- Testing Set: The Final Exam. You haven't seen these exact questions before. This measures how well you *actually* learned the concepts, not just memorized the homework.
✨ The 80/20 Rule
- Training Set (80%): Used to teach the model.
- Testing Set (20%): Used to test the model's accuracy on unseen data.
🎯 Real-World Application: Fraud Detection
Banks train models on historical transactions (Fraud vs Legit). They test the model on new, recent transactions to ensure it can catch *new* fraud attempts it hasn't seen before.
🛠️ Implementation
import numpy as np
import matplotlib.pyplot as plt
from sklearn.metrics import r2_score
np.random.seed(2)
x = np.random.normal(3, 1, 100)
y = np.random.normal(150, 40, 100) / x
# Split manually (or use train_test_split from sklearn)
train_x = x[:80]
train_y = y[:80]
test_x = x[80:]
test_y = y[80:]
# Create model
model = np.poly1d(np.polyfit(train_x, train_y, 4))
# Evaluate on Test Set
r2 = r2_score(test_y, model(test_x))
print(r2) # 0.809 (Good fit)✨ Pro Tip: Random State
When splitting data, always use a random_state (or seed) so that your split is reproducible. Otherwise, you'll get different results every time you run the code!
⚠️ Overfitting
If your model performs perfectly on the training set but poorly on the testing set, it is overfitting. It has memorized the training data instead of learning the patterns.