Bias-Variance Tradeoff
The bias-variance tradeoff describes the balance between a model being too simple (high bias, underfitting) and too sensitive to the training data (high variance, overfitting).
Data & AI
Bias is error from overly simplistic assumptions — the model misses real patterns. Variance is error from being too sensitive to the specific training examples — the model captures noise. Reducing one tends to increase the other.
The goal is the sweet spot in the middle, where the model is complex enough to capture the real pattern but not so complex that it memorizes noise. Picture an archer: high bias is consistently missing in one direction, high variance is shots scattered everywhere.
Examples
Python
# Conceptual: total error = bias^2 + variance + noise
configs = {
"too simple": "high bias, low variance (underfits)",
"too complex": "low bias, high variance (overfits)",
"just right": "balanced -> best generalization"
}
for k, v in configs.items():
print(k, "->", v)Output:
too simple -> high bias, low variance (underfits)
too complex -> low bias, high variance (overfits)
just right -> balanced -> best generalization