The Core Problem

Learning algorithms can run into problems that cause poor performance:

• Underfitting — model is too simple
• Overfitting — model is too complex

The goal is to find a model that is "just right"

Three Cases: Housing Price Example

Case 1: Underfitting (High Bias)

Model: Linear function (straight line)

$$ f_{\vec{w}, b}(x) = wx + b $$

Problem:

• Does not fit training data well
• Misses the pattern that prices flatten as size increases
• Has a strong preconception that data must be linear

Terms used interchangeably:

• Underfitting
• High bias

Case 2: "Just Right"

Model: Quadratic function

$$ f_{\vec{w}, b}(x) = w_1 x + w_2 x^2 + b $$

Result: