Feature engineering is the process of designing new features by transforming or combining original features to help the learning algorithm make better predictions.

It uses your knowledge and intuition about the problem to create more predictive features.

Example: Predicting House Prices

Original Features

Original model:

$$ f_{\vec{w}, b}(\vec{x}) = w_1 x_1 + w_2 x_2 + b $$

This might work okay, but can we do better?

The Insight

You might realize that the area of the land is more predictive of price than width and depth separately.

$$ \text{Area} = \text{Frontage} \times \text{Depth} $$

Engineered Feature

Create a new feature:

$$ x_3 = x_1 \times x_2 \quad \text{(area)} $$

New model:

$$ f_{\vec{w}, b}(\vec{x}) = w_1 x_1 + w_2 x_2 + w_3 x_3 + b $$

Now the algorithm can learn which matters most:

• Frontage alone ($w_1$)
• Depth alone ($w_2$)
• Area ($w_3$)