Gradient Descent as an Optimization Method

Gradient descent is introduced as the method machines use to learn by turning a task into a mathematical optimization problem. The example frames speech-command recognition as a labeled-data problem that can be trained by minimizing error.

• Gradient descent helps models learn patterns from data.
• It is used in many machine learning tasks, from face generation to game-playing systems.
• A learning problem can be written as an optimization problem.
• The example uses audio command recognition with labeled data.

<aside> 💡 Machine learning often starts by expressing the goal as a mathematical objective, then using gradient descent to improve the model step by step.

</aside>

Building a Cost Function for Neural Network Training

A neural network can produce an output vector for each input, and training compares that output to the target label. The per-example error is computed by taking the difference, squaring it, and summing the result, then aggregating across all training examples to form the overall cost function.

• Neural network weights are treated as unknown variables to learn.
• For each input, the network outputs a vector of class scores or command probabilities.
• Training error for one example is based on the difference between prediction and target, squared and summed.
• The total cost function is formed by summing the errors over the full dataset.

<aside> 💡 Training a neural network means defining a cost function that measures prediction error across examples so optimization can adjust the weights.

</aside>

No clear learning-relevant slide, formula, or diagram is visible from the provided visual description.

Why Gradient Descent Is Needed

The goal is to find parameter values that minimize the cost function, and gradient descent is introduced as the method for doing that. It is needed because in high-dimensional settings we usually cannot visualize the whole function, even though we can still evaluate it.

• Gradient descent is the second step after defining the cost function.