This is my note for the course (Structuring Machine Learning Projects). The codes in this note are rewritten to be more clear and concise.

• All the courses

• Course’s information

This course will give you some strategies to help analyze your problem to go in a direction that will help you get better results.

Introduction to ML Strategy

Why ML strategy?

• "ML strategy" = How to structure your ML project?
• Ideas to improve your ML systems:
  1. Collect more data.
  2. Collect more diverse training set.
  3. Train algorithm longer with gradient descent.
  4. Try different optimization algorithm (e.g. Adam).
  5. Try bigger network.
  6. Try smaller network.
  7. Try dropout.
  8. Add L2 regularization.
  9. Change network architecture (activation functions, # of hidden units, etc.)
• However, don't spend too much time to do one of above things, we need to go right direction!

Orthogonalization

• In orthogonalization, you have some controls, but each control does a specific task and doesn't affect other controls.
• Chain of assumptions in ML:
  1. You'll have to fit training set well on cost function (near human level performance if possible).
  • If it's not achieved you could try bigger network, another optimization algorithm (like Adam)...
  1. Fit dev set well on cost function.
  • If its not achieved you could try regularization, bigger training set...
  1. Fit test set well on cost function.
  • If its not achieved you could try bigger dev. set...
  1. Performs well in real world.
  • If its not achieved you could try change dev. set, change cost function...

Setting up your goal

Single number evaluation metric

• Advice: It's better and faster to set a single number evaluation metric for your project before you start it.
• Example: instead of using both precision and recall, just use f1. Check this note.
• Dev set + single row number evaluation metric → enough to make a choice!

Satisfying and Optimizing metric

• It's difficult to set all parameters to a single row number evaluation metric → set up (many) satisfying + (one) optimizing matrix.
  • Satisfying (use threshold): satisfying this is enough.
  • Optimizing: more important, it's accuracy!
• Example: call "Hi Siri",
  • Accuracy: is it awoken? → optimizing
  • False positive: it's awoken but we don't call it! → set the satisfying as less then 1 false positive per day!