Supervised

• Training with labeled data
• Pros
  • Well labeled dataset can represent the problem the best than anything else
• Cons
  • Creating labeled dataset is often labor intensive
  • Labeled dataset is often very subjective and biased

Unsupervised

• Training with unlabeled data

Semi-supervised

• Training with partially labeled data

  • Semi-supervised Learning Tutorial by Xiaojin Zhu (Univ. Wisconsin, Madison)

    

Self-supervised

• Generating label by the model itself and then train with self-generated labels when the generated label is confident enough
  • Self-training with Noisy Student improves ImageNet classification
  • Distilling the knowledge in a Neural Network

Why is it useful

• It can overcome the lack of data using cheap and abundant unlabeled data