Teacher–student self-training

How this idea evolved

Suggested from citations. No curator has recorded what this idea builds on yet. These ideas from the same theme come from papers that Billion-scale semi-supervised cites, directly or one step removed. Citation is a fact; the connection between the ideas is not verified.

• Hashtag supervision at billion scaleInstagram hashtag pre-training · 2018cited directly

  Pre-train on billions of social-media images labelled only by their hashtags.

• Data scaling in visionJFT-300M (unreasonable effectiveness) · 2017two citations away

  Performance keeps rising (logarithmically) with 100× more labelled images.

• Weakly supervised pre-trainingWeakly supervised visual features (Jouli · 2015two citations away

  Use huge amounts of cheap, noisy labels instead of clean annotations.