training techniqueIntroduced by Noisy Student · 2019
Noisy Student training
Self-training with an equal-or-larger student and noise added to it, iterated.
Drafted by AI · not yet reviewed
How this idea evolved
Suggested from citations. No curator has recorded what this idea builds on yet. These ideas from the same theme come from papers that Noisy Student cites, directly or one step removed. Citation is a fact; the connection between the ideas is not verified.
Pre-train on billions of social-media images labelled only by their hashtags.
A teacher labels a huge unlabelled pool; a student learns from those pseudo-labels.
Performance keeps rising (logarithmically) with 100× more labelled images.