training techniqueIntroduced by SimCLR · 2020
Augmentations + projection head
Strong augmentation plus a nonlinear projection head is what makes contrastive learning work.
Drafted by AI · not yet reviewed
How this idea evolved
Suggested from citations. No curator has recorded what this idea builds on yet. These ideas from the same theme come from papers that SimCLR cites, directly or one step removed. Citation is a fact; the connection between the ideas is not verified.
Treat every image as its own class and learn features that tell instances apart.
A queue of negatives encoded by a slowly-updated momentum encoder.
Predict future latents and score them against negatives with a contrastive loss.
Papers using this
- 2020ConVIRT