optimizationIntroduced by AdamW · 2017
Decoupled weight decay (AdamW)
Apply weight decay separately from Adam's adaptive step, fixing its regularisation.
Drafted by AI · not yet reviewed
How this idea evolved
No earlier ideas recorded for this concept yet.
Papers using this
- 2021Prefix-Tuning
- 2021VL-T5
- 2021CLIP
- 2021ALBEF
- 2022Simple end-to-end captioning
- 2022BEiT-3
- 2023BLIP-2