objective
Autoregressive language modelling
Predict the next token, so the model can generate text left to right.
Drafted by AI · not yet reviewed
How this idea evolved
No earlier ideas recorded for this concept yet.
Papers using this
- 2017VQ-VAE
- 2018CPC
- 2019AMDIM
- 2019XLNet
- 2020GPT-3
- 2021Natural Instructions
- 2021ViT-VQGAN
- 2021MAE
- 2021Gopher
- 2021Fairseq MoE LMs
- 2022Megatron-Turing NLG
- 2022PaLM
- 2022GPT-NeoX-20B
- 2022Super-NaturalInstructions
- 2022OPT
- 2022Emergent abilities