evaluationIntroduced by GPT-3 · 2020
In-context / few-shot learning
A large model solves a new task from a few examples in its prompt, with no gradient updates.
Drafted by AI · not yet reviewed
How this idea evolved
Suggested from citations. No curator has recorded what this idea builds on yet. These ideas from the same theme come from papers that GPT-3 cites, directly or one step removed. Citation is a fact; the connection between the ideas is not verified.
Loss falls as a power law in model size, data and compute.
Answer factual questions from model weights alone, with no retrieval.
Search for better prompts to get a truer read on what an LM knows.
Papers using this
- 2018Set Transformer
- 2020MMLU
- 2021Prefix-Tuning
- 2021CLIP
- 2021Prompt tuning
- 2021Natural Instructions
- 2021True few-shot learning
- 2021Scaling ViTs (ViT-G)
- 2021Frozen
- 2021SimVLM
- 2021FLAN
- 2021ViT-VQGAN
- 2021LAION-400M
- 2021Swin V2
- 2021Florence
- 2021GLaM
- 2021Fairseq MoE LMs
- 2022Megatron-Turing NLG
- 2022PaLM
- 2022GPT-NeoX-20B
- 2022Flamingo
- 2022OPT
- 2022BIG-bench
- 2022U-PaLM
- 2022Flan-T5 / Flan-PaLM