training techniqueIntroduced by T0 · 2021
Prompted multitask training
Train on many datasets rewritten into diverse natural-language prompts.
Drafted by AI · not yet reviewed
How this idea evolved
Suggested from citations. No curator has recorded what this idea builds on yet. These ideas from the same theme come from papers that T0 cites, directly or one step removed. Citation is a fact; the connection between the ideas is not verified.
Fine-tune on many tasks phrased as instructions, so the model follows new instructions zero-shot.
Learn a reward model from human comparisons, then optimise the LM against it.
Papers using this
- 2021Gopher
- 2022InstructGPT
- 2022Super-NaturalInstructions
- 2022BIG-bench