training techniqueIntroduced by GSM8K verifiers · 2021

Trained verifiers

Sample many solutions and let a trained verifier pick the right one.

Drafted by AI · not yet reviewed

How this idea evolved

Suggested from citations. No curator has recorded what this idea builds on yet. These ideas from the same theme come from papers that GSM8K verifiers cites, directly or one step removed. Citation is a fact; the connection between the ideas is not verified.

In-context / few-shot learningGPT-3 · 2020cited directly
A large model solves a new task from a few examples in its prompt, with no gradient updates.
Neural scaling lawsKaplan scaling laws · 2020two citations away
Loss falls as a power law in model size, data and compute.
Knowledge stored in LM parametersKnowledge in LM parameters · 2020two citations away
Answer factual questions from model weights alone, with no retrieval.

Papers using this

2022PaLM
2022U-PaLM
2022Flan-T5 / Flan-PaLM