Emergent abilities

How this idea evolved

Suggested from citations. No curator has recorded what this idea builds on yet. These ideas from the same theme come from papers that Emergent abilities cites, directly or one step removed. Citation is a fact; the connection between the ideas is not verified.

• In-context / few-shot learningGPT-3 · 2020cited directly

  A large model solves a new task from a few examples in its prompt, with no gradient updates.

• Compute-optimal trainingChinchilla · 2022cited directly

  For a fixed budget, grow parameters and training tokens together; most big LMs were undertrained.

• Neural scaling lawsKaplan scaling laws · 2020cited directly

  Loss falls as a power law in model size, data and compute.