Knowledge-grounded dialogue LMs

How this idea evolved

Suggested from citations. No curator has recorded what this idea builds on yet. These ideas from the same theme come from papers that LaMDA cites, directly or one step removed. Citation is a fact; the connection between the ideas is not verified.

• In-context / few-shot learningGPT-3 · 2020cited directly

  A large model solves a new task from a few examples in its prompt, with no gradient updates.

• Neural scaling lawsKaplan scaling laws · 2020cited directly

  Loss falls as a power law in model size, data and compute.

• Knowledge stored in LM parametersKnowledge in LM parameters · 2020two citations away

  Answer factual questions from model weights alone, with no retrieval.