dataIntroduced by The Pile · 2020
Diverse curated text corpora
Mix many high-quality text sources to improve LM generalisation.
Drafted by AI · not yet reviewed
How this idea evolved
No earlier ideas recorded for this concept yet.
Papers using this
- 2019Adapters
- 2021Codex
- 2021Gopher
- 2021Fairseq MoE LMs
- 2022GPT-NeoX-20B
- 2022OPT