objectiveIntroduced by ELMo · 2018
Deep bidirectional LM features
Use all internal layers of a pretrained bidirectional LM as features for downstream tasks.
Drafted by AI · not yet reviewed
How this idea evolved
Suggested from citations. No curator has recorded what this idea builds on yet. These ideas from the same theme come from papers that ELMo cites, directly or one step removed. Citation is a fact; the connection between the ideas is not verified.
Push LSTM language models to billions of words and huge vocabularies.
Word vectors that depend on the sentence, taken from a pretrained encoder.