Sequence models & translation
Recurrent, convolutional and encoder–decoder models that map one sequence to another.
| Concept | Introduced by | Years | Papers |
|---|---|---|---|
| Encoder–decoder seq2seq Encode an input sequence, then decode an output sequence from it. | Seq2Seq | 2014–2022 | 13 |
| LSTM recurrent networks Gated recurrent networks that keep information over long sequences. | — | 2014–2018 | 9 |
| Dilated convolutional sequence models Model long sequences with stacked dilated 1-D convolutions that run in linear time. | ByteNet | 2016–2017 | 2 |
| WordPiece subword tokens Split rare words into frequent sub-word pieces so the vocabulary stays small and open. | GNMT | 2016–2021 | 6 |
| Convolutional seq2seq A fully convolutional encoder–decoder with gated linear units and per-layer attention. | ConvS2S | 2017–2020 | 2 |
| Language-independent subword tokenizer Train subword models directly on raw text, with no language-specific pre-tokenisation. | SentencePiece | 2018–2022 | 5 |