training techniqueIntroduced by Prefix-Tuning · 2021
Prefix tuning
Optimise a few continuous 'virtual tokens' prepended at every layer of a frozen LM.
Drafted by AI · not yet reviewed
How this idea evolved
Suggested from citations. No curator has recorded what this idea builds on yet. These ideas from the same theme come from papers that Prefix-Tuning cites, directly or one step removed. Citation is a fact; the connection between the ideas is not verified.
Insert small trainable layers into a frozen network, one set per task.
Papers using this
- 2021Prompt tuning
- 2021Frozen