evaluationIntroduced by Learning like a Child · 2015
Novel object captioning
Describe objects never seen in the caption training data.
Drafted by AI · not yet reviewed
How this idea evolved
Suggested from citations. No curator has recorded what this idea builds on yet. These ideas from the same theme come from papers that Learning like a Child cites, directly or one step removed. Citation is a fact; the connection between the ideas is not verified.
Detect caption words with multiple-instance learning, then compose sentences with an LM.
Score a caption by its TF-IDF n-gram agreement with many human references.
Align image regions with sentence fragments, then generate descriptions with a multimodal RNN.
Papers using this
- 2018Neural Baby Talk
- 2018nocaps
- 2021Conceptual 12M
- 2021ViTCAP
- 2022Simple end-to-end captioning
- 2022CoCa