objectiveIntroduced by UNITER · 2019
Word–region alignment
Explicitly align words to image regions with an optimal-transport objective.
Drafted by AI · not yet reviewed
How this idea evolved
Suggested from citations. No curator has recorded what this idea builds on yet. These ideas from the same theme come from papers that UNITER cites, directly or one step removed. Citation is a fact; the connection between the ideas is not verified.
Concatenate words and image regions and run one Transformer over both.
Represent an image by features of detected object regions instead of a pixel grid.