Multimodal Transformers

Unifying Vision-and-Language Tasks via Text Generation
Tackle different V&L tasks via text generation with a single unified architecture - ICML 2021
Jaemin Cho, Jie Lei, Hao Tan, Mohit Bansal
Preprint Cite Code
Unifying Vision-and-Language Tasks via Text Generation
X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers
Generate image from text by predicting masked patches with multi-modal transformers - EMNLP 2020
Jaemin Cho, Jiasen Lu, Dustin Schwenk, Hannaneh Hajishirzi, Aniruddha Kembhavi
Preprint Cite Code Project
X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers