Vision and Lanaguage

Visual Programming for Text-to-Image Generation and Evaluation
Interpretable/explainable visual programming frameworks for T2I generation (VPGen) and evaluation (VPEval) - NeurIPS 2023
Jaemin Cho, Abhay Zala, Mohit Bansal
Preprint Cite Code Project
Visual Programming for Text-to-Image Generation and Evaluation
DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models
Evaluation of Text-to-Image Generation Models in Reasoning Skills and Social Biases - ICCV 2023
Jaemin Cho, Abhay Zala, Mohit Bansal
Preprint Cite Code
DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models
Hierarchical Video-Moment Retrieval and Step-Captioning
HiREST is a holistic, hierarchical benchmark of multimodal retrieval and step-by-step summarization for a video corpus - CVPR 2023
Abhay Zala, Jaemin Cho, Satwik Kottur, Xilun Chen, Barlas Oğuz, Yasahar Mehdad, Mohit Bansal
Preprint Cite Code Project
Hierarchical Video-Moment Retrieval and Step-Captioning
Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention
Efficient VL modeling with Perceiver-based iterative cross-attentions - WACV 2023
Zineng Tang, Jaemin Cho, Jie Lei, Mohit Bansal
Preprint Cite Code
Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention
VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks
Adapter-based Parameter-Efficient Training for V&L tasks - CVPR 2022
Yi-Lin Sung, Jaemin Cho, Mohit Bansal
Preprint Cite Code
VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks
VidLanKD: Improving Language Understanding via Video-Distilled Knowledge Transfer
Video-based grounding can improve diverse NLU tasks - NeurIPS 2021
Zineng Tang, Jaemin Cho, Hao Tan, Mohit Bansal
Preprint Cite Code
VidLanKD: Improving Language Understanding via Video-Distilled Knowledge Transfer
Unifying Vision-and-Language Tasks via Text Generation
Tackle different V&L tasks via text generation with a single unified architecture - ICML 2021
Jaemin Cho, Jie Lei, Hao Tan, Mohit Bansal
Preprint Cite Code
Unifying Vision-and-Language Tasks via Text Generation
X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers
Generate image from text by predicting masked patches with multi-modal transformers - EMNLP 2020
Jaemin Cho, Jiasen Lu, Dustin Schwenk, Hannaneh Hajishirzi, Aniruddha Kembhavi
Preprint Cite Code Project
X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers