Benchmark

MuseBench: Benchmarking Intent-Level Audiovisual Arts Understanding in MLLMs
A 4,016-question benchmark evaluating MLLMs on creative intent across audiovisual arts
Yuxuan Fan, Gyusik Seo, Jing Hao, Jaemin Cho, Mohit Bansal, Jaehong Yoon
Preprint Cite Code Project
RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation
a benchmark evaluating MLLMs’ ability to identify image rotation
Tianyi Niu, Jaemin Cho, Elias Stengel-Eskin, Mohit Bansal
Preprint Cite Code
RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation
CAPTURe: Evaluating Spatial Reasoning in Vision Language Models via Occluded Object Counting
a VLM benchmark testing spatial reasoning by making the models count objects under occlusion
Atin Pothiraj, Elias Stengel-Eskin, Jaemin Cho, Mohit Bansal
Preprint Cite Code
CAPTURe: Evaluating Spatial Reasoning in Vision Language Models via Occluded Object Counting