Benchmark

RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation
a benchmark evaluating MLLMs’ ability to identify image rotation
Tianyi Niu, Jaemin Cho, Elias Stengel-Eskin, Mohit Bansal
Preprint Cite Code
RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation
CAPTURe: Evaluating Spatial Reasoning in Vision Language Models via Occluded Object Counting
a VLM benchmark testing spatial reasoning by making the models count objects under occlusion
Atin Pothiraj, Elias Stengel-Eskin, Jaemin Cho, Mohit Bansal
Preprint Cite Code
CAPTURe: Evaluating Spatial Reasoning in Vision Language Models via Occluded Object Counting