MuseBench: Benchmarking Intent-Level Audiovisual Arts Understanding in MLLMs
Abstract
MuseBench evaluates whether multimodal large language models can interpret the creative intentions behind audiovisual artworks, going beyond recognition of their visible and audible content. Drawing on professionally narrated video essays, the benchmark contains 4,016 questions across cinema, static visual art, stage performance, and games. It uses single-answer and multiple-answer formats, with iterative filtering, challenging distractors, and expert review. Evaluations of 28 models reveal a substantial gap between model and expert artistic understanding: the strongest model scores 48.29% accuracy, compared with 87.18% for human experts.
Type
Publication
arXiv preprint
