🚀 LeaderBoard

Core Cognitive Benchmarking ... Coming Soon!

🎬 Image + Video 📷 Image only 🖼️ Single Image

Total: Image + Video

o1-2024-12-17 78.84% 63.95% 57.03% 61.22% 87.79% 75.45% 55.21% 78.38% 75.49% 87.54% 85.13% 98.16% 74.91%
gpt-4o 87.17% 63.37% 52.86% 67.95% 81.2% 58.79% 43.06% 69.97% 57.52% 85.2% 77.51% 96.91% 69.25%
qwen2.5-vl-72b-instruct 79.59% 64.53% 43.38% 62.82% 85.27% 59.09% 28.91% 81.68% 68.2% 80.45% 85.66% 97.96% 68.29%
qwen-vl-max-2025-01-25 82.4% 65.12% 41.99% 61.56% 84.33% 57.88% 27.89% 78.98% 69.66% 81.0% 86.2% 97.96% 67.91%
gemini-1.5-pro 82.58% 69.19% 59.56% 67.95% 73.26% 56.21% 30.96% 64.26% 73.62% 81.0% 80.56% 98.09% 67.8%
gemini-1.5-flash 79.12% 71.22% 47.55% 68.11% 77.62% 57.42% 30.63% 51.95% 73.95% 78.74% 72.13% 95.77% 65.5%
gpt-4-turbo 77.62% 61.34% 48.94% 54.25% 82.17% 63.48% 43.11% 50.15% 51.62% 80.92% 75.81% 98.75% 65.23%
grok-2-vision-1212 80.52% 69.19% 36.27% 67.63% 73.26% 53.64% 37.34% 50.15% 71.12% 79.67% 58.15% 99.21% 63.69%
llava-video-72b-qwen2_multi_frame 81.27% 59.88% 39.46% 53.37% 74.81% 59.85% 31.84% 59.46% 65.29% 80.3% 76.79% 92.17% 63.09%
gemini-1.5-flash-8b 76.12% 64.83% 58.01% 54.09% 75.97% 50.76% 31.28% 66.67% 60.92% 75.08% 59.86% 95.26% 62.91%
qwen2.5-vl-7b-instruct 82.3% 59.59% 41.42% 63.78% 68.6% 55.15% 32.87% 65.17% 62.38% 76.4% 63.71% 96.78% 62.69%
claude-3-5-sonnet-20240620 78.28% 56.4% 32.43% 64.74% 76.74% 58.18% 34.17% 46.55% 64.56% 72.98% 69.0% 97.96% 61.92%
qwen-vl-plus-2025-01-25 78.93% 58.72% 39.71% 63.35% 67.86% 54.85% 31.47% 64.86% 62.14% 75.93% 60.66% 96.91% 61.69%
claude-3-7-sonnet-20250219 82.87% 60.76% 32.19% 55.29% 75.0% 52.42% 34.78% 47.45% 71.36% 73.83% 69.0% 95.46% 61.44%
deepseek-vl2-small 82.68% 58.43% 39.22% 61.3% 75.0% 59.85% 32.77% 51.95% 66.99% 77.65% 48.03% 92.57% 61.36%
qwen2-vl-72b-instruct 80.34% 71.51% 43.14% 35.02% 80.81% 58.48% 27.42% 72.97% 74.68% 87.46% 81.81% 46.6% 61.12%
internvl2-40b 81.55% 70.06% 39.46% 50.0% 71.51% 58.48% 28.68% 36.04% 73.22% 81.0% 67.11% 95.79% 60.98%
ovis1.6-gemma2-9b 85.39% 60.17% 47.55% 51.68% 69.19% 55.76% 26.91% 54.35% 71.6% 77.49% 65.77% 87.7% 60.92%
gpt-4o-mini 74.25% 70.06% 38.89% 53.93% 64.73% 59.39% 41.01% 42.04% 58.66% 76.01% 69.35% 96.12% 60.89%
llava-video-7b-qwen2_multi_frame 86.52% 59.3% 36.19% 56.41% 76.16% 48.79% 30.77% 48.65% 60.68% 79.75% 63.98% 87.96% 60.13%
mplug-owl3 76.5% 59.01% 39.95% 48.96% 62.21% 44.09% 33.8% 71.47% 70.63% 77.1% 60.22% 92.89% 59.92%
qwen2.5-vl-3b-instruct 70.6% 59.01% 36.76% 52.72% 63.76% 62.12% 33.99% 54.95% 60.84% 77.65% 63.71% 96.05% 59.8%
internvl2-26b 79.96% 69.77% 40.2% 55.61% 70.35% 58.79% 28.03% 32.43% 66.18% 82.4% 58.06% 92.17% 59.62%
gemma-3-27b-it 78.18% 50.0% 35.05% 49.86% 60.47% 51.52% 35.01% 53.75% 69.74% 77.55% 70.53% 97.52% 59.38%
gemma-3-12b-it 74.53% 52.87% 40.28% 46.72% 61.82% 54.7% 34.92% 43.24% 70.47% 78.38% 70.35% 93.25% 58.76%
xgen-mm-phi3-interleave-r-v1.5 79.87% 68.31% 46.16% 48.72% 77.33% 56.36% 33.29% 47.75% 61.17% 74.07% 50.54% 72.87% 58.72%
internvl-chat-v1-2 82.02% 63.37% 53.21% 45.75% 63.95% 58.03% 30.73% 56.46% 58.09% 80.86% 65.5% 68.96% 58.66%
llava-next-video-32b-qwen_multi_frame 80.99% 67.73% 50.25% 43.43% 71.51% 55.45% 30.21% 63.06% 71.2% 80.37% 64.34% 47.11% 58.53%
aria 79.78% 56.4% 35.95% 37.98% 71.9% 56.52% 33.43% 58.56% 60.28% 73.36% 56.72% 91.91% 58.4%
vila1.5-40b 75.37% 58.14% 40.6% 59.7% 69.38% 54.55% 26.96% 30.63% 60.6% 78.27% 64.34% 95.92% 58.31%
internvl2-76b 80.52% 69.19% 45.26% 43.75% 72.09% 60.15% 30.73% 44.14% 66.42% 82.24% 78.85% 48.77% 58.17%
deepseek-vl2 76.69% 65.41% 35.87% 55.21% 69.09% 57.42% 31.98% 32.43% 64.72% 80.14% 48.84% 92.52% 58.17%
qwen2-vl-7b-instruct 83.99% 63.95% 40.77% 42.15% 70.35% 61.82% 29.8% 60.66% 75.08% 75.78% 64.78% 49.76% 57.93%
pixtral-12b-2409 72.28% 55.52% 44.28% 49.84% 66.86% 51.97% 25.19% 58.26% 59.06% 70.87% 58.42% 96.91% 57.78%
internvl2-8b-mpo 81.09% 69.77% 38.97% 36.7% 69.96% 47.27% 25.14% 52.55% 57.85% 73.36% 61.38% 89.34% 56.83%
internvl2-8b-mpo-cot 81.09% 69.77% 38.97% 36.7% 69.96% 47.27% 25.14% 52.55% 57.85% 73.36% 61.38% 89.34% 56.83%
ovis1.6-llama3.2-3b 79.96% 65.41% 40.11% 52.16% 66.86% 61.36% 34.26% 47.75% 51.38% 68.46% 52.96% 73.79% 56.73%
claude-3-opus-20240229 79.87% 48.26% 40.36% 44.71% 76.55% 49.55% 42.5% 53.15% 60.92% 56.31% 66.94% 65.35% 56.73%
ovis1.5-gemma2-9b 72.38% 62.21% 41.01% 48.08% 66.28% 50.45% 25.37% 63.06% 61.57% 77.26% 67.56% 67.28% 56.73%
ovis1.5-llama3-8b 81.46% 65.12% 44.44% 45.11% 66.86% 57.73% 29.56% 58.56% 61.97% 71.88% 63.08% 57.04% 56.73%
llama-3-vila1.5-8b-fix 75.66% 59.01% 42.4% 40.14% 72.09% 60.61% 29.56% 39.64% 59.06% 74.53% 55.56% 82.7% 56.48%
xgen-mm-phi3-dpo-r-v1.5 77.43% 70.93% 50.25% 41.19% 77.13% 56.67% 35.66% 47.75% 61.81% 73.91% 48.3% 46.91% 56.32%
llava_next_72b 79.21% 65.12% 61.03% 47.84% 68.6% 52.27% 30.12% 38.44% 60.11% 77.65% 70.07% 46.21% 56.25%
internvl2-8b 69.38% 67.44% 42.65% 39.26% 68.6% 53.18% 27.09% 36.94% 60.19% 74.77% 69.53% 84.61% 56.21%
internvl-chat-v1-2-plus 82.3% 66.86% 46.15% 44.36% 65.99% 57.88% 31.01% 29.13% 61.97% 78.6% 62.1% 67.88% 56.15%
minicpm-v-2_6 75.0% 77.33% 41.67% 47.44% 68.02% 51.36% 34.22% 54.05% 60.76% 74.77% 55.73% 50.0% 56.0%
llama-3-vila1.5-8b 74.25% 59.88% 58.09% 40.06% 57.56% 58.94% 30.96% 38.74% 55.66% 68.22% 56.54% 89.8% 55.82%
llava_next_mistral_7b 76.5% 65.7% 44.36% 47.76% 66.28% 42.27% 40.83% 64.86% 65.7% 71.03% 51.61% 45.11% 55.78%
minicpm-llama3-v-2_5 74.91% 69.77% 54.17% 33.25% 73.26% 50.0% 28.68% 64.86% 58.17% 71.65% 63.35% 47.59% 55.76%
internvl2-4b 76.87% 59.88% 37.75% 46.88% 72.67% 55.3% 30.63% 23.42% 63.83% 65.97% 69.89% 78.22% 55.63%
mini-internvl-chat-4b-v1-5 73.03% 56.4% 51.47% 46.55% 65.89% 60.0% 31.66% 48.65% 69.82% 64.64% 64.61% 48.38% 55.25%
mmalaya2 77.81% 66.86% 48.28% 42.47% 72.67% 59.39% 28.12% 28.83% 65.94% 81.07% 64.25% 46.86% 55.19%
llava_next_110b 73.03% 58.43% 45.1% 41.59% 65.41% 57.27% 29.89% 43.24% 65.94% 82.87% 66.67% 49.32% 54.88%
internvl-chat-v1-1 72.38% 68.02% 46.79% 37.83% 65.7% 61.06% 32.5% 38.74% 56.63% 74.85% 54.84% 65.12% 54.81%
internvl-chat-v1-5 77.81% 66.86% 46.32% 41.83% 72.09% 58.33% 28.72% 26.13% 65.05% 79.44% 60.13% 51.05% 54.6%
llava-onevision-qwen2-72b-ov-hf 81.84% 57.85% 40.36% 39.18% 64.63% 51.82% 29.05% 48.35% 63.27% 78.04% 66.31% 54.69% 54.46%
llava-onevision-qwen2-72b-si-hf 79.12% 54.07% 43.14% 38.62% 65.7% 51.21% 33.47% 51.05% 58.9% 82.4% 67.47% 45.83% 54.43%
llava-onevision-qwen2-72b-ov-chat-hf 79.4% 57.27% 32.76% 41.75% 66.67% 50.0% 25.79% 52.85% 63.67% 81.93% 64.61% 54.96% 54.19%
vila1.5-13b 63.86% 56.98% 37.75% 41.11% 62.21% 60.76% 32.31% 30.63% 58.17% 74.14% 54.66% 87.92% 54.11%
phi-4-multimodal-instruct 78.57% 52.59% 31.6% 36.02% 63.57% 53.64% 25.75% 51.95% 60.42% 67.29% 65.77% 75.46% 53.55%
mantis-8b-siglip-llama3 72.19% 54.36% 47.3% 37.42% 62.79% 52.27% 40.46% 15.02% 55.58% 72.74% 55.47% 73.22% 52.63%
gemma-3-4b-it 65.82% 47.13% 58.66% 37.3% 44.19% 55.15% 32.12% 50.45% 53.4% 63.44% 58.07% 83.85% 52.51%
claude-3-sonnet-20240229 71.16% 50.0% 34.4% 42.55% 62.69% 54.24% 42.88% 36.34% 52.43% 49.69% 57.17% 79.61% 52.51%
xinyuan-vl-2b-instruct 72.19% 72.38% 37.83% 37.02% 65.7% 55.61% 30.73% 60.66% 56.23% 64.8% 45.79% 47.52% 52.42%
mantis-8b-idefics2 77.72% 73.26% 45.34% 36.06% 68.02% 46.82% 29.61% 36.34% 50.24% 73.6% 55.82% 52.68% 52.25%
llava-llama-3-8b 85.11% 56.69% 44.12% 38.14% 62.79% 48.94% 31.56% 61.26% 53.48% 65.5% 53.76% 43.51% 52.19%
phi-3.5-vision 76.78% 52.03% 37.75% 34.21% 68.02% 60.61% 34.78% 39.34% 54.61% 63.94% 56.45% 56.86% 52.08%
internlm-xcomposer2d5-7b 79.78% 62.5% 53.1% 34.21% 48.84% 40.76% 34.87% 51.95% 49.76% 74.53% 56.81% 58.44% 51.87%
claude-3-haiku-20240307 72.28% 50.87% 33.74% 42.39% 65.5% 58.18% 41.71% 38.44% 46.76% 51.09% 55.02% 67.61% 51.75%
qwen2-vl-2b-instruct 72.38% 69.19% 34.23% 38.54% 70.93% 60.45% 30.63% 38.14% 46.28% 66.9% 54.93% 50.68% 51.66%
llava_next_interleave_7b_dpo 72.38% 59.01% 41.58% 39.1% 58.72% 50.15% 30.59% 45.05% 58.9% 71.03% 49.91% 59.45% 51.58%
mantis-8b-clip-llama3 67.32% 50.29% 44.53% 34.05% 58.14% 55.45% 35.52% 29.13% 51.94% 70.4% 50.0% 79.21% 51.41%
vintern-3b-beta 60.12% 68.97% 38.89% 34.32% 59.98% 53.94% 37.85% 31.23% 60.16% 63.47% 52.42% 66.73% 51.4%
h2ovl-mississippi-1b 63.76% 41.57% 43.95% 46.15% 74.03% 54.7% 38.22% 61.26% 45.87% 54.44% 45.97% 41.62% 51.2%
idefics3-8b-llama3 74.06% 58.43% 27.7% 35.5% 68.6% 65.3% 29.42% 45.05% 47.65% 65.26% 59.05% 52.06% 51.18%
llava-onevision-qwen2-7b-si-hf 71.72% 55.52% 41.91% 42.39% 62.21% 51.52% 31.52% 42.64% 54.29% 77.02% 49.01% 44.01% 50.84%
llava-onevision-qwen2-7b-ov-hf 64.14% 54.65% 43.3% 36.3% 67.15% 47.42% 31.47% 51.65% 55.18% 70.09% 56.9% 43.0% 50.82%
internvl2-2b 62.73% 61.92% 36.76% 45.75% 57.27% 47.42% 42.46% 41.44% 43.45% 63.63% 45.97% 61.56% 50.47%
llava-onevision-qwen2-7b-ov-chat-hf 73.6% 53.2% 31.37% 33.89% 59.3% 47.12% 30.87% 51.95% 59.39% 77.34% 56.18% 47.5% 50.37%
llava_next_interleave_7b 68.63% 60.47% 35.05% 41.43% 59.3% 52.42% 32.64% 27.03% 59.95% 70.48% 48.03% 58.36% 50.06%
deepseek-vl2-tiny 75.09% 63.37% 46.24% 44.95% 71.51% 59.55% 34.22% 31.53% 37.62% 58.1% 43.82% 41.18% 50.05%
hunyuan-vision 79.12% 60.76% 26.72% 30.05% 68.22% 54.39% 26.49% 36.94% 51.78% 69.39% 53.67% 55.92% 49.76%
h2ovl-mississippi-2b 72.0% 55.52% 45.67% 44.15% 45.35% 48.64% 27.51% 70.27% 56.07% 63.71% 43.73% 46.89% 49.67%
llama-3-longvila-8b-256frames 72.0% 54.36% 54.41% 34.54% 51.94% 45.91% 23.56% 61.86% 51.29% 60.9% 52.24% 51.07% 49.2%
internlm-xcomposer2-7b 75.56% 56.4% 36.93% 35.1% 67.25% 52.12% 37.8% 18.32% 41.34% 63.63% 48.92% 59.8% 49.05%
llama-3-longvila-8b-1024frames 69.29% 50.29% 55.88% 35.42% 54.65% 42.42% 25.98% 65.17% 51.38% 59.27% 47.85% 48.03% 48.92%
idefics_80b_instruct 65.82% 47.38% 35.7% 44.31% 59.59% 59.85% 44.37% 30.03% 36.97% 60.67% 46.95% 54.17% 48.89%
llava-video-7b-qwen2-video-only_multi_frame 82.49% 55.52% 34.56% 43.43% 52.33% 55.91% 34.82% 36.04% 34.87% 66.36% 56.81% 47.92% 48.72%
qwen_chat 65.64% 54.65% 31.78% 37.18% 60.17% 53.94% 41.67% 44.74% 46.04% 67.52% 42.56% 41.78% 48.62%
llama-3-longvila-8b-128frames 71.63% 52.62% 50.98% 39.98% 61.82% 50.0% 33.19% 18.62% 51.29% 60.83% 50.18% 51.01% 48.53%
vila1.5-3b 73.03% 47.97% 48.28% 41.03% 57.85% 52.12% 37.15% 26.73% 43.85% 66.04% 47.22% 45.94% 48.28%
mini-internvl-chat-2b-v1-5 74.44% 49.71% 39.79% 41.59% 48.84% 50.45% 37.99% 57.06% 51.54% 53.04% 42.83% 42.59% 48.08%
vila1.5-3b-s2 62.73% 53.49% 37.09% 40.22% 59.01% 55.0% 41.11% 30.03% 40.78% 68.61% 41.58% 47.92% 47.94%
llama-3-longvila-8b-512frames 69.66% 42.44% 54.17% 35.02% 44.48% 41.06% 22.77% 67.57% 56.72% 59.35% 51.16% 53.6% 47.83%
vintern-1b-v2 58.15% 53.78% 34.23% 44.55% 59.88% 54.39% 43.62% 29.73% 40.78% 52.1% 43.46% 40.48% 46.4%
eagle-x4-13b-plus 79.59% 55.23% 5.15% 32.13% 55.23% 28.03% 25.0% 75.98% 58.58% 73.83% 52.24% 35.09% 46.16%
eagle-x5-34b-chat 80.34% 48.26% 4.9% 27.0% 56.78% 30.45% 24.72% 61.26% 68.04% 78.12% 58.06% 38.03% 46.16%
deepseek-vl-7b-chat 81.84% 50.87% 4.41% 34.05% 57.56% 28.33% 31.05% 30.03% 52.67% 66.51% 51.88% 74.21% 46.01%
internvl2-1b 62.92% 55.23% 35.95% 35.9% 56.4% 57.27% 43.62% 22.82% 42.15% 60.51% 44.53% 37.72% 45.99%
smolvlm 72.57% 59.59% 22.39% 31.57% 67.05% 51.97% 33.85% 52.85% 39.24% 47.59% 44.71% 33.0% 45.77%
emu2-chat 64.61% 54.94% 43.79% 39.58% 57.27% 45.76% 33.24% 24.92% 52.99% 51.64% 44.71% 43.66% 45.67%
eagle-x5-13b-chat 82.49% 54.36% 1.23% 30.05% 60.47% 28.79% 25.23% 64.56% 57.2% 73.13% 49.73% 34.17% 45.23%
janus-pro-7b 78.28% 39.24% 3.43% 40.46% 59.3% 28.64% 33.57% 45.05% 47.09% 59.74% 48.39% 48.64% 43.93%
eagle-x4-8b-plus 65.64% 50.58% 7.11% 29.49% 49.61% 30.76% 25.19% 74.77% 56.23% 65.97% 39.25% 42.13% 43.34%
llava-onevision-qwen2-0.5b-ov-hf 60.11% 47.67% 27.12% 36.46% 56.59% 43.33% 38.04% 38.44% 34.87% 53.04% 38.17% 37.96% 42.65%
eagle-x5-34b-plus 67.7% 52.62% 7.11% 26.84% 47.29% 31.21% 22.02% 29.73% 70.95% 78.5% 53.14% 44.82% 42.39%
llava-onevision-qwen2-0.5b-si-hf 52.15% 39.53% 32.68% 28.37% 48.64% 51.52% 38.69% 49.55% 33.66% 51.56% 42.47% 38.14% 42.08%
openflamingo-9b-vitl-mpt7b 44.29% 32.27% 29.74% 36.86% 57.07% 58.64% 44.13% 56.46% 35.44% 31.15% 40.68% 27.11% 41.96%
mantis-8b-fuyu 58.05% 48.84% 39.22% 35.9% 52.81% 44.09% 38.73% 31.53% 36.89% 42.52% 41.31% 32.63% 41.7%
deepseek-vl-1.3b-chat 68.26% 42.44% 13.89% 41.75% 53.49% 37.12% 35.94% 28.53% 35.19% 55.3% 42.29% 40.75% 41.1%
qwen2.5-omni-7b 54.78% 37.5% 32.52% 31.33% 62.31% 54.24% 45.62% 39.94% 30.91% 28.66% 40.41% 22.61% 40.92%
eagle-x5-13b 58.33% 56.1% 6.13% 23.08% 45.93% 30.76% 31.24% 29.43% 55.83% 69.47% 47.49% 35.22% 39.57%
idefics_9b_instruct 48.97% 29.07% 26.8% 38.38% 53.59% 45.15% 34.03% 29.73% 38.03% 38.71% 40.32% 44.96% 39.4%
eagle-x5-7b 59.64% 53.2% 6.62% 23.0% 45.83% 30.3% 22.95% 28.83% 64.89% 71.18% 42.2% 36.01% 38.84%
janus-pro-1b 69.57% 34.01% 5.15% 40.63% 44.77% 33.18% 40.6% 18.62% 50.0% 49.22% 39.25% 35.9% 38.34%
janusflow-1.3b 65.17% 41.57% 4.66% 32.13% 41.28% 31.82% 41.34% 35.14% 44.58% 49.69% 35.57% 38.9% 38.25%
janus-1.3b 54.31% 30.81% 9.31% 41.59% 42.54% 20.3% 36.55% 54.65% 40.94% 45.72% 40.41% 31.73% 37.41%
kosmos2 49.25% 17.44% 35.95% 27.8% 40.5% 41.06% 32.59% 66.97% 23.95% 24.84% 36.11% 22.15% 34.93%
llava-next-video-7b-dpo_multi_frame 26.97% 1.45% 3.92% 4.57% 42.44% 21.82% 16.9% 53.15% 19.42% 57.4% 36.38% 16.58% 25.39%
llava-next-video-7b_multi_frame 21.07% 1.16% 3.92% 2.16% 37.79% 18.18% 18.72% 54.05% 13.92% 56.39% 37.81% 16.54% 23.87%

Image only

step-1.5v-mini 80.99% 71.88% 93.75% 74.62% 73.19% 55.75% 36.99% 53.02% 58.17% 86.03% 74.15% 95.83% 69.62%
step-1o-turbo-vision 82.12% 64.45% 71.88% 74.62% 76.69% 53.16% 40.01% 66.35% 61.49% 83.1% 73.96% 99.17% 69.49%
glm-4v-plus 91.67% 81.64% 75.0% 61.55% 80.42% 57.76% 26.94% 55.24% 62.86% 84.21% 66.57% 100.0% 68.22%
points-qwen-2.5-7b-chat 71.43% 71.43% 100.0% 71.43% 68.3% 60.34% 32.75% 42.86% 67.8% 76.27% 55.68% 97.92% 66.5%
nvlm-d-72b 80.99% 69.53% 62.5% 69.7% 68.18% 55.17% 32.36% 53.33% 64.48% 81.11% 77.08% 100.0% 66.03%
wemm-chat-2k-cn 80.24% 69.53% 75.0% 62.12% 67.37% 57.18% 32.19% 67.62% 68.2% 73.33% 57.77% 100.0% 65.79%
omchat 74.81% 69.92% 87.5% 56.44% 66.9% 56.61% 34.42% 64.44% 63.03% 79.76% 55.97% 94.79% 65.43%
step-1v-32k 76.5% 61.33% 87.5% 69.13% 65.5% 56.03% 33.05% 38.1% 68.04% 77.06% 60.32% 98.96% 64.5%
glm-4v-9b 80.34% 80.47% 84.38% 58.52% 66.43% 63.79% 39.27% 34.92% 53.07% 76.51% 57.39% 93.13% 64.17%
molmo-72b-0924 71.16% 56.64% 71.88% 81.06% 51.75% 54.02% 40.01% 48.89% 55.99% 80.56% 72.35% 100.0% 63.96%
cambrian_34b 77.62% 74.22% 25.0% 67.23% 76.46% 60.34% 25.11% 60.95% 69.09% 79.68% 59.47% 100.0% 63.08%
cambrian_13b 75.37% 75.78% 75.0% 69.7% 62.94% 55.75% 29.28% 43.81% 60.52% 76.59% 55.68% 95.0% 62.79%
internlm-xcomposer2-4khd-7b 73.03% 71.48% 37.5% 63.45% 69.7% 54.31% 32.99% 65.71% 51.94% 74.21% 54.45% 100.0% 61.36%
ovis1.6-gemma2-27b 82.4% 70.31% 37.5% 67.99% 65.03% 54.31% 18.95% 49.84% 65.7% 78.1% 74.62% 100.0% 61.3%
wemm 77.25% 65.63% 50.0% 60.8% 64.34% 65.52% 31.56% 40.32% 66.99% 70.87% 56.44% 100.0% 61.01%
reka-core 69.66% 62.89% 65.63% 44.89% 62.7% 56.32% 42.98% 48.25% 54.69% 76.35% 57.58% 100.0% 60.97%
bunny-llama3-8b 81.18% 64.06% 65.63% 47.73% 53.15% 62.07% 37.96% 46.35% 57.77% 77.78% 52.84% 100.0% 60.52%
cambrian_8b 78.65% 76.95% 25.0% 62.69% 77.62% 47.41% 21.35% 64.76% 67.07% 72.3% 47.54% 98.96% 60.25%
cogvlm2-llama3-chat-19b 76.31% 61.72% 78.13% 54.73% 60.37% 68.1% 43.66% 38.41% 48.87% 65.71% 40.91% 92.29% 60.08%
molmo-7b-d-0924 68.35% 41.41% 71.88% 74.05% 49.88% 63.79% 42.18% 31.11% 56.23% 75.79% 53.69% 98.33% 59.91%
mgm-7b-hd 75.66% 70.31% 50.0% 54.92% 62.94% 46.26% 38.41% 64.44% 65.37% 64.84% 47.73% 91.04% 59.85%
idefics2_8b 72.75% 71.09% 62.5% 54.36% 74.83% 48.56% 25.29% 39.37% 61.33% 73.33% 51.61% 97.57% 59.77%
360vl-70b 77.06% 63.67% 37.5% 57.77% 72.03% 57.47% 25.91% 31.43% 70.31% 74.44% 69.79% 97.5% 59.72%
internlm-xcomposer2-vl-7b 78.65% 78.91% 25.0% 63.26% 66.2% 52.3% 27.91% 63.49% 61.41% 69.13% 51.52% 98.13% 59.71%
glm-4v 76.78% 70.7% 78.13% 65.34% 58.62% 53.74% 39.95% 30.16% 38.75% 68.33% 48.39% 94.38% 59.31%
llava-internlm-7b 70.6% 59.77% 65.63% 65.34% 52.45% 59.2% 39.33% 38.73% 47.65% 68.65% 52.94% 100.0% 59.01%
aquila-vl-2b-llava-qwen 76.03% 63.67% 62.5% 53.79% 63.64% 63.22% 37.73% 15.56% 62.78% 71.19% 51.99% 96.46% 58.88%
parrot-14b 70.69% 65.63% 62.5% 57.01% 55.24% 60.63% 31.11% 46.67% 55.26% 76.51% 50.76% 89.79% 58.52%
wemm-chat-cn 85.96% 64.84% 37.5% 56.82% 54.55% 57.18% 29.85% 43.49% 65.86% 73.17% 57.67% 98.33% 58.42%
llava-v1.5-13b-xtuner 75.56% 57.03% 65.63% 59.85% 50.35% 61.49% 37.61% 39.05% 58.82% 70.32% 49.81% 91.46% 58.38%
reka-flash 65.73% 64.84% 43.75% 55.68% 59.67% 60.92% 41.5% 38.73% 49.76% 72.22% 53.5% 92.85% 57.6%
yi-vl-34b 72.57% 51.56% 25.0% 58.9% 65.03% 60.92% 28.31% 69.21% 51.78% 64.76% 52.65% 100.0% 57.49%
llama-3-mixsensev1_1 72.85% 66.41% 25.0% 71.02% 62.94% 60.63% 31.05% 37.14% 62.94% 69.05% 44.41% 96.25% 57.34%
monkey-chat 71.16% 52.73% 53.13% 47.16% 67.13% 61.78% 43.95% 39.68% 50.73% 65.48% 43.75% 89.58% 57.13%
moondream2 67.13% 70.31% 56.25% 51.89% 62.24% 50.57% 31.68% 60.95% 44.58% 62.06% 43.37% 95.0% 56.96%
parrot-7b 74.25% 59.77% 31.25% 57.2% 57.34% 50.29% 33.28% 58.1% 51.05% 69.29% 48.96% 97.08% 56.25%
llava_v1.5_13b 67.32% 66.02% 68.75% 45.08% 58.74% 55.46% 37.73% 31.11% 44.01% 68.02% 48.77% 91.04% 55.92%
phi-3-vision 82.49% 49.61% 34.38% 61.36% 68.3% 50.0% 26.88% 43.81% 57.12% 59.84% 55.11% 92.29% 55.82%
yi-vl-6b 75.19% 69.92% 40.63% 64.58% 54.2% 45.69% 25.11% 49.21% 49.35% 65.08% 48.77% 96.67% 55.33%
llava-v1.5-7b-xtuner 74.91% 58.2% 37.5% 60.04% 51.75% 58.62% 31.79% 26.35% 53.24% 65.24% 54.07% 98.13% 54.51%
molmo-7b-o-0924 64.04% 43.36% 71.88% 70.45% 46.04% 59.48% 46.29% 17.78% 40.94% 64.29% 47.35% 81.74% 54.27%
monkey 69.1% 48.83% 37.5% 65.53% 61.07% 67.24% 44.63% 35.87% 37.78% 62.78% 45.36% 69.93% 53.94%
mplug-owl2-llama2-7b 69.48% 54.69% 28.13% 58.33% 55.94% 56.03% 41.27% 38.1% 57.28% 55.48% 45.74% 88.33% 53.66%
instructblip-flan-t5-xxl 73.78% 58.2% 18.75% 50.0% 48.95% 44.54% 34.36% 66.03% 58.17% 68.1% 45.08% 92.78% 53.61%
moondream1 66.48% 54.69% 50.0% 51.14% 41.96% 52.87% 42.35% 41.27% 47.65% 56.27% 52.18% 97.64% 53.58%
reka-edge 61.05% 54.3% 34.38% 47.73% 57.34% 56.61% 44.98% 35.56% 46.84% 66.03% 48.77% 89.79% 53.54%
molmoe-1b-0924 70.88% 42.58% 59.38% 61.55% 42.07% 53.74% 44.35% 41.9% 38.67% 51.83% 44.98% 80.0% 52.15%
instructblip-flan-t5-xl 71.44% 47.27% 25.0% 58.9% 42.66% 39.37% 28.94% 66.67% 51.94% 66.35% 48.11% 95.14% 52.08%
mmalaya 61.8% 48.05% 59.38% 56.06% 54.9% 49.14% 46.46% 29.52% 53.64% 48.73% 50.85% 64.44% 51.82%
sharegpt4v_13b 71.07% 47.66% 12.5% 43.18% 58.74% 39.66% 39.55% 67.62% 28.4% 68.33% 51.8% 92.5% 51.6%
internlm-xcomposer2-vl-1_8b 68.16% 65.23% 15.63% 51.7% 69.23% 47.13% 23.23% 60.0% 41.5% 53.97% 48.11% 85.0% 51.56%
smolvlm-dpo 66.57% 58.59% 43.75% 42.23% 62.24% 56.03% 39.78% 65.08% 38.11% 45.48% 41.19% 60.69% 51.3%
minimonkey 68.73% 67.58% 18.75% 62.5% 53.85% 54.31% 43.49% 12.38% 42.88% 60.0% 47.25% 87.29% 51.27%
blip2-flan-t5-xxl 70.6% 58.98% 15.63% 63.64% 45.1% 50.86% 33.73% 11.43% 60.52% 68.49% 55.11% 91.81% 50.91%
paligemma-3b-mix-448 59.18% 46.48% 46.88% 44.51% 68.65% 58.33% 45.55% 22.22% 34.79% 45.08% 37.31% 84.51% 50.43%
sharegpt4v_7b 70.22% 56.64% 12.5% 28.79% 60.49% 50.57% 37.84% 48.25% 43.85% 59.52% 46.31% 90.63% 50.04%
blip2-flan-t5-xl 71.07% 54.69% 12.5% 50.76% 35.66% 42.24% 26.54% 60.95% 58.66% 65.95% 52.56% 93.13% 50.01%
cogvlm-chat 67.04% 35.94% 43.75% 41.48% 64.1% 44.54% 45.03% 57.46% 35.68% 40.71% 44.6% 69.17% 49.72%
transcore-m 52.53% 34.38% 62.5% 46.4% 53.15% 64.66% 51.71% 28.89% 45.47% 50.4% 48.96% 39.44% 48.66%
llava_v1.5_7b 59.93% 44.53% 37.5% 36.74% 50.23% 54.89% 44.69% 24.76% 33.01% 59.92% 44.6% 90.21% 48.54%
instructblip-vicuna-13b 48.13% 27.34% 62.5% 51.33% 56.99% 48.85% 47.6% 31.75% 34.71% 40.87% 41.95% 62.5% 47.33%
emu3-chat 67.98% 75.39% 37.5% 37.88% 57.34% 43.39% 20.95% 36.19% 40.94% 45.56% 44.32% 71.04% 46.59%
llava-v1.6-vicuna-7b-hf 51.4% 27.73% 62.5% 47.54% 48.72% 60.06% 49.14% 36.51% 30.91% 27.94% 41.76% 22.99% 42.94%
blip2-opt-6.7b 44.1% 21.48% 62.5% 40.15% 62.0% 49.71% 41.61% 61.27% 29.94% 26.98% 36.46% 22.99% 42.71%
minigpt_4_v2 38.67% 28.52% 68.75% 36.93% 51.98% 49.14% 45.78% 40.95% 25.97% 33.02% 39.96% 40.69% 42.64%
llava-v1.6-vicuna-13b-hf 42.42% 30.86% 62.5% 40.72% 55.01% 60.06% 49.83% 30.79% 30.91% 27.94% 39.49% 22.99% 42.22%
cogvlm-grounding-generalist 51.87% 25.39% 12.5% 53.6% 55.13% 44.83% 27.57% 67.94% 25.73% 43.41% 44.98% 41.74% 41.42%
falcon-11b-vlm 46.91% 26.17% 62.5% 29.36% 50.12% 54.89% 47.09% 32.7% 30.91% 27.94% 42.9% 22.99% 40.23%
xverse-v-13b 44.01% 53.13% 50.0% 26.52% 48.48% 43.97% 32.48% 43.17% 35.28% 21.11% 34.56% 22.57% 37.54%
fuyu-8b 48.5% 23.44% 12.5% 39.58% 44.52% 41.09% 37.84% 66.98% 32.69% 27.7% 42.9% 25.28% 37.24%
blip2-opt-2.7b 45.41% 13.67% 21.88% 41.48% 57.81% 40.52% 32.82% 61.27% 30.83% 26.98% 39.77% 22.99% 37.21%
visualglm-6b 49.25% 36.33% 12.5% 42.61% 39.16% 36.21% 28.54% 66.98% 26.86% 33.49% 42.8% 26.6% 36.25%
instructblip-vicuna-7b 51.22% 3.91% 3.13% 42.99% 45.69% 29.89% 4.0% 65.4% 28.64% 23.17% 41.57% 58.33% 32.67%
omnilmm-12b 44.66% 26.17% 12.5% 29.36% 39.63% 49.71% 43.66% 15.56% 30.91% 27.94% 34.38% 22.99% 32.2%
minigpt_4_13b 33.15% 26.17% 28.13% 35.23% 25.99% 38.79% 33.05% 40.95% 25.81% 24.37% 35.23% 24.38% 30.85%
minigpt_4_7b 32.87% 22.27% 18.75% 32.2% 30.07% 29.6% 34.42% 44.44% 25.97% 23.41% 26.8% 17.15% 28.58%
xcomposer 10.11% 30.47% 31.25% 3.41% 5.59% 17.24% 0.17% 5.71% 1.21% 3.57% 9.66% 14.17% 9.93%

Single Image

qwen2.5-vl-72b-instruct_video 79.59% 66.86% 38.81% 56.6% 84.69% 64.55% 27.0% 80.78% 68.45% 80.45% 85.66% 100.0% 67.75%
internvl2_5-78b 80.81% 59.59% 41.09% 63.37% 72.29% 59.7% 35.1% 65.47% 60.52% 75.0% 67.56% 95.83% 63.58%
llava-video-72b-qwen2 81.46% 57.85% 42.89% 52.6% 71.51% 57.88% 32.42% 54.05% 66.1% 80.06% 77.42% 100.0% 63.03%
qwen2.5-vl-7b-instruct_video 82.3% 60.76% 40.2% 61.11% 68.02% 58.79% 31.22% 65.17% 62.38% 76.4% 62.99% 98.33% 62.51%
llava-video-7b-qwen2 82.3% 63.79% 31.37% 70.83% 72.09% 51.06% 30.65% 51.05% 59.87% 78.53% 63.68% 97.86% 61.59%
videollama2-72b 85.11% 61.63% 38.24% 50.0% 68.41% 53.33% 27.4% 53.75% 61.73% 85.83% 69.71% 100.0% 61.12%
llama-3.2-90b-vision-instruct 70.04% 61.63% 37.83% 48.26% 62.21% 51.97% 35.56% 64.56% 59.87% 84.58% 67.56% 99.17% 60.59%
qwen2.5-vl-3b-instruct_video 70.6% 61.05% 30.88% 60.76% 64.34% 61.52% 33.9% 54.95% 60.84% 77.65% 63.17% 97.5% 60.29%
internvl2_5-26b 72.38% 66.57% 34.97% 59.55% 66.86% 60.91% 33.16% 41.14% 59.87% 76.87% 61.56% 97.64% 59.8%
internvl2_5-38b 80.24% 66.57% 37.91% 56.42% 61.43% 52.58% 26.83% 56.76% 51.21% 76.87% 70.79% 99.17% 59.46%
yi-vision-v2 79.87% 63.08% 29.33% 46.53% 67.64% 50.3% 34.25% 59.16% 64.64% 75.93% 60.39% 97.22% 59.44%
internvl2_5-8b 68.26% 60.47% 40.85% 53.13% 59.4% 50.91% 33.85% 54.35% 56.15% 72.66% 63.08% 100.0% 58.21%
llava-next-video-32b-qwen 75.84% 66.09% 34.8% 56.08% 72.38% 51.06% 28.88% 24.32% 50.65% 77.63% 67.28% 98.93% 57.59%
videollama2.1-7b-16f 66.76% 66.28% 31.13% 45.14% 65.12% 59.09% 38.07% 47.45% 55.99% 72.2% 52.06% 98.96% 57.46%
videollama2-7b-16f 74.16% 62.79% 45.26% 39.58% 65.99% 50.76% 31.91% 51.05% 49.76% 68.46% 47.31% 98.13% 56.12%
llama-3.2v-11b-cot 72.85% 59.88% 43.95% 47.4% 67.34% 48.64% 23.06% 43.54% 57.69% 67.37% 61.83% 95.42% 55.94%
llama-3.2-11b-vision-instruct 68.63% 61.63% 37.66% 45.49% 59.69% 56.97% 41.1% 43.24% 45.15% 64.41% 55.38% 92.85% 55.38%
internvl2_5-4b 72.57% 50.87% 35.29% 53.99% 64.53% 57.27% 37.04% 34.23% 53.8% 61.29% 53.85% 94.79% 55.37%
llava-video-7b-qwen2-video-only 82.49% 59.77% 37.99% 66.67% 50.58% 56.36% 35.45% 35.14% 34.71% 65.09% 56.14% 96.17% 55.17%
videollama2-7b 65.36% 56.98% 40.85% 53.99% 47.19% 46.67% 43.04% 48.05% 45.31% 65.26% 43.55% 95.42% 53.73%
videochat2_stage3_mistral_7b 69.94% 69.19% 48.69% 52.26% 59.59% 46.06% 37.79% 29.13% 51.62% 48.91% 42.65% 92.92% 53.41%
video-llava-7b 67.23% 43.02% 32.52% 59.55% 54.36% 43.33% 46.69% 40.84% 44.34% 50.55% 48.21% 91.04% 52.08%
videochat2_hd_stage4_mistral_7b_hf 71.16% 70.64% 37.34% 52.6% 48.45% 38.64% 40.7% 48.05% 41.59% 46.65% 42.92% 81.67% 50.81%
internvl2_5-2b 77.62% 54.65% 25.33% 44.27% 49.71% 54.39% 37.9% 42.94% 43.2% 59.11% 44.44% 84.17% 50.58%
internvl2_5-1b 68.82% 40.7% 37.42% 41.67% 58.72% 46.36% 32.08% 47.75% 39.48% 57.24% 45.34% 77.99% 49.07%
chat-univi 52.15% 29.36% 44.61% 56.42% 38.37% 51.97% 49.14% 30.03% 29.85% 32.71% 43.64% 66.32% 44.29%
chat-univi-13b 58.99% 35.76% 45.26% 44.1% 41.96% 40.45% 42.64% 22.82% 35.92% 40.81% 45.97% 49.65% 41.98%
video_chat_13b 44.66% 33.43% 40.85% 39.93% 46.03% 45.45% 43.95% 51.35% 32.44% 38.32% 36.11% 38.68% 41.45%
pllava-13b 43.45% 51.72% 34.72% 55.86% 50.68% 47.88% 46.7% 36.34% 31.34% 28.66% 39.87% 22.99% 41.43%
pllava-7b 40.17% 31.69% 36.44% 44.62% 57.66% 50.61% 45.03% 43.54% 30.91% 28.66% 38.8% 22.99% 40.43%
video_chat_7b 53.18% 30.52% 25.33% 39.58% 47.97% 32.88% 35.96% 54.35% 32.28% 35.2% 38.44% 37.36% 38.82%
llava-next-video-7b-dpo 18.26% 17.53% 32.52% 13.54% 45.35% 32.73% 13.47% 35.14% 7.85% 4.65% 21.49% 11.48% 21.8%
llava-next-video-7b 14.89% 16.67% 32.03% 11.46% 37.21% 31.21% 20.55% 32.13% 3.8% 3.15% 17.28% 8.98% 19.99%