LLM Image Description Benchmark Report
Models Tested
- llama-vision (Llama 3.2 Vision (11B)):
llama3.2-vision:11b
- llava (LLaVA (7B)):
llava:latest
- minicpm-v (MiniCPM-V):
minicpm-v:latest
- qwen3-vl (Qwen3-VL (8.8B)):
qwen3-vl:latest
Prompt
You are a helpful alt-text generator assisting visually impaired users. Generate a clear and concise caption (15-30 words) that highlights the most important subject and action. Focus only on essential details, avoiding unnecessary background elements. Use simple, everyday language and avoid overly descriptive or poetic words.
Model Settings
- temperature: 0.1
- top_k: 40
- top_p: 0.7
Performance Summary
| Model |
Success Rate |
Avg Time (s) |
Min Time (s) |
Max Time (s) |
Total Time (s) |
| llama-vision |
100.0% |
21.37 |
15.01 |
24.91 |
170.97 |
| llava |
100.0% |
10.51 |
3.14 |
11.96 |
84.06 |
| minicpm-v |
100.0% |
18.19 |
16.48 |
21.43 |
145.52 |
| qwen3-vl |
100.0% |
21.27 |
12.58 |
30.50 |
170.12 |
Detailed Results
HMOriginal-3F57EEFD-CE24-4DA8-BED1-9AAF8B4C0817.jpeg
| Model |
Time (s) |
Description |
| llama-vision |
23.90 |
A glass Aeropress coffee maker filled with dark brown liquid, likely coffee, on a wooden table. |
| llava |
3.14 |
A coffee press with ground coffee being steeped in hot water. |
| minicpm-v |
21.43 |
A cup of coffee is shown with 'AEROPRESS' branding, indicating it was made using an Aeropress brewing method. |
| qwen3-vl |
17.36 |
Aeropress coffee maker filled with brewed coffee, showing measurement levels 1 through 4. |
HMOriginal-70F34298-C096-4323-9E2C-B17712554CA9.jpeg
| Model |
Time (s) |
Description |
| llama-vision |
17.48 |
A plate of food on a table, accompanied by a glass of beer and a glass of white wine. |
| llava |
11.19 |
A delicious meal of fried fish with eggs, potatoes, salad, and beer on a dining table. |
| minicpm-v |
17.71 |
A plate of food with fried eggs, potatoes, salad, carrots, pickles, and a beer in a mug is shown on a wooden table. |
| qwen3-vl |
12.58 |
A plate with fried meat, sunny-side-up egg, roasted potatoes, shredded carrots, and a salad with cucumbers and greens. |
HMOriginal-7C20DE26-6983-48A2-A3EB-80A724393751.jpeg
| Model |
Time (s) |
Description |
| llama-vision |
23.74 |
A coffee maker and various coffee-related items on a wooden table. |
| llava |
11.33 |
A coffee press with a spoon inside, ready for use. |
| minicpm-v |
18.19 |
A coffee press with ground coffee inside is being used to make espresso-style coffee, placed on a wooden surface along with other kitchen items like a water jug and utensils. |
| qwen3-vl |
22.60 |
AeroPress coffee maker filled with brewed coffee, filter in place, and plunger positioned for brewing. |
HMOriginal-8F4A51DA-AF46-4EAF-8B75-21EF89F78BB5.jpeg
| Model |
Time (s) |
Description |
| llama-vision |
24.91 |
A hand holding a glass jar filled with paper stars and small lights. |
| llava |
11.96 |
Hand holding a jar filled with colorful origami stars, ready to be lit up. |
| minicpm-v |
19.02 |
A hand holds up a jar filled with colorful origami stars against a blue backdrop, showcasing various shades of yellow, gold, blue, and red paper art illuminated by small lights inside the container. |
| qwen3-vl |
25.52 |
A hand holds a glass jar with a green lid. |
HMOriginal-C6F176D6-4FAA-4AC8-AEB8-C2B722717DEC.jpeg
| Model |
Time (s) |
Description |
| llama-vision |
24.91 |
A glass container filled with sushi rolls, arranged in a neat and organized manner. |
| llava |
11.42 |
A tray of sushi rolls with salmon and avocado toppings. |
| minicpm-v |
17.91 |
A glass baking dish filled with assorted sushi rolls placed on a wooden table next to utensils for serving. |
| qwen3-vl |
30.50 |
Glass tray holds neatly arranged sushi rolls. |
IMG_6386.jpeg
| Model |
Time (s) |
Description |
| llama-vision |
15.01 |
A laptop screen with a black background and blue text, displaying the name "Chongqing Urlaub" and a date "03. |
| llava |
11.77 |
Person holding a ruler next to a laptop with a video playing. |
| minicpm-v |
16.48 |
A laptop displaying 'Chongqing Urlaub' with money being counted in front of it. |
| qwen3-vl |
13.06 |
Laptop screen shows "Chongqing Urlaub" and date 03. |
IMG_6408.jpeg
| Model |
Time (s) |
Description |
| llama-vision |
19.06 |
A serene landscape with a lake, trees, and a sunny sky. |
| llava |
11.55 |
A serene forest scene with a sunny day, trees, and a lake in the distance. |
| minicpm-v |
17.45 |
A tranquil forest scene with sunlight filtering through trees towards water in the distance. |
| qwen3-vl |
27.22 |
Sun shines through tall trees over a grassy path leading to a lake under a clear blue sky. |
IMG_6411.jpeg
| Model |
Time (s) |
Description |
| llama-vision |
21.96 |
A covered outdoor area with a roof made of red tiles, surrounded by various plants and objects. |
| llava |
11.70 |
A rustic outdoor shop with logs stacked outside at night. |
| minicpm-v |
17.33 |
A cozy outdoor area with colorful lights around a wooden structure at night time. |
| qwen3-vl |
21.28 |
A large stack of chopped wood logs sits under a covered structure with colorful string lights at night. |
