---
title: "Open Source LLM Leaderboard"
description: "Compare open-source and open-weight LLM benchmarks. Updated rankings across reasoning, coding, math, and multilingual tasks."
canonical_url: "https://www.vellum.ai/open-llm-leaderboard"
md_url: "https://www.vellum.ai/md/open-llm-leaderboard"
type: "leaderboard"
---

# Open Source LLM Leaderboard

Compare open-source and open-weight LLM benchmarks. Updated rankings across reasoning, coding, math, and multilingual tasks.

## Featured Benchmark

### Best Overall (Humanity's Last Exam)

| Rank | Model | Score |
| --- | --- | --- |
| 1 | GLM 5.2 | 54.7% |
| 2 | Kimi K2.6 | 54% |
| 3 | DeepSeek V4 Flash | 51.6% |
| 4 | DeepSeek V4 Pro | 48.2% |
| 5 | Kimi K2 Thinking | 44.9% |

## Top Open Source Models per Benchmark

### Best in Reasoning (GPQA Diamond)

| Rank | Model | Score |
| --- | --- | --- |
| 1 | MiniMax M3 | 93% |
| 2 | GLM 5.2 | 91.2% |
| 3 | Kimi K2.6 | 90.5% |
| 4 | DeepSeek V4 Pro | 90.1% |
| 5 | DeepSeek V4 Flash | 88.1% |

### Best in Agentic Coding (SWE Bench)

| Rank | Model | Score |
| --- | --- | --- |
| 1 | DeepSeek V4 Pro | 80.6% |
| 2 | MiniMax M3 | 80.5% |
| 3 | Kimi K2.6 | 80.2% |
| 4 | DeepSeek V4 Flash | 79% |
| 5 | Kimi K2.5 | 76.8% |

### Best in Computer Use (OSWorld)

| Rank | Model | Score |
| --- | --- | --- |
| 1 | Kimi K2.6 | 73.1% |
| 2 | MiniMax M3 | 70.1% |

### Best in Browsing (BrowseComp)

| Rank | Model | Score |
| --- | --- | --- |
| 1 | DeepSeek V4 Flash | 85.9% |
| 2 | MiniMax M3 | 83.5% |
| 3 | DeepSeek V4 Pro | 83.4% |
| 4 | Kimi K2.6 | 83.2% |

### Best in Terminal Use (Terminal-Bench 2.1)

| Rank | Model | Score |
| --- | --- | --- |
| 1 | GLM 5.2 | 81% |
| 2 | MiniMax M3 | 66% |
| 3 | Kimi K2.5 | 50.8% |
| 4 | Kimi K2 Thinking | 35.7% |

### Best in Visual Reasoning (ARC-AGI 2)

| Rank | Model | Score |
| --- | --- | --- |
| 1 | Kimi K2.5 | 12% |

## Fastest Models (Tokens/sec)



| Rank | Model | Throughput |

| --- | --- | --- |

| 1 | Llama 4 Scout | 2600 t/s |

| 2 | Llama 3.1 405b | 969 t/s |

| 3 | GLM 5.2 | 347 t/s |

| 4 | Kimi K2.6 | 342.6 t/s |

| 5 | Kimi K2.5 | 337.7 t/s |

## Lowest Latency (TTFT)



| Rank | Model | Latency |

| --- | --- | --- |

| 1 | Llama 4 Scout | 0.33s |

| 2 | Llama 4 Maverick | 0.45s |

| 3 | Kimi K2.6 | 0.68s |

| 4 | Kimi K2.5 | 0.69s |

| 5 | Llama 3.1 405b | 0.73s |

## Cheapest Models (per 1M tokens)



| Rank | Model | Input / Output |

| --- | --- | --- |

| 1 | Llama 4 Scout | $0.11 / $0.34 |

| 2 | DeepSeek V4 Flash | $0.14 / $0.28 |

| 3 | Llama 4 Maverick | $0.2 / $0.6 |

| 4 | DeepSeek V3 0324 | $0.27 / $1.1 |

| 5 | DeepSeek V4 Pro | $0.435 / $0.87 |

## All Open Source Models



| Model | Provider | Context Window | Input Cost (1M) | Output Cost (1M) | Knowledge Cutoff |

| --- | --- | --- | --- | --- | --- |

| Kimi K2.5 | Kimi | 33,000 | $0.6 | $2.5 | Apr 2024 |

| Llama 3.1 405b | Meta | 4096 | $3.5 | $3.5 | Dec 2023 |

| Llama 3.3 70b | Meta | 32,768 | $0.59 | $0.7 | July 2024 |

| DeepSeek V3 0324 | DeepSeek | 8,000 | $0.27 | $1.1 | Dec 2024 |

| Qwen2.5-VL-32B | Qwen | 8,000 | - | - | Dec 2024 |

| DeepSeek-R1 | DeepSeek | 8,000 | $0.55 | $2.19 | Dec 2024 |

| Gemma 3 27b | Google | 8192 | $0.07 | $0.07 | Nov 2024 |

| Llama 4 Maverick | Meta | 8,000 | $0.2 | $0.6 | November 2024 |

| Llama 4 Scout | Meta | 8,000 | $0.11 | $0.34 | November 2024 |

| Llama 4 Behemoth | Meta | - | - | - | November 2024 |

| Nemotron Ultra 253B | NVIDIA | - | - | - | - |

| GPT oss 120b | OpenAI | 131,072 | $0.15 | $0.6 | April 2025 |

| GPT oss 20b | OpenAI | 131,072 | $0.08 | $0.35 | April 2025 |

| Kimi K2 Thinking | Kimi | 16,400 | $0.6 | $2.5 | April 2025 |

| DeepSeek V4 Flash | DeepSeek | 384000 | $0.14 | $0.28 | Jan 2026 |

| DeepSeek V4 Pro | DeepSeek | 384000 | $0.435 | $0.87 | Jan 2026 |

| GLM 5.2 | Z-AI | 128,000 | $0.95 | $3 | Mar 2026 |

| Kimi K2.6 | Kimi | 256,000 | $0.95 | $4 | - |

| MiniMax M3 | MiniMax | 512,000 | $0.6 | $2.4 | Mar 2026 |