x

Evaluate your AI workflows with Vellum

Send me a demo
Thank you!
Your submission has been received!
Oops! Something went wrong while submitting the form.
OS LeaderboardCompare models
updated
17 April 2025

LLM Leaderboard

This LLM leaderboard displays the latest public benchmark performance for SOTA model versions released after April 2024. The data comes from model providers as well as independently run evaluations by Vellum or the open-source community. We feature results from non-saturated benchmarks, excluding outdated benchmarks (e.g. MMLU). If you want to evaluate these models on your use-cases, try Vellum Evals.

Top models per tasks
Best in Reasoning (GPQA Diamond)
Score (Percentage)
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Grok 3 [Beta]
84.6
Gemini 2.5 Pro
84
OpenAI o3
83.3
OpenAI o4-mini
81.4
OpenAI o3-mini
79.7
Best in High School Math (AIME 2024)
Score (Percentage)
100%
90%
80%
70%
60%
50%
OpenAI o4-mini
93.4
Grok 3 [Beta]
93.3
Gemini 2.5 Pro
92
OpenAI o3
91.6
Gemini 2.5 Flash
88
Best in Agentic Coding (SWE Bench)
Score (Percentage)
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Claude 3.7 Sonnet [R]
70.3
OpenAI o3
69.1
OpenAI o4-mini
68.1
Gemini 2.5 Pro
63.8
Claude 3.7 Sonnet
62.3
Idependent evals
Best in Tool Use (BFCL)
Score (Percentage)
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Llama 3.1 405b
81.1
Llama 3.3 70b
77.3
GPT-4o
72.08
GPT-4.5
69.94
Nova Pro
68.4
Best in Adaptive Reasoning (GRIND)
Score (Percentage)
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Gemini 2.5 Pro
82.1
Claude 3.7 Sonnet [R]
60.7
Nemotron Ultra 253B
57.1
OpenAI o1
57.1
Llama 4 Maverick
53.6
Best Overall (Humanity's Last Exam)
Score (Percentage)
50
40
30
20
10
0
OpenAI o3
20.32
Gemini 2.5 Pro
18.8
OpenAI o4-mini
14.28
OpenAI o3-mini
14
Gemini 2.5 Flash
12.1
Fastest and most affordable models
Fastest Models
Tokens/seconds
2500
2000
1500
1000
500
0
Llama 4 Scout
2600
Llama 3.3 70b
2500
Llama 3.1 70b
2100
Llama 3.1 8b
1800
Llama 3.1 405b
969
Lowest Latency (TTFT)
Seconds to first token
0.6s
0.5s
0.4s
0.3s
0.2s
0.1s
0.0s
Nova Micro
0.3
Llama 3.1 8b
0.32
Llama 4 Scout
0.33
Gemini 2.0 Flash
0.34
GPT-4o mini
0.35
Cheapest Models
Input
Output
USD per 1M tokens
0.8
0.65
0.5
0.35
0.2
0.05
Nova Micro
$
0.04
$
0.14
Gemma 3 27b
$
0.07
$
0.07
Gemini 1.5 Flash
$
0.075
$
0.3
Gemini 2.0 Flash
$
0.1
$
0.4
Compare models
Select two models to compare
GPT-4o
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
vs
Claude 3.5 Sonnet
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Model
Context size
Cutoff date
I/O cost
Max output
Latency
Speed
Gemini 2.5 Flash
1,000,000
n/a
May 2024
n/a
$
n/a
0.15
/
$
0.6
30,000
n/a
0.35
s
n/a
200
t/s
n/a
OpenAI o3
200,000
n/a
May 2024
n/a
$
n/a
10
/
$
40
100,000
n/a
28
s
n/a
94
t/s
n/a
OpenAI o4-mini
200,000
n/a
May 2024
n/a
$
n/a
1.1
/
$
4.4
100,000
n/a
35.3
s
n/a
135
t/s
n/a
Nemotron Ultra 253B
n/a
n/a
$
n/a
/
$
n/a
s
n/a
t/s
n/a
GPT-4.1 nano
1,000,000
n/a
December 2024
n/a
$
n/a
0.1
/
$
0.4
32,000
n/a
s
n/a
t/s
n/a
GPT-4.1 mini
1,000,000
n/a
December 2024
n/a
$
n/a
0.4
/
$
1.6
16,000
n/a
s
n/a
t/s
n/a
GPT-4.1
1,000,000
n/a
December 2024
n/a
$
n/a
2
/
$
8
16,000
n/a
s
n/a
t/s
n/a
Llama 4 Behemoth
n/a
November 2024
n/a
$
n/a
/
$
n/a
s
n/a
t/s
n/a
Llama 4 Scout
10,000,000
n/a
November 2024
n/a
$
n/a
0.11
/
$
0.34
8,000
n/a
0.33
s
n/a
2600
t/s
n/a
Llama 4 Maverick
10,000,000
n/a
November 2024
n/a
$
n/a
0.2
/
$
0.6
8,000
n/a
0.45
s
n/a
126
t/s
n/a
Gemma 3 27b
128,000
n/a
Nov 2024
n/a
$
n/a
0.07
/
$
0.07
8192
n/a
0.72
s
n/a
59
t/s
n/a
Grok 3 mini [Beta]
/
n/a
Nov 2024
n/a
$
n/a
/
$
n/a
s
n/a
t/s
n/a
Grok 3 [Beta]
/
n/a
Nov 2024
n/a
$
n/a
/
$
n/a
s
n/a
t/s
n/a
Gemini 2.5 Pro
1,000,000
n/a
Nov 2024
n/a
$
n/a
1.25
/
$
10
65,000
n/a
30
s
n/a
191
t/s
n/a
Claude 3.7 Sonnet
200,000
n/a
Nov 2024
n/a
$
n/a
3
/
$
15
128,000
n/a
0.91
s
n/a
78
t/s
n/a
GPT-4.5
128,000
n/a
Nov 2024
n/a
$
n/a
75
/
$
150
16,384
n/a
1.25
s
n/a
48
t/s
n/a
Claude 3.7 Sonnet [R]
200,000
n/a
Nov 2024
n/a
$
n/a
3
/
$
15
64,000
n/a
0.95
s
n/a
78
t/s
n/a
DeepSeek-R1
128,000
n/a
Dec 2024
n/a
$
n/a
0.55
/
$
2.19
8,000
n/a
4
s
n/a
24
t/s
n/a
OpenAI o3-mini
200,000
n/a
Dec 2024
n/a
$
n/a
1.1
/
$
4.4
8,000
n/a
14
s
n/a
214
t/s
n/a
OpenAI o1-mini
128,000
n/a
Dec 2024
n/a
$
n/a
3
/
$
12
8,000
n/a
11.43
s
n/a
220
t/s
n/a
Qwen2.5-VL-32B
131,000
n/a
Dec 2024
n/a
$
n/a
/
$
8,000
n/a
s
n/a
t/s
n/a
DeepSeek V3 0324
128,000
n/a
Dec 2024
n/a
$
n/a
0.27
/
$
1.1
8,000
n/a
4
s
n/a
33
t/s
n/a
OpenAI o1
200,000
n/a
Oct 2023
n/a
$
n/a
15
/
$
60
100,000
n/a
30
s
n/a
100
t/s
n/a
Gemini 2.0 Flash
1,000,000
n/a
Aug 2024
n/a
$
n/a
0.1
/
$
0.4
8,192
n/a
0.34
s
n/a
257
t/s
n/a
Llama 3.3 70b
128,000
n/a
July 2024
n/a
$
n/a
0.59
/
$
0.7
32,768
n/a
0.52
s
n/a
2500
t/s
n/a
Nova Micro
128,000
n/a
July 2024
n/a
$
n/a
0.04
/
$
0.14
4096
n/a
0.3
s
n/a
t/s
n/a
Nova Lite
300,000
n/a
July 2024
n/a
$
n/a
/
$
4096
n/a
0.4
s
n/a
t/s
n/a
Nova Pro
300,000
n/a
July 2024
n/a
$
n/a
1
/
$
4
4096
n/a
0.64
s
n/a
128
t/s
n/a
Claude 3.5 Haiku
200,000
n/a
July 2024
n/a
$
n/a
0.8
/
$
4
4096
n/a
0.88
s
n/a
66
t/s
n/a
Llama 3.1 405b
128,000
n/a
Dec 2023
n/a
$
n/a
3.5
/
$
3.5
4096
n/a
0.73
s
n/a
969
t/s
n/a
Llama 3.1 70b
128,000
n/a
Dec 2023
n/a
$
n/a
/
$
4096
n/a
s
n/a
2100
t/s
n/a
Llama 3.1 8b
128,000
n/a
Dec 2023
n/a
$
n/a
/
$
4096
n/a
0.32
s
n/a
1800
t/s
n/a
Gemini 1.5 Flash
1,000,000
n/a
May 2024
n/a
$
n/a
0.075
/
$
0.3
4096
n/a
1.06
s
n/a
166
t/s
n/a
Gemini 1.5 Pro
2,000,000
n/a
May 2024
n/a
$
n/a
/
$
4096
n/a
s
n/a
61
t/s
n/a
GPT-3.5 Turbo
16,400
n/a
Sept 2023
n/a
$
n/a
/
$
4096
n/a
s
n/a
t/s
n/a
GPT-4o mini
128,000
n/a
Oct 2023
n/a
$
n/a
0.15
/
$
0.6
4096
n/a
0.35
s
n/a
65
t/s
n/a
GPT-Turbo
128,000
n/a
Dec 2023
n/a
$
n/a
/
$
4096
n/a
s
n/a
t/s
n/a
GPT-4o
128,000
n/a
Oct 2023
n/a
$
n/a
2.5
/
$
10
4096
n/a
0.51
s
n/a
143
t/s
n/a
Claude 3 Haiku
200,000
n/a
Apr 2024
n/a
$
n/a
/
$
4096
n/a
s
n/a
t/s
n/a
Claude 3.5 Sonnet
200,000
n/a
Apr 2024
n/a
$
n/a
3
/
$
15
4096
n/a
1.22
s
n/a
78
t/s
n/a
Claude 3 Opus
200,000
n/a
Aug 2023
n/a
$
n/a
/
$
4096
n/a
s
n/a
t/s
n/a
GPT-4
8192
n/a
Dec 2023
n/a
$
n/a
/
$
4096
n/a
s
n/a
t/s
n/a
Gemini 2.5 Flash
1,000,000
n/a
May 2024
n/a
$
0.15
n/a
/
$
0.6
30,000
n/a
0.35
s
n/a
200
t/s
n/a
OpenAI o3
200,000
n/a
May 2024
n/a
$
10
n/a
/
$
40
100,000
n/a
28
s
n/a
94
t/s
n/a
OpenAI o4-mini
200,000
n/a
May 2024
n/a
$
1.1
n/a
/
$
4.4
100,000
n/a
35.3
s
n/a
135
t/s
n/a
Nemotron Ultra 253B
n/a
n/a
$
n/a
/
$
n/a
s
n/a
t/s
n/a
GPT-4.1 nano
1,000,000
n/a
December 2024
n/a
$
0.1
n/a
/
$
0.4
32,000
n/a
s
n/a
t/s
n/a
GPT-4.1 mini
1,000,000
n/a
December 2024
n/a
$
0.4
n/a
/
$
1.6
16,000
n/a
s
n/a
t/s
n/a
GPT-4.1
1,000,000
n/a
December 2024
n/a
$
2
n/a
/
$
8
16,000
n/a
s
n/a
t/s
n/a
Llama 4 Behemoth
n/a
November 2024
n/a
$
n/a
/
$
n/a
s
n/a
t/s
n/a
Llama 4 Scout
10,000,000
n/a
November 2024
n/a
$
0.11
n/a
/
$
0.34
8,000
n/a
0.33
s
n/a
2600
t/s
n/a
Llama 4 Maverick
10,000,000
n/a
November 2024
n/a
$
0.2
n/a
/
$
0.6
8,000
n/a
0.45
s
n/a
126
t/s
n/a
Gemma 3 27b
128,000
n/a
Nov 2024
n/a
$
0.07
n/a
/
$
0.07
8192
n/a
0.72
s
n/a
59
t/s
n/a
Grok 3 mini [Beta]
/
n/a
Nov 2024
n/a
$
n/a
/
$
n/a
s
n/a
t/s
n/a
Grok 3 [Beta]
/
n/a
Nov 2024
n/a
$
n/a
/
$
n/a
s
n/a
t/s
n/a
Gemini 2.5 Pro
1,000,000
n/a
Nov 2024
n/a
$
1.25
n/a
/
$
10
65,000
n/a
30
s
n/a
191
t/s
n/a
Claude 3.7 Sonnet
200,000
n/a
Nov 2024
n/a
$
3
n/a
/
$
15
128,000
n/a
0.91
s
n/a
78
t/s
n/a
GPT-4.5
128,000
n/a
Nov 2024
n/a
$
75
n/a
/
$
150
16,384
n/a
1.25
s
n/a
48
t/s
n/a
Claude 3.7 Sonnet [R]
200,000
n/a
Nov 2024
n/a
$
3
n/a
/
$
15
64,000
n/a
0.95
s
n/a
78
t/s
n/a
DeepSeek-R1
128,000
n/a
Dec 2024
n/a
$
0.55
n/a
/
$
2.19
8,000
n/a
4
s
n/a
24
t/s
n/a
OpenAI o3-mini
200,000
n/a
Dec 2024
n/a
$
1.1
n/a
/
$
4.4
8,000
n/a
14
s
n/a
214
t/s
n/a
OpenAI o1-mini
128,000
n/a
Dec 2024
n/a
$
3
n/a
/
$
12
8,000
n/a
11.43
s
n/a
220
t/s
n/a
Qwen2.5-VL-32B
131,000
n/a
Dec 2024
n/a
$
n/a
/
$
8,000
n/a
s
n/a
t/s
n/a
DeepSeek V3 0324
128,000
n/a
Dec 2024
n/a
$
0.27
n/a
/
$
1.1
8,000
n/a
4
s
n/a
33
t/s
n/a
OpenAI o1
200,000
n/a
Oct 2023
n/a
$
15
n/a
/
$
60
100,000
n/a
30
s
n/a
100
t/s
n/a
Gemini 2.0 Flash
1,000,000
n/a
Aug 2024
n/a
$
0.1
n/a
/
$
0.4
8,192
n/a
0.34
s
n/a
257
t/s
n/a
Llama 3.3 70b
128,000
n/a
July 2024
n/a
$
0.59
n/a
/
$
0.7
32,768
n/a
0.52
s
n/a
2500
t/s
n/a
Nova Micro
128,000
n/a
July 2024
n/a
$
0.04
n/a
/
$
0.14
4096
n/a
0.3
s
n/a
t/s
n/a
Nova Lite
300,000
n/a
July 2024
n/a
$
n/a
/
$
4096
n/a
0.4
s
n/a
t/s
n/a
Nova Pro
300,000
n/a
July 2024
n/a
$
1
n/a
/
$
4
4096
n/a
0.64
s
n/a
128
t/s
n/a
Claude 3.5 Haiku
200,000
n/a
July 2024
n/a
$
0.8
n/a
/
$
4
4096
n/a
0.88
s
n/a
66
t/s
n/a
Llama 3.1 405b
128,000
n/a
Dec 2023
n/a
$
3.5
n/a
/
$
3.5
4096
n/a
0.73
s
n/a
969
t/s
n/a
Llama 3.1 70b
128,000
n/a
Dec 2023
n/a
$
n/a
/
$
4096
n/a
s
n/a
2100
t/s
n/a
Llama 3.1 8b
128,000
n/a
Dec 2023
n/a
$
n/a
/
$
4096
n/a
0.32
s
n/a
1800
t/s
n/a
Gemini 1.5 Flash
1,000,000
n/a
May 2024
n/a
$
0.075
n/a
/
$
0.3
4096
n/a
1.06
s
n/a
166
t/s
n/a
Gemini 1.5 Pro
2,000,000
n/a
May 2024
n/a
$
n/a
/
$
4096
n/a
s
n/a
61
t/s
n/a
GPT-3.5 Turbo
16,400
n/a
Sept 2023
n/a
$
n/a
/
$
4096
n/a
s
n/a
t/s
n/a
GPT-4o mini
128,000
n/a
Oct 2023
n/a
$
0.15
n/a
/
$
0.6
4096
n/a
0.35
s
n/a
65
t/s
n/a
GPT-Turbo
128,000
n/a
Dec 2023
n/a
$
n/a
/
$
4096
n/a
s
n/a
t/s
n/a
GPT-4o
128,000
n/a
Oct 2023
n/a
$
2.5
n/a
/
$
10
4096
n/a
0.51
s
n/a
143
t/s
n/a
Claude 3 Haiku
200,000
n/a
Apr 2024
n/a
$
n/a
/
$
4096
n/a
s
n/a
t/s
n/a
Claude 3.5 Sonnet
200,000
n/a
Apr 2024
n/a
$
3
n/a
/
$
15
4096
n/a
1.22
s
n/a
78
t/s
n/a
Claude 3 Opus
200,000
n/a
Aug 2023
n/a
$
n/a
/
$
4096
n/a
s
n/a
t/s
n/a
GPT-4
8192
n/a
Dec 2023
n/a
$
n/a
/
$
4096
n/a
s
n/a
t/s
n/a
Standard Benchmarks
Dynamic Chart
BENCHMARKS
Model Comparison
Showing 0 out of 20 results
Reset All
Models
Average
GRIND
AIME 2024
GPQA
SWE Bench
MATH 500
BFCL
Alder Polyglot
Gemini 2.5 Flash
1,000,000
n/a
%
88
n/a
%
78.3
n/a
%
%
n/a
%
n/a
%
n/a
51.1
%
n/a
OpenAI o3
200,000
n/a
%
91.6
n/a
%
83.3
n/a
%
69.1
%
n/a
%
n/a
%
n/a
81.3
%
n/a
OpenAI o4-mini
200,000
50
n/a
%
93.4
n/a
%
81.4
n/a
%
68.1
%
n/a
%
n/a
%
n/a
68.9
%
n/a
Nemotron Ultra 253B
57.1
n/a
%
80.08
n/a
%
76
n/a
%
%
n/a
%
n/a
%
n/a
%
n/a
GPT-4.1 nano
1,000,000
n/a
%
29.4
n/a
%
50.3
n/a
%
%
n/a
%
n/a
%
n/a
9.8
%
n/a
GPT-4.1 mini
1,000,000
n/a
%
49.6
n/a
%
65
n/a
%
23.6
%
n/a
%
n/a
%
n/a
34.7
%
n/a
GPT-4.1
1,000,000
n/a
%
48.1
n/a
%
66.3
n/a
%
55
%
n/a
%
n/a
%
n/a
%
n/a
Llama 4 Behemoth
n/a
%
n/a
%
73.7
n/a
%
%
n/a
95
%
n/a
%
n/a
%
n/a
Llama 4 Scout
10,000,000
n/a
%
n/a
%
57.2
n/a
%
%
n/a
%
n/a
%
n/a
%
n/a
Llama 4 Maverick
10,000,000
53.6
n/a
%
n/a
%
69.8
n/a
%
%
n/a
%
n/a
%
n/a
15.6
%
n/a
Gemma 3 27b
128,000
n/a
%
n/a
%
42.4
n/a
%
10.2
%
n/a
89
%
n/a
59.11
%
n/a
4.9
%
n/a
Grok 3 [Beta]
/
n/a
%
93.3
n/a
%
84.6
n/a
%
%
n/a
%
n/a
%
n/a
%
n/a
Gemini 2.5 Pro
1,000,000
82.1
n/a
%
92
n/a
%
84
n/a
%
63.8
%
n/a
%
n/a
%
n/a
72.9
%
n/a
Claude 3.7 Sonnet
200,000
n/a
%
23.3
n/a
%
68
n/a
%
62.3
%
n/a
82.2
%
n/a
58.3
%
n/a
60.4
%
n/a
GPT-4.5
128,000
46.4
n/a
%
36.7
n/a
%
71.4
n/a
%
38
%
n/a
%
n/a
69.94
%
n/a
44.9
%
n/a
Claude 3.7 Sonnet [R]
200,000
60.7
n/a
%
61.3
n/a
%
78.2
n/a
%
70.3
%
n/a
96.2
%
n/a
58.3
%
n/a
64.9
%
n/a
DeepSeek-R1
128,000
53.6
n/a
%
79.8
n/a
%
71.5
n/a
%
49.2
%
n/a
97.3
%
n/a
57.53
%
n/a
64
%
n/a
OpenAI o3-mini
200,000
50
n/a
%
87.3
n/a
%
79.7
n/a
%
61
%
n/a
97.9
%
n/a
65.12
%
n/a
60.4
%
n/a
OpenAI o1-mini
128,000
n/a
%
63.6
n/a
%
60
n/a
%
%
n/a
90
%
n/a
52.2
%
n/a
32.9
%
n/a
Qwen2.5-VL-32B
131,000
42.9
n/a
%
n/a
%
46
n/a
%
18.8
%
n/a
82.2
%
n/a
62.79
%
n/a
62.84
%
n/a
DeepSeek V3 0324
128,000
n/a
%
59.4
n/a
%
64.8
n/a
%
38.8
%
n/a
94
%
n/a
58.55
%
n/a
55.1
%
n/a
OpenAI o1
200,000
57.1
n/a
%
79.2
n/a
%
75.7
n/a
%
48.9
%
n/a
96.4
%
n/a
67.87
%
n/a
61.7
%
n/a
Gemini 2.0 Flash
1,000,000
53.6
n/a
%
n/a
%
62.1
n/a
%
51.8
%
n/a
89.7
%
n/a
60.42
%
n/a
22.2
%
n/a
Llama 3.3 70b
128,000
n/a
%
n/a
%
50.5
n/a
%
%
n/a
77
%
n/a
77.3
%
n/a
51.43
%
n/a
Nova Pro
300,000
n/a
%
n/a
%
46.9
n/a
%
%
n/a
76.6
%
n/a
68.4
%
n/a
61.38
%
n/a
Claude 3.5 Haiku
200,000
n/a
%
n/a
%
41.6
n/a
%
40.6
%
n/a
69.4
%
n/a
54.31
%
n/a
28
%
n/a
Llama 3.1 405b
128,000
n/a
%
23.3
n/a
%
49
n/a
%
%
n/a
73.8
%
n/a
81.1
%
n/a
%
n/a
GPT-4o mini
128,000
n/a
%
n/a
%
40.2
n/a
%
%
n/a
70.2
%
n/a
64.1
%
n/a
3.6
%
n/a
GPT-4o
128,000
n/a
%
13.4
n/a
%
56.1
n/a
%
31
%
n/a
60.3
%
n/a
72.08
%
n/a
27.1
%
n/a
Claude 3.5 Sonnet
200,000
n/a
%
16
n/a
%
65
n/a
%
49
%
n/a
78
%
n/a
56.46
%
n/a
51.6
%
n/a
*
This comparison view excludes other benchmarks and focuses on MMLU, HellaSwag, HumanEval, BBHard, GSM-8K, and MATH due to the absence of data in the model reports.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
MODEL COMPARISON
Context window, cost and speed comparison
Showing 0 out of 20 results
Reset All
Models
Context Window
Input Cost / 1M tokens
Gemini 2.5 Flash
1,000,000
$
0.15
n/a
$
0.6
n/a
200
n/a
t/s
0.35
seconds
n/a
OpenAI o3
200,000
$
10
n/a
$
40
n/a
94
n/a
t/s
28
seconds
n/a
OpenAI o4-mini
200,000
$
1.1
n/a
$
4.4
n/a
135
n/a
t/s
35.3
seconds
n/a
GPT-4.1 nano
1,000,000
$
0.1
n/a
$
0.4
n/a
n/a
t/s
seconds
n/a
GPT-4.1 mini
1,000,000
$
0.4
n/a
$
1.6
n/a
n/a
t/s
seconds
n/a
GPT-4.1
1,000,000
$
2
n/a
$
8
n/a
n/a
t/s
seconds
n/a
Llama 4 Scout
10,000,000
$
0.11
n/a
$
0.34
n/a
2600
n/a
t/s
0.33
seconds
n/a
Llama 4 Maverick
10,000,000
$
0.2
n/a
$
0.6
n/a
126
n/a
t/s
0.45
seconds
n/a
Gemma 3 27b
128,000
$
0.07
n/a
$
0.07
n/a
59
n/a
t/s
0.72
seconds
n/a
Grok 3 [Beta]
/
$
n/a
$
n/a
n/a
t/s
seconds
n/a
Gemini 2.5 Pro
1,000,000
$
1.25
n/a
$
10
n/a
191
n/a
t/s
30
seconds
n/a
Claude 3.7 Sonnet
200,000
$
3
n/a
$
15
n/a
78
n/a
t/s
0.91
seconds
n/a
GPT-4.5
128,000
$
75
n/a
$
150
n/a
48
n/a
t/s
1.25
seconds
n/a
Claude 3.7 Sonnet [R]
200,000
$
3
n/a
$
15
n/a
78
n/a
t/s
0.95
seconds
n/a
DeepSeek-R1
128,000
$
0.55
n/a
$
2.19
n/a
24
n/a
t/s
4
seconds
n/a
OpenAI o3-mini
200,000
$
1.1
n/a
$
4.4
n/a
214
n/a
t/s
14
seconds
n/a
OpenAI o1-mini
128,000
$
3
n/a
$
12
n/a
220
n/a
t/s
11.43
seconds
n/a
Qwen2.5-VL-32B
131,000
$
n/a
$
n/a
n/a
t/s
seconds
n/a
DeepSeek V3 0324
128,000
$
0.27
n/a
$
1.1
n/a
33
n/a
t/s
4
seconds
n/a
OpenAI o1
200,000
$
15
n/a
$
60
n/a
100
n/a
t/s
30
seconds
n/a
Gemini 2.0 Flash
1,000,000
$
0.1
n/a
$
0.4
n/a
257
n/a
t/s
0.34
seconds
n/a
Llama 3.3 70b
128,000
$
0.59
n/a
$
0.7
n/a
2500
n/a
t/s
0.52
seconds
n/a
Nova Pro
300,000
$
1
n/a
$
4
n/a
128
n/a
t/s
0.64
seconds
n/a
Claude 3.5 Haiku
200,000
$
0.8
n/a
$
4
n/a
66
n/a
t/s
0.88
seconds
n/a
Llama 3.1 405b
128,000
$
3.5
n/a
$
3.5
n/a
969
n/a
t/s
0.73
seconds
n/a
GPT-4o mini
128,000
$
0.15
n/a
$
0.6
n/a
65
n/a
t/s
0.35
seconds
n/a
GPT-4o
128,000
$
2.5
n/a
$
10
n/a
143
n/a
t/s
0.51
seconds
n/a
Claude 3.5 Sonnet
200,000
$
3
n/a
$
15
n/a
78
n/a
t/s
1.22
seconds
n/a
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.