Compare
to
Overview
GPT-4o | Claude 3 Opus | |
|---|---|---|
Provider Organization responsible for this model. | OpenAI | Anthropic |
Input Context Window The total number of tokens that the input context window can accommodate. | 128K tokens | 200K tokens |
Maximum Output Tokens The maximum number of tokens this model can produce in one operation. | Not specified. | 4.1K tokens |
Release Date The initial release date of the model. | May 19th, 2024 about 2 months ago | March 10th, 2024 4 months ago |
Knowledge Cutoff The latest date for which the information provided is considered reliable and current. | October 2023 | August 2023 |
Pricing
GPT-4o | Claude 3 Opus | |
|---|---|---|
Input Costs associated with the data input to the model. | $5.00 per million tokens | $15.00 per million tokens |
Output Costs associated with the tokens produced by the model. | $15.00 per million tokens | $75.00 per million tokens |
Benchmark
GPT-4o | Claude 3 Opus | |
|---|---|---|
MMLU Assesses LLMs' ability to acquire knowledge in zero-shot and few-shot scenarios. | 88.7 | 88.2 |
MMMU Comprehensive benchmark covering multiple disciplines and modalities. | Not specified. | 59.4 |
HellaSwag A demanding benchmark for sentence completion tasks. | Not specified. | 95.4 |
Arena Elo Ranking metric for LMSYS Chatbot Arena. | Not specified. | 1251 |
Are you building an AI product?
Lunary is an open-source platform to monitor, manage and improve your AI app.
Open Source
Self Hostable
Evaluations
Alerts
Public API
Exports
Prompt Templates
Chat Replays
Agent Tracing
Metrics
Feedback Tracking
LangChain Support
Open Source
Self Hostable
Evaluations
Alerts
Public API
Exports
Prompt Templates
Chat Replays
Agent Tracing
Metrics
Feedback Tracking
LangChain Support
