Vellum's compilation of the same set of public benchmarks shows GPT-5.5 performing strongly on Terminal, GDPval, OSWorld, long-context tasks, and CyberGym, but Opus 4.7 leads on SWE Pro, MCP Atlas, and some HLE tasks; no single leaderboard can substitute for matching a model to the task.
Suitable tasks: Teams that need to put GPT-5.5, Claude, and Gemini on the same candidate list for initial screening.
Unsuitable tasks: Treating Vellum's summary table as an independent replication or using it directly to make safety-critical deployment decisions.
Applicable model versions: GPT-5.5 and the comparison versions available at the time discussed in the article; verify the specific snapshots against the original article and official model pages.
Applicable clients, Agents, or APIs: The article covers public results from API/Agent benchmarks; it does not provide a single, directly runnable Vellum harness.
Recommended reasoning tier and parameters: The article does not disclose a consistent set of parameters; for reproduction, hold the same model snapshot and harness constant.
Data source: Vellum's compilation of GPT-5.5 release data and public benchmarks for other models.
Models/tools: Different benchmarks use different tools and Agent harnesses; the article explicitly warns that vendor-reported results, harness differences, and model versions affect comparisons.
Scoring method: The scores reported by each benchmark; raw inputs, random seeds, and complete trajectories for each task are not provided.
No consistent inputs, system prompts, tool versions, temperature, reasoning tier, or number of runs are disclosed. Therefore, this content can only serve as a source-traceable secondary comparison table and cannot be labeled an independent controlled test.
| Test | GPT-5.5 | Competitive results/observations listed in the article |
|---|---|---|
| Terminal-Bench 2.0 | 82.7 | GPT-5.5 leads in this table; note the Terminus/harness version |
| SWE Pro | 58.6 | Claude Opus 4.7 at 64.3; the article says it leads this test |
| SWE Verified | 88.6 | Listed in the article as the GPT-5.5 result; inputs and Agent configuration are not disclosed |
| OSWorld | 78.7 | GPT-5.5 is higher than most comparisons listed in the article, but the result depends on a computer-use harness |
| GDPval | 84.9 | GPT-5.5 leads in the table |
| MRCR 512K–1M | 74.0 | A long-context retrieval task affected by context truncation and retrieval settings |
| ARC-AGI-2 | 85.0 | GPT-5.5 result listed in the article |
| CyberGym | 81.8 | Recorded only as a publicly reported safety-evaluation score; harmful content is not reproduced |
| MCP Atlas | 75.3 | Claude Opus 4.7 at 79.1; the article lists Opus as leading |
| HLE (no tools) | 41.4 | Claude Opus 4.7 at 46.9; the article lists Opus as leading |
Vellum's comparison has two practical values: first, it highlights GPT-5.5's strengths in terminal use, computer use, and knowledge work; second, it shows that GPT-5.5 does not lead every coding or tool benchmark. Model selection should divide real tasks into buckets such as terminal coding, code repair, MCP tools, browsing/computer operation, and long-document retrieval, then compare success rate, human rework, and cost for each.
Limitations: The results are primarily a compilation of scores publicly reported by vendors or benchmark providers, not a unified experiment rerun by Vellum; the article does not provide complete prompts, tool trajectories, or per-question data.
Reproduction steps: First select public benchmarks according to the task buckets in the article; hold the model snapshot, tool version, system instructions, reasoning effort, time limit, and scoring rules constant; run at least three rounds and save the raw trajectories; report the results separately from the figures in the article.
Safety boundary: For safety projects such as CyberGym, record compliant statistics only; do not reuse evaluation questions or exploitation steps as prompts.
The article's table lists GPT-5.5, GPT-5.4, Claude Opus 4.7, Gemini 3.1, and others; the body explicitly notes GPT-5.5's strong performance on Terminal, GDPval, OSWorld, ARC-AGI-2, and other tasks, while also listing Opus 4.7's advantages on SWE Pro, MCP Atlas, HLE without tools, and other tasks. This article transcribes the key figures from the comparison table and retains the qualifications about “vendor-reported” results and different harnesses.
The article's claim of “leading” cannot be interpreted as statistical significance; the number of repetitions and confidence intervals are not public.
Scores from different projects cannot be added together, nor can they be used to infer the completion rate for a specific codebase.
Prices, model versions, context, and safety policies may change; before deployment, return to the official OpenAI pages for verification.
The article reminds readers that “vendor-reported” results and harness selection can change the comparison; this is precisely the boundary that must be fixed in reproduction and reported separately.
GPT-5.5
