Wade Foster says GPT-6.2 Sol scored 41.2% on Zapier’s AutomationBench at High effort, at a cost of $0.21 per task. The post does not disclose the test size or full methodology, so the result cannot be independently reproduced from the information provided.
Tasks this benchmark can help assess: Task completion quality and cost per task for the automation workflows described by Zapier.
Tasks this result should not be generalized to: Other types of tasks, other test environments, or performance at different reasoning levels or under different pricing conventions.
Applicable model version: GPT-6.2 Sol. The original post says the model was released that day. This entry records only this version and does not represent other Sol versions.
Test environment or client: Zapier’s in-house AutomationBench. The runtime environment, tool configuration, and benchmark version were not disclosed.
Reasoning level and parameters: High effort. Other parameters were not disclosed.
Wade Foster says Zapier built AutomationBench in-house and uses its workflow tasks to measure the cost of task completion by models. The original post does not provide the number of tasks, scoring criteria, run configuration, or a cost calculation breakdown.
GPT-6.2 Sol: 41.2%, High effort, $0.21 per task.
The same post mentions GPT-6 Astra: 41.4%, Max effort, $1.73 per task. This was a September result for a different model at a different reasoning level. It can serve only as a cost and score reference cited in the post, not as a comparison at the same reasoning level.
| Model | AutomationBench score | Reasoning level | Cost per task | Note in original post |
|---|---|---|---|---|
| GPT-6.2 Sol | 41.2% | High | $0.21 | Result covered in this article |
| GPT-6 Astra | 41.4% | Max | $1.73 | Described in the post as a September record |
The post specifies the model version, reasoning level, score, and cost per task, and can be treated as Zapier’s benchmark report for GPT-6.2 Sol. The 41.2% score applies only to the High effort run described in the post. Since the post does not provide the benchmark version, task count, variance, or raw results, the score’s stability cannot be assessed, and it should not be generalized to overall task capability or used to rank models across benchmarks. The post also includes a financial workflow example, but provides no inputs and outputs that can be independently checked, so it cannot be used to verify the AutomationBench score.
The original post does not provide the task set, scoring rules, detailed run parameters, or results file, so it cannot be reproduced from the post alone. To verify the result, obtain Zapier’s documentation on the AutomationBench methodology and confirm the benchmark version, task count, execution configuration, and cost-per-task convention.
GPT-6.2 Sol