Nate Herk: Sol costs less on creative builds, while Fable wins more blind selections

Test environment

• Agent build: The same /goal prompt with complete creative freedom; Fable ran in Claude Code, Sol in Codex; the author reviewed the results blind before revealing the models.

• Three builds: A playable bicycle game, an interactive scrolling website, and five completely different visual objects.

• API runs: Fast, stateless tasks without an Agent loop, comparing response rate, capability score, speed, and cost.

Inputs/configuration

• The bicycle-game prompt called for an open-world game in the browser, with WASD steering, spacebar jumping, Q/E aerial tricks, and Shift acceleration.

• The website prompt asked for “the most impressive interactive scrolling website.”

• The third task only asked for five fundamentally different visual elements and required the model to return a gallery and five sister sites.

• These builds used different harnesses, so the comparison was of complete working configurations rather than bare models.

Results data

• The API fast-task response records were 24 for Sol and 3 for Fable; the author noted that most of the difference came from Fable refusing to answer.

• Among the answers that were actually returned, Sol's capability score was 0.98 and Fable's was 0.966; the batch cost $16 for Sol and $63 for Fable.

• The author's routing judgment was Fable for management/strategy, and Sol for execution, verification, and delivery.

Conclusions

In this personal blind test, Sol was clearly more efficient in tokens and cost, and won on the open-ended “five objects” task; Fable was more often chosen for the final aesthetics and completeness of creative builds. The result supports the “Fable sets direction, Sol executes” workflow hypothesis, but it is not an overall ranking of model capabilities.

Limitations

• Each of the three builds was run once, so subjective choices and the author's design preferences affect the results.

• The toolchains, default prompts, and context management in Codex and Claude Code differed; the cost difference cannot all be attributed to the models.

• The 24–3 API gap is confounded by refusals; 0.98 and 0.966 are not public standard benchmarks either.

• The article does not provide the complete API inputs, grader, latency distribution, or random seed.

Reproduction steps

1. In two isolated repositories, fix the same commit, identical /goal text, and identical asset permissions.

2. Randomize model run order and clear git history and caches, preventing the second model from reading the first model's artifacts.

3. For each build, record completion time, input/output tokens, tool calls, playability, and blind-review results.

4. For each API task, record “complete, refusal, error, or timeout”; do not conflate refusals with capability failures.

5. Repeat for multiple rounds and report the mean, dispersion, and harness differences.

Original evidence and data

• The article publicly disclosed the intended prompts, time, cost, and approximate output tokens for the three builds.

• The author explicitly said Sol cost about half as much in tokens as Fable, and considered Sol's price tier closer to Opus 4.8 for comparison.

Scope of applicability

• Suitable for considering cost/quality trade-offs in Agent builds, but not as a general benchmark for writing, mathematics, or coding.

• “Sol is the worker” is the author's empirical model; it could reverse under other harnesses, task boundaries, or design standards.

• For creative outputs, define the blind-review rubric in advance to avoid presenting personal aesthetics as an objective win/loss.

Source excerpt or observation (compliance short quote only)

The author's core routing metaphor was “Fable is the manager, Sol is the worker”.