Jonathan Fulton: Sol builds more complete apps, but the long migration is slower

Summary

The author reused the prompts, methodology, and 100,000-line code-porting task from the Fable 5 evaluation to document Sol’s performance on a financial-planning app, browser testing, user experience, and domain decisions.

Original article

The following is the visible body text extracted during this visit. It includes page navigation, machine translation, advertising, comments, and other page elements; verify against the original link before citing it.

Sidebar menu Write Notifications Home Library Profile Stories Stats Following Jonathan’s Musings

An eclectic set of musings from Jonathan Fulton

Technology Artificial Intelligence Software Development Software Engineering A Short Review of GPT 5.6 Sol Nearly as capable as Claude Fable 5, sometimes better — and actually available Jonathan Fulton 4 min read · Jul 16, 2026

--

1

--

Press enter or click to view image in full size

Last month I published a short review of Anthropic’s Fable 5 shortly before access to the model was cut off for a couple weeks. Now OpenAI has shipped GPT 5.6 Sol, its entry in the same frontier tier, and I wanted to run it through the exact same gauntlet: the same benchmark prompts, the same methodology, and the same 100k-line porting task as the final head-to-head.

One difference in circumstances worth noting up front: unlike Fable, I do have access to Sol at work. But to keep the comparison apples-to-apples, all of the benchmarks below were run on my personal OpenAI subscription.

The Lyfetime.io Test

Quick recap for new readers: my standard benchmark is a spec prompt describing my Lyfetime.io financial planning app — 20 forms and fairly complicated financial simulation math. Historically, models botched it badly. Fable 5 was the first to one-shot the whole thing, in 30 minutes.

Sol one-shotted it too, in about an hour. What impressed me was the process: it ran actual browser tests along the way to verify that everything worked as it built. When it finished, I tested everything myself, and overall it did a great job.

Here’s the interesting part: the UX was better than what Fable produced. But Sol made some suboptimal domain decisions along the way — unrealistic tax brackets, and home values that don’t appreciate with inflation. The UI had subtle flaws too, like using a free-text field for birth year instead of a dropdown.

So: better polish, slightly worse judgment on the details.

The Linear Clone Test

Next up, my Linear.app clone test. My methodology here is two parts: first I ask the model to create a plan and save it to a file, then I use /goal to implement the plan.

Sol false-started. Its initial plan was to clone only the marketing site. I started over and was more explicit in my instructions, and the second attempt was on target.

I didn’t read the plan — I just let it run. And run it did: 24 hours of implementation, and the result was far more feature-complete than the Fable clone (which, to be fair, only took 30 minutes).

But there were UX bugs. I iterated a few times, essentially prompting “Fix all the UX bugs. Make it super polished.” Each pass improved things, but issues remained, and eventually I decided it wasn’t worth more of my time. Thorough, but not quite polished.

The Notion Clone I Didn’t Run

In my Fable review, the Notion clone was part of the suite. This time I skipped it. Between implementation and polish passes, the Linear clone consumed 24+ hours of wall-clock time, and I didn’t want a repeat. Consider this a data point in itself: Sol’s thoroughness comes at the cost of patience.

Sol at Work

I also used Sol for a few genuinely demanding tasks at Datadog, and it seemed to do great. Unfortunately I can’t offer a head-to-head here — we don’t have Fable access at work, so there’s nothing to compare against. Take it simply as: Sol holds up on real production work, not just benchmarks.

The Real Test: sqlglot to Golang, Round Two

The centerpiece of my Fable review was converting sqlglot — a ~100k line Python SQL parser — to Golang. Fable did it in 5.5 hours, spinning up as many as 9 sub-agents in parallel, with 99.5% of unit tests passing.

For Sol, I used Ultra mode so it would use sub-agents and parallelize. The task is well suited to it — sqlglot supports 30+ SQL dialects, which decompose naturally into parallel workstreams. That said, Codex with Sol on Ultra mode tops out at (I believe) 4 sub-agents, versus Fable’s 9.

The timeline: I blew through my weekly subscription limit at about the 22-hour mark. I had one usage reset available, used it, and the conversion finished at around hour 26.

The results: 100k+ lines of Go. 20k+ tests passing — at a 100% pass rate. Roughly 20M tokens consumed.

So Sol actually edged out Fable on correctness (100% vs. 99.5% of tests passing) while taking nearly 5x as long in wall-clock time.

The Economics

This is where the comparison gets interesting. Fable’s 5.5-hour run cost me real money — I blew through Anthropic’s 5-hour usage limits and the experiment ran me about $1,000. Sol’s 26-hour run fit almost entirely within my existing OpenAI subscription, costing me one weekly limit and a usage reset.

Faster and expensive, or slower and “reasonably” priced. For a task running unattended overnight, I know which one my wallet prefers.

Conclusion

Sol is nearly as good as Fable, and in some respects better: nicer UX out of the box, more feature-complete builds, and a perfect test pass rate on the sqlglot port. Fable remains dramatically faster on long-horizon, parallelizable work — the kind of sustained systematic execution that made it feel like a step function.

Honestly, it’s hard for me to choose between them. But OpenAI and Anthropic are doing the choosing for me: GPT 5.6 Sol is accessible through the OpenAI subscription I already have, while Fable is disappearing from Anthropic’s subscription soon. And I have Sol at work.

Sometimes the best model is the one you can actually use.

Technology Artificial Intelligence Software Development Software Engineering

--

1

--

Published in Jonathan’s Musings 73 followers · Last published Jul 27, 2026

An eclectic set of musings from Jonathan Fulton

Written by Jonathan Fulton 11.2K followers · 28 following

Staff Software Engineer at Datadog. Formerly Eppo, SVP Product & Engineering at Storyblocks, McKinsey consultant. Catholic, husband, father of three.

Help

Status

About

Careers

Press

Blog

Store

Privacy

Rules

Terms

Text to speech

Repost to your network. A new and easy way to share your recommended stories with your followers.

Okay, got it