The author did not collapse the models into a single overall score, instead measuring them in roles such as strategic decision-making, repository execution, and code repair. The results show that Sol high is suited to the main strategic session, while Sol medium is faster and uses fewer resources; in this small-sample execution test, Luna high/max did not beat the existing routing and even showed signs of over-reasoning.
In strategic tasks, Luna max scored 88, tying Terra max, but below Sol high at 94 and Sol max at 90.
Repository execution brief: Sol high scored 93 in 80.73 seconds with 1,818 reasoning tokens; Sol medium scored 91 in 70.66 seconds with 779 tokens.
In the small repair, Luna high went 4/4 in 52.59 seconds; Luna max went 4/4 in 121.34 seconds, which the author described as “heavily over-reasoned.”
The final routing did not give Luna a fixed slot; the author excluded Luna from the main-session, review, and implementation-rollback paths.
The author explicitly noted that the sample was small, the evaluator knew the model identities, CLI latency was mixed in, and subscription consumption was not equivalent to API token cost; the results cannot be generalized.
I ran a small role-based benchmark: three strategic decision memos, one repository-grounded execution brief, and two pla… This is a necessary excerpt; read the original source for full context.
GPT-5.6 Luna
