Deliver code with prediction, planning, review, and verification

One-sentence takeaway

Breaking Sol's long-running coding tasks into five stages—prediction, planning, implementation, adversarial review, and verification—makes it possible to separately check whether the “model claims completion” and whether the work is “delivered according to plan.”

Use cases

• Suitable tasks: Multi-file coding, agent orchestration, test-driven implementation, and long-running tasks prone to endless repair loops.

• Unsuitable tasks: One-off Q&A or creative tasks with no verifiable output.

• Applicable model version: GPT‑5.6 Sol; it can also be transferred to other coding agents.

• Applicable clients, Agents, or APIs: Agents such as Codex and OpenCode that can save plans, run tests, and review code.

• Recommended reasoning tier and parameters: The comment did not specify a fixed tier; start with the lowest tier that can complete the task, and track costs separately for iterative tasks.

Ready-to-use content

The following is a reusable skeleton reconstructed from the stage names made public in the comment, not the author's complete public system prompt:

prediction_stage:
  Write down expected behavior, key risks, and the minimum acceptance signals.
planning_stage:
  Break the goal into checkable steps; define scope, interfaces that must not change, and stop conditions.
implementation_stage:
  Implement only the current plan, running targeted tests as you go.
review_stage:
  Adversarially inspect the implementation and tests, focusing on reward hacking, overengineering, and tests that go easy on the implementation.
verifier_stage:
  Compare the actual output item by item against the plan and acceptance signals; deliver if it passes, and return only to the missing stage if it fails.
loop_policy:
  If verification fails, state the specific gap and next step; once the stop conditions are met, prohibit further scope expansion.

Testing/workflow steps

1. Save the prediction and plan before starting the task to prevent goal drift later.

2. During implementation, submit only code and tests related to the current plan.

3. During review, check whether the tests truly cover the requirements and whether assertions were weakened just to “pass.”

4. During verification, run independent commands or black-box checks and record pass/fail item by item.

5. In a loop, fix only failed items; if the model repeatedly changes the same area, pause and ask a human to confirm the scope.

Original evidence and data

• The sequence publicly mentioned in the comment was prediction_stage, planning_stage, review_stage, and verifier_stage, with a recommendation to check for reward hacking after the code and tests are in place.

• The commenter explicitly said this was a hack from their own testing, not a controlled benchmark or an official Codex configuration.

• Experiences of Sol Ultra/Max in the main post of the same thread are strongly disputed; some users in the comments reported efficiency, while others reported overengineering and exhausting their quotas.

Applicability boundaries

• This is a community workflow suggestion, not Sol's default behavior or an official OpenAI best practice.

• The stage names and examples need to be mapped to the actual Agent's hooks, files, or commands; the source does not provide a complete configuration file that can be copied.

• The verification stage cannot replace human review of high-risk changes, especially database, permission, and production deployment operations.

Source excerpt or observation (for a compliant short quotation only)

The comment recommends using review_stage after implementation and testing are complete to check for “signs reward hacking in tests”.