The author reports that Gemini 4 Argon can turn residual bounds checks in Go source and RISC-V assembly into Z3 constraints, finding crashes in a set of deliberately seeded and real-code cases that go test can verify. This is a vendor-authored first-hand case study with relatively detailed methodology, not an independent benchmark, and it does not establish general reliability on other Go programs or model tasks.
Tasks this is suited to assess: Targeted crash finding along residual slice/index bounds paths in pure functions and parsers after the compiler has removed checks it can prove safe; evaluating a workflow in which a model generates constraints, a solver supplies inputs, and real tests verify them.
Tasks this should not be generalized to: It cannot be used to estimate general code correctness, crash-free rates for complete programs, formal proof capability, concurrency safety, or relative performance against other models or benchmarks.
Model version tested: Gemini 4 Argon; the article does not provide an API model ID, model snapshot, or reasoning level.
Test environment or client: Go 1.27.3; Linux/RISC-V 64-bit cross-compilation; Z3 constraint solving; go test for real validation of candidate inputs; pure functions and slice helper functions from skill-lint. The author does not disclose the calling client, complete harness, hardware, or model interface parameters.
Reasoning level and parameters: Not specified.
The Go compiler's SSA prove pass first removes bounds checks it can prove safe. The author compiles the target functions with GOOS=linux GOARCH=riscv64 and -gcflags="-S -d=ssa/check_bce/debug=1", then provides Argon with the Go source and disassembly. The model generates Z3 constraints for each remaining runtime bounds trap; Z3 checks whether the constraints are satisfiable, and any candidate input is then run against the real function with go test. The author emphasizes that a sat candidate counts as a reproduction only if the real test triggers a crash; unsat is not a proof of correctness.
The experiment has two parts:
Small parsers: First check 3 safe parsers, then 3 parsers with deliberately seeded crashes. The solver is restricted to new slices with lengths of no more than 64 bytes and cap(buf) == len(buf).
skill-lint: Select 5 pure functions or slice helper functions from a command-line tool, avoiding file I/O, command-line handling, and network operations in the solver. Two scanners return unsat; the other 3 functions reveal defects. The author also reruns these targets after removing comments to check whether the model relies on comments in the test files.
The fuzzing comparison uses unseeded go test -fuzz on Go 1.27.3. Each seeded defect is run once for 3 seconds and three times for 60 seconds, with a fresh fuzz cache each time. The author also performs a 10-second input-distribution probe. All test targets are local functions; they do not represent coverage of a complete CLI or binary.
In the small cases, Argon and Z3 find all 3 seeded crashes. For the 3 safe parsers, the compiler removes all bounds checks from the TOML key parser, while the remaining traps in the other two targets return unsat.
The first two seeded issues are related to differences between slice length and capacity: default fuzz inputs often have extra capacity, so a slicing operation that goes past len may still stay within cap and therefore not panic. Under the original fuzz settings, neither issue was found in one 3-second run or any of the three 60-second runs.
The 10-second probe records 386,741 fuzz calls. The author reports that 89% of inputs are views into a shared ~100 MB buffer, and only 0.07% satisfy cap == len. After trimming fuzz input capacity to its length, the two bounds issues trigger at median times of 0.28 seconds and 0.17 seconds, respectively, across three runs each. The third, negative-index control case is found in every run both before and after trimming.
In the skill-lint cases, 3 of the 5 target functions reveal defects, while 2 scanners return unsat. The author says Argon and Z3 still find those 3 defects after comments are removed, and return unsat for 3 patched versions. After extracting the helper functions, 3-second fuzzing also finds all 3 defects.
For timing, the author reports that Argon takes about 1 minute per function to translate assembly into constraints, while Z3 itself takes milliseconds. Complete per-target latency runs, model-call costs, and retry records are not published.
| Item | Reported in the article | Interpretive limits |
|---|---|---|
| Go version | 1.27.3 | Used for the fuzz comparison; targets are cross-compiled for Linux/RISC-V 64-bit |
| Small safe parsers | 3 | The compiler removes all bounds checks from 1; Z3 returns unsat for the remaining traps in the other 2 |
| Small parsers with seeded defects | 3 | Argon/Z3 find all 3; no independent reproducer or repeated solve count is provided |
| skill-lint target functions | 5 | 2 scanners return unsat, and 3 reveal defects; the author also says all 3 patched versions return unsat |
| Manual fuzz distribution probe | 386,741 calls / 10 seconds | 89% are views into a shared buffer; 0.07% of inputs have cap == len |
| Default fuzz comparison | One 3-second run and three 60-second runs per seeded issue; a new cache each time | Both capacity-related issues are missed; the negative-index control issue is found every time |
| Fuzzing after capacity trimming | Median times of 0.28 seconds and 0.17 seconds | Correspond to the two bounds issues missed earlier; the article says each was reproduced in all three runs |
| Solver workflow time | Argon: about 1 minute/function; Z3: milliseconds | Approximate figures from the article, not systematic performance measurements |
The author uses cap(buf) == len(buf) as a bounds-check testing condition. It makes slicing operations that exceed len panic, but this is a specific test setup; in production code, a slice's capacity can exceed its length. The article also notes that the solver targets only selected functions and does not cover data races, deadlocks, or other concurrency issues. The constraint model is written by an LLM and lacks a complete machine-semantics model, so an unsat result cannot serve as a proof.
The article presents a set of targeted cases verified with real Go tests. Source code, disassembly, Z3, and go test each serve different roles in the process; the result is not “the model proves the code is correct.”
The comparison shows that the capacity distribution of fuzz inputs can mask certain out-of-bounds panics. After capacity is trimmed to length, the two related issues in this article reproduce quickly. This finding applies to the Go fuzz harness and target functions used in the article.
The sample is small: 3 safe and 3 seeded-defect small parsers, plus only 5 selected skill-lint functions. The article provides no preregistered test set, blind test, independent replication, comparison model, or complete per-target data. It does not support a claim that Argon outperforms other models or fuzzing in general.
The author says 3 patched versions were checked, but unsat means only that the current constraint model found no input; it does not mean the vulnerability is absent. If a helper function's boundary conditions differ from those of its callers, candidate inputs may also fail to reflect real call patterns.
This is a first-hand usage report by a Google developer relations leader. Its specific workflow can help others reproduce and assess the approach, but the author selected the samples, performed the measurements, and wrote the report; it is not broad independent validation.
The public article gives the Go version, cross-compilation target, compiler flags, capacity condition, solving and real-test verification steps, and some runtime and fuzz statistics. However, it does not provide the complete repository version, all source/assembly/constraint scripts, model interface and snapshot, prompts, hardware, or all per-run records. The method can be implemented from this information, but the exact reported results cannot be replayed from the article alone. For a replication, fix the Go version and commit, target functions and entry preconditions, compiler output, model snapshot and inference parameters, Z3 version, random seeds, and fuzz harness, and record model generation time, solve time, and real reproduction results separately.
Gemini 4 Argon