OpenAI outlines a process for monitoring cache hits, diagnosing cache misses, keeping prompt prefixes and tool definitions stable, adjusting reasoning effort, and prewarming the cache.
Suitable tasks: Multi-turn GPT-6 API agents that reuse long instructions or tool definitions and need to reduce the latency and cost of processing repeated input.
Not suitable for: Requests where the input changes completely each time or the application cannot maintain a consistent context prefix.
Applicable model versions: The GPT-6 family, including GPT-6 Luna.
Applicable clients, agents, or APIs: Multi-turn agents using the OpenAI API. Implementation should follow the official Prompt Caching and Responses API documentation.
Recommended reasoning effort and parameters: Adjust to the task. OpenAI says reasoning effort can be changed between requests without breaking the cache; it does not specify a fixed effort level.
Monitor the hit rate: Use the Prompt Caching Dashboard to review the hit rate and the composition of cached and uncached tokens.
Diagnose misses: Compare the current request with a recent response to identify changes to the model, tools, settings, or input that prevented reuse.
Choose a cache prefix: Use explicit cache breakpoints to select reusable prompt prefixes; put stable shared context at the beginning.
Keep tool definitions stable: Keep tool definitions, schemas, and ordering consistent where possible. Use allowed_tools to limit which tools can be called, or tool_choice: "none" to disable tools, rather than frequently removing tool definitions.
Append new instructions: When tools or instructions change, append overriding rules in a later developer message to preserve the shared prefix above it.
Adjust reasoning for the task: Append configuration_update before a later response to change reasoning effort without rewriting the original prompt prefix.
Prewarm in advance: Prepare known shared instructions, tool definitions, or reference material at application startup or before the user submits a request.
Measure the result again: Review the hit rate, latency, and cost to confirm that the optimization works for the actual workload.
OpenAI says GPT-6 offers cache discounts for eligible shared prefixes reused within a 30-minute window; cache hits depend on request prefixes and other conditions.
The article describes product features and a workflow, but does not guarantee a quantitative benefit for every application. Confirm cache hits and savings using the application's own dashboard and diagnostic results.
Implement the exact request structures for configuration_update, explicit cache breakpoints, and prewarming according to the relevant API documentation. This record does not invent complete API payloads that the article does not show.
GPT-6 Luna
