Kimi K3.1 Pro dynamic tool-loading workflow: inject tools on demand while preserving the cache prefix

One-sentence takeaway

When a conversation needs them, Kimi K3.1 Pro can inject complete tool definitions by appending a message with role="system" and tools; retaining loaded definitions exactly as they are reduces tool-definition tokens and preserves the existing prefix cache.

Use cases

• Suitable tasks: API workflows with many tools that need to search for tools by keyword and then execute multi-step Agent calls.

• Unsuitable tasks: Requests with a fixed, small tool set, or requests that need to treat a tool-name reference as a complete definition.

• Applicable model version: kimi-k3.1-pro; the documentation also explicitly supports kimi-k3.

• Applicable client, agent, or API: Kimi API Chat Completions, and clients or backends that can append complete tool definitions by message.

• Recommended reasoning level and parameters: The documentation does not specify a separate reasoning level for dynamic loading; put fixed tools in the top-level tools and on-demand tools in a message's tools field.

Ready-to-use content

The minimal structure from the official example is retained below. A dynamic-tool message cannot also contain content:

{
  "model": "kimi-k3.1-pro",
  "messages": [
    {"role": "user", "content": "Help me compute 23 * 47."},
    {
      "role": "system",
      "tools": [{
        "type": "function",
        "function": {
          "name": "Calculator",
          "description": "A calculator that evaluates a single arithmetic expression",
          "parameters": {
            "type": "object",
            "properties": {
              "expr": {
                "type": "string",
                "description": "An arithmetic expression in JavaScript syntax"
              }
            },
            "required": ["expr"]
          }
        }
      }]
    }
  ]
}

Testing or workflow steps

1. Put the core tools needed in every round in the top-level tools, and keep their definitions unchanged.

2. In the first round, send only the core tools and the current conversation; avoid declaring the entire tool catalog in advance.

3. Implement search_tools in the application, returning the names and summaries of matching tools; the documentation provides no dedicated tool-search API.

4. Based on the search results, append a role="system" message to the end of messages, with the complete definitions in its tools field.

5. The tools become visible to the model starting at the position where that message appears in messages; top-level and dynamic tools are visible at the same time.

6. Continue generation so the model can call the newly loaded tools in subsequent generation; the application's own tool system determines actual execution, permissions, and result handling.

7. Continue carrying the loaded tool declarations in the next request, keeping their contents unchanged; after a declaration is omitted, that tool no longer takes effect unless it is declared elsewhere again.

8. When new tools need to be loaded, append them to the end of the messages; do not modify the intervening history or declarations already injected.

9. Check that the dynamic-tool message contains no content, and confirm that the tool object includes complete fields such as name, description, and parameters.

10. Observe input tokens, cache reads and writes, and call results separately to confirm the benefit of tool loading and the outcome of business execution.

Original evidence and data

• Tool definitions are written in the tools field of a role="system" message; the format is the same as the request's top-level tools.

• A dynamic declaration takes effect from the position of its message and has the status of an ordinary input message; global and dynamic tools can coexist.

• A dynamic declaration must be a complete definition: it cannot contain only a tool name or refer to a tool already declared at the top level.

• Declaring many tools in advance creates Tool Definition Bloat: descriptions and parameter schemas add tokens to every request, while too many candidate tools also make it easier to choose the wrong tool or generate invalid parameters.

• A dynamic declaration takes effect for the request, and the server does not retain it automatically; the official recommendation is to carry loaded declarations back unchanged in subsequent requests to preserve tool availability and a stable prefix.

• Every system message carrying tools consumes context length, so load only the tools the current conversation actually needs.

• Appending a declaration does not change the existing prefix; modifying or inserting a message in the middle invalidates the cache after the point of change. The cache is stored in blocks; content that does not fill a complete block cannot be written to the cache and is billed at the cache-miss rate.

• Dynamic tools currently support only kimi-k3.1-pro and kimi-k3; other models (such as kimi-k2.6) fail with tokenization failed.

• If a system message with tools also contains content, the request returns 400; the OpenAI SDK can pass tools directly in messages without extra_body.

• This note did not use an API Key to make a local or online Kimi request; it does not present static-page verification as measured tool-call success, latency, or cost.

Scope and limitations

• The page's explicit basis for K3.1 Pro is the official curl example and support list for the model ID kimi-k3.1-pro; dynamic-loading capability cannot be extrapolated to models not listed there.

• “Retaining history” means that the client continues carrying loaded declarations in subsequent requests; the documentation does not say that the server automatically saves dynamic tool definitions.

• “Client execution” covers only the workflow boundary required for the application to insert declarations and handle subsequent calls; the documentation does not specify permission checks, parameter validation, timeouts, retries, concurrency, or tool-result formats.

• Global tool definitions should remain stable, while on-demand tools are suitable for dynamic injection. The documentation says that top-level global tools do not affect cache hits, but gives no fixed token-saving ratio.

• Dynamic loading reduces definition overhead, but does not make the tool call itself free; system messages still count toward context, and cache hits remain affected by blocks and prefix stability.

Reproduction notes

1. Open the official Dynamically Loaded Tools page and confirm its title and supported-model description.

2. Using an available Kimi API Key, first declare a small top-level tool, then append a system message with complete tools to the end of messages in a subsequent request.

3. Test adding, deleting, and modifying dynamic declarations separately; check the tools' visible positions, their availability for subsequent requests, and cache usage; do not judge success only by HTTP 200.

4. Use kimi-k3.1-pro as the model ID; do not use the kimi-k2.6 mentioned on the page as an equivalent regression model.

5. This note records only the workflow and limitations published on the official page; no local API test was performed.