Hugging Face Official MiMo-V2.6-Pro-RL Local Deployment and Chat Template Configuration

One-sentence conclusion

The official model card provides SGLang and vLLM service commands for MiMo-V2.6-Pro-RL and defines chat-template behavior for text, image, video, audio, thinking, and tool calls in the repository tokenizer configuration; local deployment must use the checkpoint name and must not treat it as the same model identifier as the hosted API's mimo-v2.6-pro.

Applicable scenarios

• Suitable tasks: Self-hosted text reasoning, code and Agent services, and server-side workflows that need to process text, image, video, and audio inputs through a unified interface.

• Unsuitable tasks: Single consumer-grade GPUs or use cases that only need a low-latency hosted API; the model card does not provide a single-GPU deployment plan and does not directly map local-checkpoint parameters to API quotas or latency guarantees.

• Applicable model version: The local checkpoint is only XiaomiMiMo/MiMo-V2.6-Pro-RL. Do not rename MiMo-V2.5 or MiMo-V2.5-Pro from the V2.5 deployment references below and treat them as V2.6 evidence, and do not generalize this to MiMo-V2.6-Flash-RL or mimo-v2.6-pro-ultraspeed.

• Applicable clients, Agents, or APIs: The SGLang OpenAI-compatible service and vLLM OpenAI-compatible service provided by the model card; the chat template comes from tokenizer_config.json / chat_template.jinja in the Hugging Face repository.

• Recommended reasoning mode and parameters: The model card recommends sampling with temperature=1.0 and top_p=0.95. The SGLang example uses two nodes, TP16, and DP2; the vLLM example uses --tensor-parallel-size 8. GPU count, memory, and parallelism must be revalidated for the local environment.

Ready-to-use content

1. SGLang service configuration (command from the model card)

The model card specifies the Docker image lmsysorg/sglang:latest and provides the following two-node service command. <node-rank> and <node0-ip> are placeholders that the deployer must replace and cannot be executed as written.

docker pull lmsysorg/sglang:latest

sglang serve \
  --trust-remote-code \
  --model-path XiaomiMiMo/MiMo-V2.6-Pro-RL \
  --tp 16 \
  --dp 2 \
  --enable-dp-attention \
  --mm-enable-dp-encoder \
  --ep 16 \
  --moe-a2a-backend deepep \
  --moe-dense-tp-size 1 \
  --mem-fraction-static 0.7 \
  --max-running-requests 128 \
  --chunked-prefill-size 32768 \
  --page-size 64 \
  --swa-full-tokens-ratio 0.3 \
  --speculative-algorithm EAGLE \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4 \
  --enable-multi-layer-eagle \
  --reasoning-parser mimo \
  --tool-call-parser mimo \
  --host 0.0.0.0 \
  --port 30000 \
  --nnodes 2 \
  --node-rank <node-rank> \
  --dist-init-addr <node0-ip>:20000

2. vLLM service configuration (command from the model card)

vllm serve XiaomiMiMo/MiMo-V2.6-Pro-RL \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --gpu-memory-utilization 0.95 \
  --max-model-len auto \
  --reasoning-parser mimo \
  --tool-call-parser mimo \
  --enable-auto-tool-choice \
  --generation-config vllm

The model card only asks users to follow the vLLM MiMo-V2.5 recipe; it does not provide a V2.6-specific Docker tag. The recipe currently explicitly shows XiaomiMiMo/MiMo-V2.5 and vllm/vllm-openai:mimov25-cu129, so this V2.5 image must not be written up as a verified V2.6 dependency. Before deploying V2.6, confirm the actual vLLM version, remote code, and checkpoint support.

3. Reproducible chat-template behavior

The tokenizer_config.chat_template returned by the model card API and the repository's chat_template.jinja define the following rules. When using the Hugging Face tokenizer, retain the repository template rather than writing a simplified template that drops special tokens.

Ordinary message: <|im_start|>{role}\n{rendered_content}<|im_end|>
Assistant message: <|im_start|>assistant\n<think>{reasoning_content}</think>{content}<|im_end|>
Generation prompt: <|im_start|>assistant\n
Disable thinking (enable_thinking=false): append <think></think> after the generation prompt

The special mappings in the content section are:

image  -> <|vision_start|><|image_pad|><|vision_end|>
video  -> <|vision_start|><|video_pad|><|vision_end|>
audio  -> <|mimo_audio_start|><|audio_pad|><|mimo_audio_end|>

The template also places tool definitions inside the You are provided with the following tools: and <tools>...</tools> blocks, and serializes tool calls as <tool_call><function=function_name>...</function></tool_call>. Tool parameters can use the <parameter=parameter_name>value</parameter> form. These are behaviors of the template source, not additional task prompts published in the model card.

Test/workflow steps

1. First confirm that the runtime environment can support the model-card configuration: the model summary describes a sparse MoE with approximately 1.02T total parameters and 42B active parameters, a maximum context of 1M, and repository weights that include an FP8 configuration and many sharded files. Do not estimate memory as you would for an ordinary single-GPU Transformers model.

2. Choose SGLang or vLLM and use the checkpoint name XiaomiMiMo/MiMo-V2.6-Pro-RL directly. Before startup, check that --tp / --tensor-parallel-size, the node count, and network ports match the actual topology.

3. The SGLang multi-node configuration requires a different --node-rank on each node, and every node must be able to reach <node0-ip>:20000; the model card sets the service port to 30000.

4. When the vLLM configuration enables --reasoning-parser mimo, --tool-call-parser mimo, and --enable-auto-tool-choice, the client should also send tool definitions through the OpenAI-compatible interface and validate the returned fields; checking only for HTTP 200 is insufficient.

5. Use the repository tokenizer template to construct a minimal text request, then verify the image, video, and audio placeholder mappings and tool calls one by one. Start with a small batch before increasing max-running-requests or enabling EAGLE.

6. Use temperature=1.0 and top_p=0.95 as the starting point recommended by the model card. SGLang's EAGLE parameters are part of the model-card service command, but their actual acceptance rate and gains depend on the task distribution and should be benchmarked separately; successful startup is not an acceleration conclusion.

Original evidence and data

• Hugging Face API model metadata identifies the repository as XiaomiMiMo/MiMo-V2.6-Pro-RL, with architecture MiMoV2ForCausalLM, library_name=transformers, model type mimo_v2, and an exposed tokenizer_config.chat_template.

• The model-card summary states: sparse MoE; approximately 1.02T total parameters and 42B active parameters; a context length of 1M; text, image, video, and audio modalities; a 681M-parameter vision encoder; an audio encoder made up of a 308M AudioTokenizer and a 127M audio patch encoder; and 5 layers of an MTP speculative decoder.

• The model card's Deployment section directly provides the SGLang command with --tp 16, --dp 2, --ep 16, --enable-dp-attention, --mm-enable-dp-encoder, DeepEP, multi-layer EAGLE speculative decoding, --reasoning-parser mimo, and --tool-call-parser mimo, and sets --nnodes 2.

• The model card's vLLM section directly provides --tensor-parallel-size 8, --max-model-len auto, --reasoning-parser mimo, --tool-call-parser mimo, --enable-auto-tool-choice, and --generation-config vllm, and recommends temperature=1.0 and top_p=0.95.

• The official Cookbook page directly linked by SGLang is a MiMo-V2.5 page. The page itself distinguishes MiMo-V2.5-Pro from MiMo-V2.5 and provides V2.5 installation, inference, tool-calling, and multimodal examples. These examples illustrate the calling pattern of an OpenAI-compatible service and are not independent validation of MiMo-V2.6-Pro-RL.

• The official Recipe directly linked by vLLM is also a MiMo-V2.5 page. Its current example model is XiaomiMiMo/MiMo-V2.5, and it explicitly shows V2.5 hardware and image configuration. Accordingly, this article accepts it only as the deployment reference pointed to by the model card and does not rewrite its V2.5 figures as V2.6 figures.

Boundary between the Pro-RL checkpoint and the API mimo-v2.6-pro

Scope and limitations

• This is a compilation of official configuration information, not a local reproduction. The approximately 524 GB-scale repository weights were not downloaded in the current environment, and this article does not claim that the commands started successfully on this machine.

• The 1M context, parameter scale, parallelism, and sampling parameters in the model card are official information; they should not be used to infer throughput, latency, or cost on arbitrary hardware.

• The SGLang and vLLM documentation directly linked by the model card is currently for V2.5. V2.6 validity is determined by the V2.6 commands explicitly given in the model card and by actual runtime verification.

• trust-remote-code executes custom code in the repository. Production environments should pin the commit, image, and dependency versions and review the code in an isolated environment.

• Special tokens for multimodal content only represent template placeholders; decoding, preprocessing, and server-side support for images, videos, and audio still depend on the selected inference framework and version.

• The tool-calling template can serialize tool definitions and calls, but it does not execute tools for the application. A production Agent still needs permission controls, parameter validation, timeouts, retries, and result forwarding.

• This article does not copy the online-interface parameters, quotas, or available clients of API mimo-v2.6-pro to the local checkpoint; configure the two separately according to their respective official documentation.

Source excerpts or observations (short compliance excerpts only)

1. Hugging Face model card: The page defines MiMo-V2.6-Pro-RL as the flagship checkpoint in the MiMo-V2.6 series and provides links to “follow the SGLang MiMo cookbook” and the vLLM recipe.

2. Model-card Deployment: The public SGLang configuration contains --model-path XiaomiMiMo/MiMo-V2.6-Pro-RL, two-node parameters, EAGLE parameters, and reasoning/tool-call parsers; the vLLM configuration contains TP8, automatic tool choice, and generation-config vllm.

3. Model-card tokenizer configuration: The template source writes the corresponding vision or audio special tokens for image, video, and audio content, and uses enable_thinking to control whether an empty <think></think> is appended.

4. Official deployment references: The SGLang Cookbook and vLLM Recipe both have MiMo-V2.5 in their page titles and body text. They are deployment references directly linked by the model card, not independent evaluations of V2.6-Pro-RL.