SGLang Official LongCat-2.0-FP8 Multi-GPU Deployment Configuration and Verification Workflow

One-sentence takeaway

This Cookbook provides a reusable path for launching, tuning, and verifying LongCat-2.0-FP8 on B300, B200, H200, and H20 with SGLang. It is best to deploy first on a validated topology and then gradually enable experimental parallelism options.

Use cases

• Suitable tasks: Self-host LongCat-2.0-FP8; multi-GPU/multi-node inference; services requiring an OpenAI-compatible API, long context, and a controllable thinking toggle

• Unsuitable tasks: A single consumer GPU; quick trials without a multi-GPU cluster or without the capacity to store FP8 weights

• Applicable model version: meituan-longcat/LongCat-2.0-FP8

• Applicable client, agent, or API: SGLang server, OpenAI SDK, cURL; after deployment, use the local /v1/chat/completions

• Recommended reasoning level and parameters: Start with the Balanced configuration to validate the matrix; retain a BF16 KV cache and a static memory fraction of 0.92 on the server, and disable thinking for the single verification request to reduce variables

Ready-to-use content

1. Install the SGLang nightly build

The Cookbook says that LongCat support is on the SGLang main branch; use a nightly wheel or rolling nightly Docker image until it is included in a stable release.

pip install --upgrade pip
pip install uv

# B300 / CUDA 13
SGLANG_WHL_INDEX=https://docs.sglang.ai/whl/cu130
# For CUDA 12.9, use instead:
# SGLANG_WHL_INDEX=https://docs.sglang.ai/whl/cu129

uv pip install --prerelease=allow \
  --extra-index-url "\${SGLANG_WHL_INDEX}" \
  "sglang[all]"

2. Select a validated topology first

Before deployment, fill in environment values such as HOST_IP, PORT, NODE0_IP, and NODE_RANK in the Cookbook's configuration panel; do not use the example addresses directly in production.

3. Key parameters for LongCat's prefill path

Use the complete command produced by the generator as the main command, and only confirm that the following LongCat-related items are present:

--trust-remote-code
--nsa-prefill-backend fa3
--chunked-prefill-size 2048
--kv-cache-dtype bfloat16
--mem-fraction-static 0.92
--model-loader-extra-config '{"enable_multithread_load":true,"num_threads":12}'

The Cookbook explicitly recommends not manually adding fp8-gemm-runner-backend; let SGLang automatically select the backend according to LongCat's FP8 scale layout. Long-context, multi-node, and PD-disaggregation switches should be selected through the page generator according to the hardware matrix, rather than manually combining unvalidated options.

4. Minimal health check

After the service starts, first send a short request with thinking disabled to confirm that the port, model name, and chat template all work:

curl http://localhost:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meituan-longcat/LongCat-2.0-FP8",
    "messages": [
      {"role": "user", "content": "A shop has 17 apples and sells 8. Then it buys 6 more. Answer with only the final number."}
    ],
    "max_tokens": 32,
    "chat_template_kwargs": {"enable_thinking": false}
  }'

The expected final answer is 15. The equivalent OpenAI SDK configuration points base_url to http://localhost:30000/v1, uses any non-empty placeholder value for api_key, and passes chat_template_kwargs.enable_thinking=false in extra_body.

Test/workflow steps

1. Download the LongCat-2.0-FP8 weights and verify the cluster GPUs, CUDA, and SGLang nightly version.

2. Select the Cookbook's Balanced recipe for either 8 B300 GPUs in one node or 16 H200/B200/H20 GPUs across two nodes.

3. Fill in node IPs, ports, and ranks; initially keep only the validated command generated by the page.

4. Send the 17-8+6 health check with thinking disabled; then gradually load-test thinking, concurrency, and long context.

5. Record startup time, memory usage, throughput, KV-cache hits, and error logs; mark every new parallel combination separately as an experimental configuration.

Raw evidence and data

• Cookbook hardware matrix: one node with 8 B300 GPUs uses TP=8/EP=8; H200, B200, and H20 use 16 GPUs across two nodes with TP=16/EP=16.

• Cookbook default recommendations: FA3 prefill, chunked prefill 2048, BF16 KV cache, a static memory fraction of 0.92, and 12 threads for multi-threaded loading.

• Cookbook B300 validation: CUDA graph capture succeeded on 8x B300; GSM8K accuracy was 98.0% on 200 questions and 95.8904109589041% on 1,314 questions.

• The page positions the model as a sparse MoE serving recipe with FP8 weights, LongCat Sparse Attention, and n-gram embedding; it also provides OpenAI-compatible cURL/Python requests.

Applicability boundaries

• This source is the deployment provider's official Cookbook, not an independent cross-hardware performance evaluation; the GSM8K figures use the SGLang/recipe validation methodology and cannot establish a general capability ranking.

• The B300 recipe is explicitly validated end to end, while the other hardware is presented as a two-node matrix on the page; real-world stability still depends on drivers, networking, memory, and the nightly version.

• A 1M training context does not mean every deployment can reliably handle 1M-token requests; first run stress tests at the target context length.

• SGLang's required nightly version and command parameters may change; record the documentation date and the commit/image tag when reproducing.

Source excerpts or observations (compliance short quotations only)

• The page says it provides “config-driven recipes” and separates validated recipes from Playground experimental options.

• The page explicitly says not to pass an FP8 GEMM runner backend manually, but to let SGLang select it automatically.