LongCat-2.0 Official Technical Blog: Architecture, Training on Domestic Compute, and Inference Deployment (Release Notes)

One-sentence takeaway

The official technical blog provides the complete technical foundation for LongCat-2.0 (LSA sparse attention, N-gram Embedding, 6D parallel training on domestic compute, and prefill-decode disaggregated deployment), making it useful for assessing the model's intended long-context and Agent capabilities, as well as reproducing the official benchmarks and deployment path.

Official core content

Release claims (consistent with the model card)

• A 1.6T-total-parameter MoE with approximately 48B parameters activated per token; all full training and large-scale deployment use domestic compute clusters.

• Pretraining: more than 50,000 domestic compute chips, taking more than a month, on 35T+ tokens, with no rollback throughout and no unrecoverable loss spikes.

• Long-range capabilities: LongCat sparse attention + training on hundreds of billions of tokens of million-token-context data + dedicated post-training → strong performance on coding and agent tasks.

• Deeply adapted to Claude Code, OpenClaw, and Hermes; online experience at https://longcat.chat; API access at https://longcat.chat/platform/docs/.

Architecture upgrades (key points from the official text)

1. LongCat Sparse Attention (LSA) — evolved from DeepSeek Sparse Attention (DSA), targeting the bottlenecks in DSA's Lightning Indexer: "discontinuous index outputs + quadratic index scoring":

   • Streaming-aware Indexing (SI): hardware-aligned contiguous access + dynamic random selection; turns fragmented GPU-memory access into sequential reads and merges HBM accesses.

   • Cross-Layer Indexing (CLI): adjacent attention layers have substantially consistent distributions of salient tokens, so one index can be reused across multiple consecutive layers (supported by cross-layer distillation during training).

   • Hierarchical Indexing (HI): two-stage coarse-to-fine scoring (block-level coarse retrieval → fine selection within candidates), enabled on demand.

   • The three components are orthogonal and can be toggled independently; all are extended to 3-step MTP speculative decoding (the target model shares one index every 2 layers; the 3 draft steps share one index).

2. N-gram Embedding: inherited from LongCat-Flash-Lite; n-gram size=5, 135B parameters; expands the embedding space by more than 100x. Expansion principles: MoE sparsity has passed its sweet spot (~97%), while the N-gram share is constrained to the optimal range (<10% of total parameters; experiments show the advantage disappears above 50%).

Training (domestic compute infrastructure)

• 6D parallelism: EMBP parallelism for N-gram Embedding is added alongside TP/CP/EP/DP/PP.

• Supernodes: physical supernodes contain up to 48 machines, fully interconnected within a node and connected between nodes via RoCE; this brings approximately a 30% pretraining-throughput improvement.

• Memory optimization: ZeRO-1, selective recomputation, automatic OOM offloading, and routing padding tokens to zero-computation experts.

• Large-scale deployment of the Muon optimizer (TP parallelism, deduplication of DP state, and dedicated optimization of symmetric matrix-multiplication kernels).

• Long context: LSA warm-up forward-only + KL loss; all-gather context parallelism scaled beyond 512 ways; overlapping computation and communication (ScMoE, and overlap between top-k indexing and KV all-gather).

• Reliability: deterministic operators with dual communication/computation paths; segmented binary-tree accumulation for reduction operators; bit-flip detection; automatic fault identification, traffic switching, and recovery, with repaired links reused after stress testing.

Inference and deployment (service layer)

• Million-token-context inference under memory constraints: absorb computation mode (prefill/decode), parallel indexer and MLA prolog, and multi-GPU KV-cache partitioning (KVP); ScMoE uses core-control capabilities to fully parallelize dense and MoE streams; super kernel reduces operator-launch overhead; Weight Prefetch uses a large L2 cache to hide I/O latency; 200Gbps network cards transfer layer-wise KV caches.

• Deployment: prefill–decode (PD) disaggregation; Prefill nodes use multi-node Chunked Pipeline Parallel (CPP) to reduce the Expert-Parallel domain, plus Attention Sequence Parallelism (SP); Decode nodes use KVP to shard the KV cache.

• Official deployment entry points: SGLang cookbook for GPUs, SGLang-FluentLLM for NPUs; weights released on GitHub at https://github.com/meituan-longcat/LongCat-2.0 and on HF (MIT).

Review and scope

• The blog and model card share a source (the same team, released the same day), so their technical details mutually corroborate each other; they also agree with AlphaSignal's independent analysis (Review 04) on the three LSA strategies, N-gram, and the three MOPD expert groups (AlphaSignal adds: approximately 128B N-gram parameters versus the official 135B, a minor difference in scope).

• The blog does not name the chip vendor ("domestic compute chips"); the community infers Huawei Ascend 910C from the official acknowledgment of HCCL (Huawei's communication library) (see the r/LocalLLaMA discussion in Review 14), but this is an inference rather than official confirmation.

• Scope: the blog is an engineering note rather than an independent evaluation. Official throughput/reliability figures (such as a 70%+ reduction in failure rate) have no third-party verification; the deployment approach targets very large clusters and offers individual developers mainly the SGLang cookbook as a reference.

• Related documents: Review 01 (official benchmark table), Review 04 (AlphaSignal verification).