The official technical blog provides the complete technical foundation for LongCat-2.0 (LSA sparse attention, N-gram Embedding, 6D parallel training on domestic compute, and prefill-decode disaggregated deployment), making it useful for assessing the model's intended long-context and Agent capabilities, as well as reproducing the official benchmarks and deployment path.
A 1.6T-total-parameter MoE with approximately 48B parameters activated per token; all full training and large-scale deployment use domestic compute clusters.
Pretraining: more than 50,000 domestic compute chips, taking more than a month, on 35T+ tokens, with no rollback throughout and no unrecoverable loss spikes.
Long-range capabilities: LongCat sparse attention + training on hundreds of billions of tokens of million-token-context data + dedicated post-training → strong performance on coding and agent tasks.
Deeply adapted to Claude Code, OpenClaw, and Hermes; online experience at https://longcat.chat; API access at https://longcat.chat/platform/docs/.
LongCat Sparse Attention (LSA) — evolved from DeepSeek Sparse Attention (DSA), targeting the bottlenecks in DSA's Lightning Indexer: "discontinuous index outputs + quadratic index scoring":
Streaming-aware Indexing (SI): hardware-aligned contiguous access + dynamic random selection; turns fragmented GPU-memory access into sequential reads and merges HBM accesses.
Cross-Layer Indexing (CLI): adjacent attention layers have substantially consistent distributions of salient tokens, so one index can be reused across multiple consecutive layers (supported by cross-layer distillation during training).
Hierarchical Indexing (HI): two-stage coarse-to-fine scoring (block-level coarse retrieval → fine selection within candidates), enabled on demand.
The three components are orthogonal and can be toggled independently; all are extended to 3-step MTP speculative decoding (the target model shares one index every 2 layers; the 3 draft steps share one index).
N-gram Embedding: inherited from LongCat-Flash-Lite; n-gram size=5, 135B parameters; expands the embedding space by more than 100x. Expansion principles: MoE sparsity has passed its sweet spot (~97%), while the N-gram share is constrained to the optimal range (<10% of total parameters; experiments show the advantage disappears above 50%).
6D parallelism: EMBP parallelism for N-gram Embedding is added alongside TP/CP/EP/DP/PP.
Supernodes: physical supernodes contain up to 48 machines, fully interconnected within a node and connected between nodes via RoCE; this brings approximately a 30% pretraining-throughput improvement.
Memory optimization: ZeRO-1, selective recomputation, automatic OOM offloading, and routing padding tokens to zero-computation experts.
Large-scale deployment of the Muon optimizer (TP parallelism, deduplication of DP state, and dedicated optimization of symmetric matrix-multiplication kernels).
Long context: LSA warm-up forward-only + KL loss; all-gather context parallelism scaled beyond 512 ways; overlapping computation and communication (ScMoE, and overlap between top-k indexing and KV all-gather).
Reliability: deterministic operators with dual communication/computation paths; segmented binary-tree accumulation for reduction operators; bit-flip detection; automatic fault identification, traffic switching, and recovery, with repaired links reused after stress testing.
Million-token-context inference under memory constraints: absorb computation mode (prefill/decode), parallel indexer and MLA prolog, and multi-GPU KV-cache partitioning (KVP); ScMoE uses core-control capabilities to fully parallelize dense and MoE streams; super kernel reduces operator-launch overhead; Weight Prefetch uses a large L2 cache to hide I/O latency; 200Gbps network cards transfer layer-wise KV caches.
Deployment: prefill–decode (PD) disaggregation; Prefill nodes use multi-node Chunked Pipeline Parallel (CPP) to reduce the Expert-Parallel domain, plus Attention Sequence Parallelism (SP); Decode nodes use KVP to shard the KV cache.
Official deployment entry points: SGLang cookbook for GPUs, SGLang-FluentLLM for NPUs; weights released on GitHub at https://github.com/meituan-longcat/LongCat-2.0 and on HF (MIT).
The blog and model card share a source (the same team, released the same day), so their technical details mutually corroborate each other; they also agree with AlphaSignal's independent analysis (Review 04) on the three LSA strategies, N-gram, and the three MOPD expert groups (AlphaSignal adds: approximately 128B N-gram parameters versus the official 135B, a minor difference in scope).
The blog does not name the chip vendor ("domestic compute chips"); the community infers Huawei Ascend 910C from the official acknowledgment of HCCL (Huawei's communication library) (see the r/LocalLLaMA discussion in Review 14), but this is an inference rather than official confirmation.
Scope: the blog is an engineering note rather than an independent evaluation. Official throughput/reliability figures (such as a 70%+ reduction in failure rate) have no third-party verification; the deployment approach targets very large clusters and offers individual developers mainly the SGLang cookbook as a reference.
Related documents: Review 01 (official benchmark table), Review 04 (AlphaSignal verification).
LongCat 2.0
