Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 0376f81530 [Perf][Pooling] Avoid blocking seq_lens GPU-to-CPU copy for pooling in FlashInfer metadata builder (#57214) 4 people 2026-09-27 09:06:38 -07:00
• 44af287ebe [ROCm][Perf] Enable layer-aware CSA2 multi-stream overlap for DeepSeek-V4.1-Flash (#57407) Shanshan Shen 2026-09-27 23:48:01 +08:00
• 924707f1bf [CI] Split (H200 MIG 35GB) Spec Decode Speculators + MTP into 4 named jobs (#57237) 4 people 2026-09-27 08:01:31 -04:00
• c8d7a7dd13 [Perf][Attention] Remove D2H sync from FlashInfer SM90 sparse MLA plan under async scheduling (#58684) Nicolò Lucchesi 2026-09-27 13:06:23 +02:00
• 521235d6df Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes lesj0610 2026-09-27 18:25:21 +09:00
• 5567cc1b25 [Mamba] RecoverSSM align: keep a boundary-ending state in its block 3 people 2026-09-27 11:06:54 +02:00
• 24c9772d19 [Attention][CPU] Use zentorch SDPA for CPU MLA prefill (#54967) 3 people 2026-09-27 14:00:55 +05:30
• ed2a9665ea Merge branch 'main' into gdn-recoverssm Jürgen Schmied 2026-09-27 08:01:32 +02:00
• a9eafde59c [Perf][DSv4.1] Fuse small-batch WO-A with inverse RoPE and MXFP8 quant on SM100/SM103 (#58634) Juntian LiuandClaude Opus 5.5 2026-09-26 21:16:12 -07:00
• 5da84df6b7 [BugFix] Honor drop_eagle_block in MambaManager MOVZX 2026-09-27 10:43:44 +07:00
• fba47397f3 [Bugfix] V1: clear stale allowed_token_ids mask in InputBatch.condense (#43931) VarshithandClaude 2026-09-26 22:44:54 -05:00
• 5e6656e7a0 [Bugfix][Spec Decode] Add get_top_tokens() to Qwen4ExpMTP draft model MOVZX 2026-09-27 10:31:47 +07:00
• fff03267da [Bugfix][NIXL] Release a dead peer's NIXL state without waiting for TTL (#50047) YannikHintereggerandxijiade.aihemaiti 2026-09-27 05:30:09 +02:00
• 43fbd6510b [Qwen4Exp] Serve PLE rows from the loader's mapped checkpoint shards (PR #58815) MOVZXandHinata 2026-09-26 21:14:50 +00:00
• 5af48b498c [Qwen3.5] Build MTP head unquantized when checkpoint ships it dense (PR #58813) MOVZXandHinata 2026-09-26 21:07:13 +00:00
• 835c44af15 [Qwen4Exp] Reinterpret FP8 E4M3 PLE as int8 for SM86 pinned-host lookup (PR #58856) MOVZXandHinata 2026-09-26 20:50:17 +00:00
• 758cc4f9e1 [Qwen3.5] Pass quant_config and prefix to embed_tokens in Qwen3_5Model MOVZXandHinata 2026-09-26 20:48:08 +00:00
• 3137ff0773 [ROCm][Kimi-K3] Make VLLM_ROCM_USE_AITER_MOE_SITUV2 select a4w4/a8w4/a16w4 (#58201) Hongxia Yang 2026-09-26 22:26:49 -04:00
• 0c87a197b8 [Bugfix] Disable sequence parallelism / async TP under batch invariance and add a TP regression test (#56377) Lio Einaudiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-09-27 10:19:08 +08:00
• 7a877ae34d [Qwen3.8-Flash-Next] Avoid memory fragmentation in QSA indexer logits workspace (#57105) Thien Tran 2026-09-27 10:11:52 +08:00
• eb0f2ca37f [mooncake] support CUSTOM_MEM_POOL in vllm (#49300) 4 people 2026-09-27 08:26:05 +08:00
• cd1daf58a9 [Metrics][KV Offload] Add Prometheus metrics for SimpleCPUOffloadConnector (#57251) mevince 2026-09-26 17:17:11 -07:00
• 03cae7837f [Spec Decode][Qwen4Exp] RecoverSSM for Qwen GDN and the PLE short conv Jürgen SchmiedandClaude 2026-09-27 00:07:04 +02:00
• 4be061c5af [Kernel] Bump FlashKDA to keep the recurrent state in fp32 (#58846) 3 people 2026-09-27 00:20:44 +02:00
• 927c87b347 [PCP][DCP] Support DCP target model with non-DCP Dspark (#56723) 3 people 2026-09-26 18:12:35 -04:00
• a80e41d690 [Mamba] Share RecoverSSM commit kernels and helpers across models Jürgen SchmiedandClaude 2026-09-27 00:07:04 +02:00
• 77871126f9 [Kernel][DSV4.1] Fuse MoE finalize into the TP all-reduce + mHC boundary (#58586) Yongye ZhuandClaude Opus 5.5 2026-09-26 12:08:36 -07:00
• 379e9a1ea8 [Security] Harden message sanitization (#58832) Cyrus Leung 2026-09-27 00:28:32 +08:00
• 7d8c5fe9a9 [Bugfix][DSV4.1] Avoid host sync in ViT CUDA graph replay metadata (#58499) Kevin H. Luu 2026-09-26 08:37:36 -07:00
• 0239bd2549 [CI] Stabilize batch submission in full CUDA graph tests (#58810) Andreas Karatzas 2026-09-26 10:35:56 -05:00
• 3576691c42 [GLM5.3 Bug] Fix sparse indexer attn topk backend selection (#58594) Wentao Ye 2026-09-26 11:35:28 -04:00
• 79ac28c5bd [CI] Reduce CUDA graph mode test overhead (#58749) Misha Goin 2026-09-26 11:22:33 -04:00
• 5113cf9d16 [Mypy] Fix mypy typing for Qwen and Qianfan models (#58046) Ashraf Bhuiyan 2026-09-26 10:31:51 -04:00
• dfab504333 [Bugfix][Frontend] Detect Anthropic inline-system merge against the resolved chat template (#58754) Robert ShawandRobert Shaw 2026-09-26 10:00:56 -04:00
• d64f277c54 [Bugfix] Fix Anthropic Thinking Disabled with P/D (#58786) Robert ShawandRobert Shaw 2026-09-26 09:50:57 -04:00
• ad6817b68d [Perf][MoE] Index expert mapping lookups in RoutedExperts.load_weights (#58720) Willian Zandmgoin 2026-09-26 09:48:24 -04:00
• bf93e1f92c [Refactor] Remove dead tests utils (#58803) Wentao Ye 2026-09-26 09:16:53 -04:00
• 2bb7605ae1 [Elastic EP] Fix EPLB load statistics during scaling (#58473) Itay AlroyandTyler Michael Smith 2026-09-26 15:05:37 +03:00
• 31f2e70cd3 [Security] Gate per-request multimodal processor kwargs (#58830) Juan Pérez de AlgabaandCursor 2026-09-26 13:10:35 +02:00
• 2f41e00235 [CI] Split (B200) Miscellaneous Kernels into mHC, FLA Ops and Misc named jobs (#58609) 3 people 2026-09-26 07:10:14 -04:00
• a4eb3f25d6 [Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer (#57263) 4 people 2026-09-26 02:01:17 -07:00
• 3b4566c5cf [CI] Only isolate the registry tests that need a fresh process (#58764) Aarushi JainandAndreas Karatzas 2026-09-26 01:03:45 -05:00
• ddd6fbca14 [Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasoning effort mappings (#58316) 3 people 2026-09-26 10:18:58 +08:00
• 5840d95284 [Bugfix][ROCm] AMD-Quark mixed-precision DeepSeek-V4.1 support (#57071) xiao-llmandCursor 2026-09-25 21:51:18 -04:00
• b6d2b4fc25 [RL] Add sharding-aware NCCL M2N weight transfer (#51520) 3 people 2026-09-25 18:38:53 -07:00
• 4bb804cc5b [Perf][DSv4.1] Shard the Engram wkv projection across TP ranks (#58678) Shuolei Wang 2026-09-26 09:15:57 +08:00
• 8a2364605c [Core] Bound draft-token RPC waits by the execute-model timeout (#58779) Chandra MouryaandVS Chandra Mourya 2026-09-25 17:40:22 -07:00
• fdcce47e9b [Bugfix][Frontend] Fix Inkling tool name leaking into content after reasoning (#58792) Baljinder HothiandFlora Feng 2026-09-25 20:38:52 -04:00
• da1aaec31f [CI] Split (H200 MIG/MI300) Basic Correctness into named jobs (#57054) 4 people 2026-09-25 20:05:33 -04:00
• 9bf44c4013 [Bugfix][Reasoning] Count Kimi K3 reasoning tokens (#58372) Elvir CrnčevićandFlora Feng 2026-09-26 01:29:56 +02:00
• 974cb65155 [Bugfix] V1: fix allowed_token_ids_mask aliasing in InputBatch.swap_states (#48419) Rui ZhuandClaude 2026-09-25 19:23:11 -04:00
• b040321b33 fix: perf: use startswith(x, i) instead of string slicing to avoid O(N^2) (#52580) Ricardo-M-LandClaude Opus 4.6 2026-09-26 07:22:13 +08:00
• 6ec2f84e1a [Pooling] Preserve BERT-family heads for raw logits (#57664) Taneem IbrahimandWentao Ye 2026-09-25 18:10:14 -05:00
• 614b55f0b6 [AuxOutput] Only require Model Runner V2 on GPU platform (#58205) Linkun 2026-09-25 15:55:21 -07:00
• e55d076f89 [Bugfix] Don't drop the rest of the allocator config when toggling expandable segments (#57982) Oxana Korzh 2026-09-25 16:34:19 -06:00
• 4ccfe12398 [watermarking] golden tests for backwards compatibility (#56809) 4 people 2026-09-26 00:01:48 +02:00
• b6761e8ded [Bugfix] Default missing detail for Responses API input images (#57241) yzong-rhandBen Browning 2026-09-25 17:08:55 -04:00
• 8b47e8b22c [Perf][Frontend] Defer reasoning usage recounts for non-continuous chat streams (#56067) 3 people 2026-09-26 05:08:23 +08:00
• de2d478d9a [Bugfix][Frontend] Document 404 response for /generative_scoring (#58788) Nick HillandKimi Code 2026-09-25 14:08:00 -07:00
• 7871963fcc [Bugfix][Frontend] Respect max_output_tokens in the Harmony tool-call loop (#58551) 5 people 2026-09-25 23:00:39 +02:00
• 83db8c8390 [Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 (#55957) Yousafandsfeng33 2026-09-25 16:43:08 -04:00
• e440b75bb6 [ROCm] Bump torch 2.13, triton 3.8, torchaudio, torchvision (#50605) 3 people 2026-09-25 14:55:52 -05:00
• 31cc226401 [ROCm][CI] Run the MLA attention+quant fusion test on ROCm (#58717) 3 people 2026-09-25 21:45:32 +02:00
• 2d7dd0e5c1 [Bugfix][CI] Report subprocess test skips as skips, not passes (#58701) stefankoncarevicandAndreas Karatzas 2026-09-25 21:44:49 +02:00
• 2bc902eb0f [CI] [MRV2] Restore MRV2 pp dp coverage (#57735) Taneem Ibrahim 2026-09-25 14:13:59 -05:00
• 22bbe3f102 [ROCm][CI] Expand single-GPU coverage on MI355 DPX (#57599) 6 people 2026-09-25 14:27:53 -04:00
• 353e491efd [Perf] DiffusionGemma: one-pass sampler statistics kernel (#58226) Matt MastracciandClaude Fable 5.1 2026-09-25 12:14:55 -06:00
• e3dd5b5a75 [ROCm][CI] Test AMD DeepSeek V4 MoE routing against a PyTorch reference (#58740) 3 people 2026-09-25 20:05:54 +02:00
• 1b922ffcbe [ROCm][CI] Add quantized MoE serving test for gfx950 (#58748) 3 people 2026-09-25 20:02:15 +02:00
• b21757555a [ROCm][CI] Cover the AITER MQA logits dispatch on gfx950 (#58724) 3 people 2026-09-25 19:43:40 +02:00
• 1417022c3d [ROCm][Kimi-K3] Optimize low-concurrency speculative KDA (#58045) 3 people 2026-09-25 10:33:30 -07:00
• c89b5b4176 [Bugfix][KV Connector] Reap expired NIXL leases behind a heartbeated head (#58292) GokayAIandGokayAI 2026-09-25 20:23:44 +03:00
• c88f4824ff [GLM5.3 Perf] Optimize glm 5.3 metadata op, 1.6~4.8x kernel level performance improvement (#58450) Wentao Ye 2026-09-25 13:15:18 -04:00
• 2617fe9383 [Bugfix][GLM-5.3-Flash] kpool corruption with speculative decoding (#58454) 5 people 2026-09-25 10:51:34 -06:00
• f83ea61cfe [Benchmark] Record model_id in bench latency/throughput --output-json (#58112) Yashasvi Chaurasia 2026-09-25 22:13:37 +05:30
• 2edbb32e58 [Bugfix][GLM-5.3-Flash] SM90 sparse MLA: index_kpool mismatch leads to corruption via unread query token (#58704) 3 people 2026-09-25 10:36:36 -06:00
• f339e0e757 [Bugfix][Frontend] Keep logprobs of parser-suppressed streaming chunks (#58583) 4 people 2026-09-25 18:33:19 +02:00
• 6491f481a7 [Bugfix] Stop allocator fragmentation from shrinking the KV cache during memory profiling (#58430) 5 people 2026-09-25 12:16:27 -04:00
• a44d7b5151 [Core][Logging] Fix JSON logging process decoration (#57957) Mark McLoughlin 2026-09-25 17:12:28 +01:00
• 38cc054fc2 [Perf][Kernel] Vectorized flat abs-max for dynamic per-tensor FP8 quantization (#58194) 3 people 2026-09-25 11:19:48 -04:00
• f550ad8bf7 [Kernel][Perf] Register-resident path for per-token-group 8-bit quant (#55330) chuan932andMisha Goin 2026-09-25 23:11:42 +08:00
• bb35e2587b [Pooling] Preserve reranker tokenization with document limits (#57666) Taneem IbrahimandWentao Ye 2026-09-25 10:04:41 -05:00
• 50f78084da [Bugfix] Fix generative scoring body cancellation (#57729) Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-09-25 10:01:31 -05:00
• 234df712ed [MRV2] Minor model_runner.py code cleanup (#58610) Nick Hill 2026-09-25 07:59:27 -07:00
• 73a78e6f1f [SpecDecode] Add LiLiCorr drafter (#57934) 4 people 2026-09-25 16:08:28 +02:00
• 25b0add7b8 [ROCm][CI] Pass weight_shape in MXFP8 block32 linear tests (#58698) djramicandAndreas Karatzas 2026-09-25 12:52:15 +02:00
• 45d56c0baf [Core] Model console logging as CLI configuration (#57205) Mark McLoughlinandAI Assistant 2026-09-25 11:36:59 +01:00
• fa6c407d15 [CI] Shard (H100) Helion Kernels five ways (#58645) Thang NguyenandClaude Opus 5.5 2026-09-25 06:25:11 -04:00
• afea5c20c7 [Docs] Add return annotation to fused_mm_input_norm_triton (#58687) Harry MellorandClaude Opus 5.5 2026-09-25 12:01:11 +02:00
• 378504a544 [MoE] Defer the TRTLLM-Gen top-k finalize on the modular path (#58635) Yongye ZhuandClaude Opus 5.5 2026-09-25 00:38:29 -07:00
• 24928a4d62 [UX][Frontend] Introduce vllm preload cli for fast restart (#56680) Isotr0py 2026-09-25 14:56:39 +08:00
• 5ff3bbfb08 [Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA (#58621) Yongye ZhuandClaude Opus 5.5 2026-09-24 23:26:50 -07:00
• 29468dde8b [Bugfix][KV Cache][MLA] Align packed block strides for V3.2 sparse MLA (#55528) 5 people 2026-09-25 14:33:36 +09:00
• 3a58aeb1a1 [Bugfix] Stop leaking the internal field name in the max_tokens validation error (#58336) shallow10 2026-09-25 12:55:16 +08:00
• 5a2ed2e7ee [Frontend] Handle Disable Thinking in /v1/messages (#58613) 4 people 2026-09-25 00:52:08 -04:00
• 35f92d2abc [ROCm] Credit ROCm/aiter for the block32 GEMM's packed kernel and in-launch split-K (#58659) Lingpeng JinandAndreas Karatzas 2026-09-25 12:37:41 +08:00
• 1f0bc49ee6 [Structured Outputs] Parse Lark grammars natively in the xgrammar backend (#58321) 3 people 2026-09-25 12:35:10 +08:00
• 8a269919f4 [MM] Add Triton kernel for mm_input_normal. (#56798) 3 people 2026-09-25 12:31:20 +08:00
• 267eee54bf [Qwen4Exp][ROCm] PLE n-gram table CPU offload (#57497) 4 people 2026-09-24 23:28:20 -05:00
• fa94450b74 [CI][ROCm] Prevent Model Executor apt stalls (#58607) Andreas KaratzasandCodex 2026-09-24 23:18:45 -05:00