iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,683 Commits 3 Branches 0 Tags
42fadebecb79290ad722f33f3094de23b121f33d
Commit Graph
2424 Commits
Author SHA1 Message Date
Benjamin Chislett 9040cd40af [DSV3.2][MTP] Optimize Indexer MTP handling (#36723) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-03-11 12:16:56 +08:00
fangyuchu fa0d353acf [Bugfix] Surface exceptions from non-blocking execute_model in UniProcExecutor to avoid DP deadlocks (#35194) 
Signed-off-by: fangyuchu <fangyuchu@qq.com>
 2026-03-11 03:22:21 +00:00
Matthew Bonanni 8ab3d7427c [Bugfix] Fix DeepSeek V3.2 OOM during CG memory profiling (#36691) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-11 03:01:07 +00:00
Woosuk Kwon 195d1ca3e8 [Minor] Enhance error message for TRTLLM decode uniformity check (#36609) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-10 15:38:45 -07:00
Nick Hill 65b2f405dc [Core] Simplify core kv-cache blocks initialization logic (#36521) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-03-10 20:20:02 +00:00
Woosuk Kwon f088a831dd [Model Runner V2] Use unpadded num_tokens for PW CUDA graph attn metadata (#36626) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-10 09:30:56 -07:00
Harry Mellor f83b933b84 [CI] Bump mypy version to 1.19.1 (#36104) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-10 09:18:28 -07:00
Pleaplusone 82f3f30e26 [ROCm][Perf] Enable sparse_mla's cudagraph on ROCm platform (#35719) 
Signed-off-by: ganyi <ygan@amd.com>
 2026-03-10 09:14:35 -07:00
Matthew Bonanni 9095cbbfb6 [Bugfix][Sparse MLA] report indexer CG support properly (#36519) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-10 09:14:31 -07:00
Srinivasoo7srinivas_oo7Srinivasoo7Or Ozeri
 		106ff69c4e feat(kv-offload): Strategy A — StoreReusedOffloadingManager gates CPU stores on reuse frequency (#35342) 
Signed-off-by: srinivas_oo7 <Sriusa4414@gmail.com>
Signed-off-by: Sriusa4414@gmail.com
Signed-off-by: Srinivasoo7 <158864704+Srinivasoo7@users.noreply.github.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: Srinivasoo7 <158864704+Srinivasoo7@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-03-10 14:43:40 +00:00
SoluMilken 409c4e632d [Misc] fix typo: homogenous-> homogeneous (2 lines change) (#36508) 
Signed-off-by: SoluMilken <ypiheyn.imm02g@g2.nctu.edu.tw>
 2026-03-10 06:25:37 -07:00
Mark McLoughlin 234860399b [Frontend][Core] Revert "Add shutdown timeout" (#34730 and #36270) (#36628) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-03-10 06:20:41 -07:00
Vadim Gimpelson 4ff8c3c8f9 [BUGFIX][Mamba][Qwen3.5] Zero freed SSM cache blocks on GPU (#35219) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2026-03-10 03:32:20 -07:00
Nick Hill ddbb0d230a [Model Runner V2] Fix mm input embeddings lookup (#36588) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-03-10 00:24:58 -07:00
Nick Hill 9efc3bdcd6 [Model Runner V2] Fix _compute_slot_mappings_kernel for chunked prefill (#36580) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-03-10 00:23:42 -07:00
Wentao Ye 7279374f91 [Perf] Compute maxsim in worker side, reducing redundant copies, 2.7% E2E throughput improvement (#36159) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-09 20:55:58 -07:00
Woosuk Kwon 006aea17d7 [BugFix] Remove incorrect assert in split_decodes_and_prefills (#36553) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-09 20:02:02 -07:00
Woosuk Kwon 2a194ddd72 [Model Runner V2] Add model_state inputs to CUDA graph capture (#36544) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-09 15:14:51 -07:00
Lucas Wilkinson 483463f735 [MRV2] Extensible CG dispatch rework (#35959) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-03-09 13:58:45 -07:00
Matthew Bonanni 4e571ce643 [MTP][Misc] Clean up dead code (#36507) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-09 14:43:06 -04:00
Woosuk Kwon 10a5f4d53d [Model Runner V2] Use NamedTuple for execute_model_state (#35930) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-09 11:17:34 -07:00
Woosuk Kwon 6e956d9eca [Model Runner V2] Add dummy profile_cudagraph_memory API (#36520) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-09 10:20:13 -07:00
Andreas Karatzas c174d54f86 [ROCm][CI] Fix ROCm attention backend validation for head sizes, block sizes, and compute capability checks (#36292) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-09 12:02:41 -05:00
Roberto L. Castro 580864d81e [Attention][Perf][Kernel] Replace torch.cat with vectorized CUDA kernel MLA query concat - DeepSeek-V3.2 (#34917) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
 2026-03-09 09:50:36 -07:00
Matthew Bonanni 00c4cb5606 [Bugfix] Clear stale CG keys after memory profiling (#36416) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-09 11:56:00 -04:00
Matthew Bonanni 77a73458e3 Reapply [Attention] Refactor check_and_update_config (#35122) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-09 07:17:14 -07:00
Cyrus Leung f96c3ab08c [Deprecation][1/2] Remove items deprecated in v0.18 (#36470) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-09 03:43:23 -07:00
Li, Jiang 217f27598d [Bugfix] Avoid to replace non-tensor members in cpu model runner (#36430) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-03-09 13:06:28 +08:00
cong-or 747431044d feat(attention): extract KV-cache update from FlexAttention backend (#36263) 
Signed-off-by: cong-or <conchubhar.gannon@gmail.com>
 2026-03-08 20:40:12 -07:00
Weiguang LiClaude Opus 4.6Li, Jiang <jiang1.li@intel.com>
 		43aa389231 [Bugfix] Fix CPU OMP autobind assertion to use local_world_size (#35815) 
Signed-off-by: liweiguang <codingpunk@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-03-08 20:07:29 -07:00
Sage 4497431df6 [Frontend] Add GPU-less render serving path (vllm launch render) (#36166) 2026-03-08 16:35:09 +01:00
Wei Zhao 379689d533 [Perf] Support FP8 KV cache for Flashinfer MLA Sparse (#35891) 2026-03-07 13:51:54 -08:00
PatchyTIS a6be75dbd2 [Core] NGram GPU Implementation compatible with Async Scheduler (#29184) 2026-03-07 13:51:37 -08:00
Matthew Bonanni ebb9cc5f2b [UX][Startup] Account for CUDA graphs during memory profiling (#30515) 2026-03-07 13:49:23 -08:00
Mengtao (Martin) YuanandMartin Yuan 1a9718085c Fix CUDA graph decode capture crash in AITER FlashAttention (#36042) 
Signed-off-by: Martin Yuan <myuan@meta.com>
Co-authored-by: Martin Yuan <myuan@meta.com>
 2026-03-06 18:12:07 -08:00
Nick Hill b354686524 [Model Runner V2] Fix warmup for pipeline parallel (#36280) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-03-06 16:58:51 -08:00
Nick HillandMark McLoughlin 6a18d8789b [Core] Fix benign error log during normal shutdown (#36270) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
 2026-03-07 00:39:21 +00:00
Chuan (Richard) Li c188749bcd [ROCm] Support MLA with nhead<16 and FP8 KV cache for TP=8 (Kimi K2.5/Linear) (#35850) 
Signed-off-by: Li <chuali@amd.com>
 2026-03-06 20:24:03 +00:00
Nick Hill 26bd43b52d Revert "[BugFix] Fix engine hanging after KV cache initialization fai… (#36262) 2026-03-06 08:28:09 -08:00
Travis JohnsonandNick Hill 6b625a8807 [Bugfix] Quickfix followups to busy loop removal in #28053 (#36068) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-03-06 08:13:05 -08:00
Raphaël Rialland 39f9ea0da4 [Bugfix] Fix cudagraph_mode:FULL dispatch (This does not impact FULL_AND_PIECEWISE (default)) (#36165) 2026-03-06 09:15:31 -05:00
Nicolò Lucchesi 5b3ba94ab4 [Core][KVConnector] Support HMA+NixlConnector (#35758) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-06 08:51:21 +01:00
zhanqiuhuandNicolò Lucchesi 90f3c01fa4 [Spec Decode][KV Connector] Fix KV transfer in PD + speculative decoding (#35158) 
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-03-06 08:50:44 +01:00
Andreas Karatzas 807d680337 [ROCm][CI] Fix tool use test stability - disable skinny GEMM, prefix caching, eliminate batch variance (#35553) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-06 15:15:12 +08:00
Yongye ZhuClaude Opus 4.5Nick HillNick Hill
 		86e1060b17 [Bugfix] Fix inner_dp_world initialization order for multi-node TP (#35892) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2026-03-05 22:04:44 -08:00
Mark McLoughlinandClaude Sonnet 4.5 27066d1b2b [Frontend][Core] Add shutdown timeout - allowing in-flight requests to finish (#34730) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-03-05 22:04:31 -08:00
cong-or 57c84ff129 perf: add __slots__ to KVCacheBlock (#36164) 
Signed-off-by: cong-or <conchubhar.gannon@gmail.com>
 2026-03-05 22:04:09 -08:00
Shiyan DengandLu Fang 0a208d1f54 [BugFix] Fix engine hanging after KV cache initialization failure (#35478) 
Signed-off-by: Shiyan Deng <dsy842974287@meta.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
 2026-03-05 20:58:09 -08:00
Shiyan DengandLu Fang 03a49bb8f0 [Feature] Add --distributed-timeout-seconds CLI option (#36047) 
Signed-off-by: Shiyan Deng <dsy842974287@meta.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
 2026-03-05 20:57:51 -08:00
Cyrus Leung 6dd302653f [Misc] Rename group_mm_kwargs_by_modality -> group_and_batch_mm_kwargs (#36158) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-06 12:32:48 +08:00