iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
16,454 Commits 3 Branches 0 Tags
006af4b9562d0b60de0f1b11ab2fb33dea646cde
Commit Graph
16454 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
aoshen02 006af4b956 [Bugfix] Skip routed-experts hot path when disabled (#42148) 2026-05-09 18:01:04 -07:00
Wentao Ye ea0e501bb1 [KV Connector] Remove compat support for pre-v0.12.0 constructor signatures without KVCacheConfig (#39832) 
The v0.12.0 release contained initial support for HMA in KV Connectors. As part
of these changes, a KVCacheConfig argument was added to KV connector
constructors. Backwards compatibility support for out-of-tree connectors was
included in this change, with a very prominent warning. See #25712 and #27887.

Since the warning has been around for over 5 months, we can safely remove
the support of it.

Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-09 23:39:46 +00:00
Wentao Ye f80aa53c9d [Refactor] Nixl util using lazy init (#41392) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-09 17:46:52 -04:00
Juhi Mittal 7a2b596982 [Quantization] Add ModelOpt NVFP4 W4A16 (4-bit weights, fp16/bf16 activations) support (#41769) 
Signed-off-by: Juhi Mittal <juhim@nvidia.com>
 2026-05-09 21:15:50 +00:00
Jiangyun Zhu 2ee8c2a56e [SpecDecoding] extend mtp support for mimo 2.5 (#41905) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-05-09 18:22:59 +00:00
SoluMilken cd74911d92 [Model] use AutoWeightsLoader for DeepSeekV2 (#41706) 
Signed-off-by: SoluMilken <ypiheyn.imm02g@g2.nctu.edu.tw>
 2026-05-10 01:55:25 +08:00
SoluMilken 25abddc1a5 [BugFix] Fix Gemma4 'layers.0.moe.experts.0.down_proj_packed' KeyError issue (#40708) 
Signed-off-by: SoluMilken <ypiheyn.imm02g@g2.nctu.edu.tw>
 2026-05-09 17:20:44 +00:00
Nicolò LucchesiandZhanqiuHu 171d59ae8d [Bugfix][PD] Fix DSv4 Disaggregated (#41957) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: ZhanqiuHu <zhu@redhat.com>
 2026-05-09 16:48:24 +00:00
Thomas ParnellandJiangyun Zhu 3dda9aeb54 [Bugfix] Remove nested torch.compile in GDN rearrange_mixed_qkv causing CUDA graph capture failure (#42070) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-05-09 08:30:55 -07:00
Kermit adb6d96516 [Bugfix] Fix GDN KKT precision loss on Hopper GPUs by aligning tl.dot operand layout with WGMMA (#42076) 
Signed-off-by: kermit <ckeming@outlook.com>
 2026-05-09 13:08:46 +00:00
Thien Tran 530d371302 [DSv4] Improved fused Indexer Q quant kernel (#41428) 2026-05-09 01:20:32 -07:00
Micah Williamson 34ab4f2565 [ROCm] Upgrade aiter to v0.1.13-rc5 (#42113) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-05-09 08:13:45 +00:00
Jee Jee Li ecd0b60aad [LoRA] Initial EP support for LoRA (#40867) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-05-09 00:31:23 -07:00
Richard BarnesandClaude d6563d693c Require C++20 for compatibility with PyTorch (#40380) 
Signed-off-by: Richard Barnes <rbarnes@meta.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-08 22:04:43 -07:00
Rishapveer Singh f6490a2841 [Bugfix] Preserve leading/trailing whitespace in GLM non-streaming tool parser (#42026) 
Signed-off-by: Rishapveer Singh <singhrishapveer@gmail.com>
 2026-05-08 21:49:15 -07:00
Terrence ZhaoandCursor a2812becd6 [Models] Cohere Eagle + fix to Cohere MoE (#42078) 
Signed-off-by: Terrencezzj <terrence@cohere.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-08 21:46:26 -07:00
Markus HartikainenandTJian e8f9038ebd [ROCm][Bugfix] Re-tag AITER MoE weights as preshuffled after replace_parameter (#42061) 
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-05-08 21:42:07 -07:00
David Zhenggemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Andreas KaratzasKunshang Ji
 		df2636a9d8 [Bugfix] Fix LOGITPROC_SOURCE_ENTRYPOINT test to use spawn-compatible dist-info registration for XPU/ROCm (#42040) 
Signed-off-by: dqzhengAP <dqzheng1996@gmail.com>
Signed-off-by: David Zheng <153074367+dzhengAP@users.noreply.github.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-05-09 12:32:04 +08:00
Shengqi Chen 97cc7685c4 Add @Harry-Chen in CODEOWNERS (#42130) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
 2026-05-09 04:08:22 +00:00
haosdent e934e459e6 [CI][Bugfix] Make test_gpt2_cache_hit observable across V1 EngineCore (#42037) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-05-09 11:53:15 +08:00
David Zheng 845ca327ce [Bugfix] Fix test_whisper distributed test process handling (#42038) 
Signed-off-by: dqzhengAP <dqzheng1996@gmail.com>
 2026-05-09 11:37:21 +08:00
Yan MaandKunshang Ji 4f6fa6341d [XPU] update supported models on XPU (#41911) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-05-09 10:44:03 +08:00
Ethan Feng a43bc34baf [Docs] Update server entrypoint examples (#42077) 
Signed-off-by: Ethan Feng <ethan.fengch@gmail.com>
 2026-05-09 02:03:52 +00:00
Ethan Feng 236bf9d152 [Docs] Fix RLHF example links (#42073) 
Signed-off-by: Ethan Feng <ethan.fengch@gmail.com>
 2026-05-09 02:03:42 +00:00
Lucas Wilkinson b1728c1e66 [Attention][Cleanup] Remove tree attention (#42121) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-05-08 18:36:19 -07:00
Yan MaandKunshang Ji be0dcc29dc [XPU] remove q/k/v force contiguous for flash_attn (#40356) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-05-09 01:19:05 +00:00
Sumanth R Hegde e3b65a5ba0 [feat] Add explicit /start_weight_update and /finish_weight_update APIs for weight transfer (#39212) 2026-05-08 18:03:33 -07:00
Harry Mellor 30f519e947 Use pre-commit / pre-run-check to gate docs build too (#42053) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-05-09 00:02:51 +00:00
Roy Wang 60851b1d22 [Bugfix][KV Transfer] Reject NixlConnector + expandable_segments:True (#41237) 2026-05-08 16:47:33 -07:00
Michael Goin 8bcd8a260c [Bugfix] Fix FlashInfer CUTLASS MXFP4-MXFP8 MoE by restoring swizzled scale (#42089) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-05-08 15:59:06 -07:00
John Calderon 8a2fc80b84 [CUDA][CUTLASS] Enable cutlass scaled mm for non-compatible sizes (#41868) 
Signed-off-by: John Calderon <jcalderon@nvidia.com>
 2026-05-08 15:58:05 -07:00
pmaybankandTJian 6881c754e1 use HIP_VERSION variables to guard against duplicate atomicAdd definitions (#41802) 
Signed-off-by: Philip Maybank <pmaybank@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-05-08 18:44:37 -04:00
Kevin H. Luu 0c2e9d4892 [CI] Narrow misc.yaml source dependencies (#42059) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-05-08 15:10:12 -07:00
Kevin H. Luu d2f22dfc9f [CI] Narrow engine.yaml source dependencies (#42055) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-05-08 14:55:33 -07:00
Kevin H. Luu f4dd5c116c [CI] Narrow Platform Tests (CUDA) source dependencies (#42054) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-05-08 14:54:06 -07:00
Kevin H. Luu f47ccc8b1c [CI] Narrow pytorch.yaml compile job source dependencies (#42057) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-05-08 14:43:17 -07:00
David OyandClaude dbd86a67e3 [Bugfix][Gemma4] Fix infinite loop and array boundary issues in tool parser (#41991) 
Signed-off-by: David Oy <david.oy@baseten.co>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-08 17:24:37 -04:00
Nico HolmbergnholmberTresTres PoppDoug LehrCursorDouglas Lehr
 		2c6b59b807 [ROCm][Perf] Add Fused Shared Expert (FSE) support for Qwen3-Next (#39280) 
Signed-off-by: nholmber <nholmber@users.noreply.github.com>
Signed-off-by: Tres Popp <tres.popp@amd.com>
Signed-off-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: nholmber <nholmber@users.noreply.github.com>
Co-authored-by: Tres <tpopp@users.noreply.github.com>
Co-authored-by: Tres Popp <tres.popp@amd.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-05-08 15:38:00 -04:00
haosdentandNicolò Lucchesi 44e6b44a21 [CI][Elastic EP] Fix Elastic EP Scaling Test Failure (#41792) 
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-05-08 15:17:44 -04:00
Hiroaki Mikami 90f145aaf7 [Models][Gemma3/Gemma4] Support hidden_act variants in gated MLP (#40588) 
Signed-off-by: Hiroaki Mikami <hiroaki8270+github@gmail.com>
 2026-05-08 11:29:11 -07:00
Ethan Feng 4140faa4a5 [Docs] Fix OpenAI batch model argument examples (#42066) 
Signed-off-by: Ethan Feng <ethan.fengch@gmail.com>
 2026-05-08 14:02:46 +00:00
liuzhenwei f2bbd575e2 [CI][XPU] Skip fork-dependent logits processor test (#42013) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-05-08 06:10:19 -07:00
haosdent 52458b60a8 [CI][Examples][RLHF] Disable async scheduling in rlhf_async_new_apis (#42042) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-05-08 04:58:48 -07:00
Harry Mellor 630820a59b Make docs environment deterministic (#41926) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-05-08 10:13:03 +00:00
Chaojun Zhang 19df11f5d1 [CI][XPU]Ignore some lora tests from LoRA Intel CI pipeline (#42010) 
Signed-off-by: chaojun-zhang <chaojun.zhang@intel.com>
 2026-05-08 17:34:27 +08:00
haosdent 36b2c79d4b [CI][Bugfix] Drop duplicated examples/ prefix in tensorize_vllm_model command (#42039) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-05-08 02:23:22 -07:00
haosdent 160858cba4 [CI][Bugfix] Surface subprocess output in spawn_new_process_for_each_test (#41943) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-05-08 10:39:37 +02:00
Simon Danielsson f9b9bf3bbb [CI][ROCm] Ship RIXL with vllm/vllm-openai-rocm (#41634) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
 2026-05-08 07:05:17 +00:00
TJianandCyrus Leung 445d747434 [Bugifx] Missing Renderer for fastokens mode (#41984) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-05-07 23:45:14 -07:00
wang.yuqi 77b13b9602 [Docs] Reorganize examples docs. (#41082) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-07 23:23:44 -07:00