iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,845 Commits 3 Branches 0 Tags
78300cdabffbaaffe7625ed34412ec2aec75535c
Commit Graph
20845 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Woosuk Kwon e644c8cd8c [Perf] Optimize DeepSeek V3.2 sequence parallelism (#51434) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-07 16:51:01 -07:00
Wentao Ye c39076feff [CI Test] Add specific unit test for mrv2 offloading (#51440) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-07 22:49:45 +00:00
Andrii SkliarandAleksandr Skliar a801e71cb8 [Perf] Improve --linear-backend filtering (#48735) 
Signed-off-by: Aleksandr Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Aleksandr Skliar <askliar@nvidia.com>
 2026-08-07 15:15:54 -07:00
Woosuk Kwon 46b5864054 [Perf] Narrow DeepSeek V3.2 eager CUDA graph region (#51425) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-07 14:37:43 -07:00
Michael GoinandOpenAI Codex c8fe1d5715 [Spec Decode] Register Qwen3.6 dSpark acceptance coverage (#51310) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-07 17:07:55 -04:00
Nick Hill 9823714346 [Bugfix] Drop stale layer kwarg from online MXFP4 kernel creation (fix precommit) (#51442) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 13:49:00 -07:00
Jiangyun Zhu 27d7303802 [CI] Fix Batch Invariance (B200) (#51417) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-08-07 13:45:29 -07:00
Wentao Ye 4a11ed5bc4 [DSv32/GLM Perf] Skip short prefill topk for dense mha layer, 97.9% kernel level latency reduction (#51298) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-07 20:30:42 +00:00
Elvir Crnčević 99950b0b86 [Profiler] Stamp vLLM version/commit into torch profiler trace metadata (#51389) 
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
 2026-08-07 16:29:59 -04:00
Wei-Ming Chen 1c94e8dc7d Add NVFP4 KV 4-over-6 scale search (#45187) 
Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com>
 2026-08-07 13:22:27 -07:00
Walter Beller-MoralesandHarry Mellor 70456e5e6f [Bugfix] when loading weights skip empty expert bias if model does not support them (#50937) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-07 12:45:52 -07:00
Nick Hill ac70ce96e0 [Frontend] Disable uvicorn signal handlers instead of racing them (#50916) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 13:40:24 -06:00
fxmarty-amd 3518110f2f [Online quantization] Add online MXFP4 quantization support (#49347) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-08-07 12:39:46 -07:00
Wentao Ye a671679e9f [MRv2 Feature] MR v2 weight offloading support (#51413) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-07 13:20:21 -06:00
Lucas Wilkinson 021b7d985b [Perf] Raise Blackwell CUDA graph capture default to 1024 (#49390) 2026-08-07 15:18:14 -04:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 56a4b63d44 [K3 Perf] Optimize k3 dspark fused kv, 4.5~4.6x kernel performance improvement (#50585) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-07 12:16:11 -07:00
coltonottleyandColton Ottley 0df620d429 [Test] Add packed DeepSeek-V4 KV zeroer geometry regression (#51288) 
Signed-off-by: Colton Ottley <colton@ottleyengineering.com>
Co-authored-by: Colton Ottley <colton@ottleyengineering.com>
 2026-08-07 12:11:09 -07:00
Aarushi JainandAarushi Jain a0056e103e [Test] Add ROCm AITER MLA op registration and env gating tests (#50930) 
Signed-off-by: Aarushi Jain <aarushi@amd.com>
Co-authored-by: Aarushi Jain <aarushi@amd.com>
 2026-08-07 14:07:31 -05:00
stefankoncarevic ebb2972562 [ROCm][CI][Bugfix] Do not microbatch a step that splits a prefix from its writer (#51402) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-08-07 13:55:34 -05:00
wangxiyuan 4eccf906ca [Attention] Mamba attention module refactor - Final part (#44857) 2026-08-07 11:27:10 -07:00
Harry Mellor 45273b8dcb [1/N] Harden Transformers modelling backend multi-modal path (#51408) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-07 17:30:43 +00:00
Jinzhen LinandOpenAI Codex fcde8e1460 [Refactor] refactor humming linear and moe backends to use explicit layer configs (#49610) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-07 10:03:19 -07:00
vanshbhatia-amd e229fdbc87 [ROCm] Add tuned selective_state_update float32 config for AMD Instinct MI325X (#50007) 
Signed-off-by: vanshbhatia-amd <vansh.bhatia@amd.com>
 2026-08-07 12:58:36 -04:00
Jack HuandAndreas Karatzas 34c1cd20a5 [Bugfix][ROCm] Fix ROCM_AITER_FA & ROCM_AITER_UNIFIED_ATTN QK-Norm+RoPE+KVCache fusion for the packed KV-cache [BLOCKS, HEADS, BLOCK_SIZE, 2*HEAD_DIM] layout (#49373) 
Signed-off-by: Jack Hu <Jack.Hu@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-07 09:54:55 -07:00
Flora Cui 7e85d3a42c [ROCm] Enable pinned memory on supported WSL2 kernels (#50126) 
Signed-off-by: Flora Cui <flora.cui@amd.com>
 2026-08-07 10:43:42 -05:00
aoshen02Chenglun Huferkans-amiraoshen02Claude Fable 5
 		448344c0e2 [Bugfix] Fix get_open_port() livelock on DP-reserved ports and cover get_open_ports_list (#50965) 
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Chenglun Hu <chenglunhu@gmail.com>
Co-authored-by: ferkans-amir <amir.rezaei@tu-berlin.de>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-07 22:39:33 +08:00
limewardJaeyoun KimClaude Opus 4.8Redwan Khanharishk-mangoboostrkhan055Peiran QinQinPR
 		47228db84c [Bugfix][KV-transfer] MoRIIO: per-layer READ-completion barrier in wait_for_layer_load (#48534) 
Signed-off-by: Edwin Lim <edwin.lim@mangoboost.io>
Signed-off-by: Redwan Khan <redwan.khan@mangoboost.io>
Signed-off-by: harishk-mangoboost <harish.kambhampaty@mangoboost.io>
Signed-off-by: limeward <32970461+edwinlim0919@users.noreply.github.com>
Signed-off-by: QinPR <1905873179@qq.com>
Co-authored-by: Jaeyoun Kim <jaeyoun.kim@mangoboost.io>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Redwan Khan <redwan.khan@mangoboost.io>
Co-authored-by: harishk-mangoboost <harish.kambhampaty@mangoboost.io>
Co-authored-by: rkhan055 <redwankhan055@gmail.com>
Co-authored-by: Peiran Qin <66068739+QinPR@users.noreply.github.com>
Co-authored-by: QinPR <1905873179@qq.com>
 2026-08-07 14:23:33 +00:00
Nicolò Lucchesi d4ecb75ba2 [PD][NixlPush][Bugfix] Fix prefix caching (#48758) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-07 16:21:17 +02:00
haic0 f2bfad9167 [Model] Enable Qwen3.8 for AMD Rocm (#50068) 
Signed-off-by: haic0 <haic0@users.noreply.github.com>
Signed-off-by: haic0 <haichzha@amd.com>
 2026-08-07 21:19:07 +08:00
music-dinoandHarry Mellor a231c5ceac [Bugfix] Skip fetching revision for model when model and weights_model are different (#51260) 
Signed-off-by: Dino Music <Dino.Music@amd.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-07 12:55:50 +00:00
Xiaochang WuandKunshang Ji d5aae2b464 Fix ROCm architecture import on non-ROCm platforms (#51357) 
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-07 05:32:56 -07:00
Julien Debache ae934ba8a5 feat: extended EPLB support for Mistral Large 3 and additional MoE backends (#48355) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-08-07 05:32:52 -07:00
Yan Ma 8d9b52f7c2 [XPU] quick fix online quantization UT break (#51365) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-08-07 10:25:31 +00:00
pmanczakandpmanczak 6b5bec7bed [Misc] Add and enable Triton kernel unit tests on XPU (#45694) 
Signed-off-by: pmanczak <pmanczak@users.noreply.github.com>
Signed-off-by: pmanczak <pawel.manczak@intel.com>
Co-authored-by: pmanczak <pmanczak@users.noreply.github.com>
 2026-08-07 02:53:03 -07:00
Nicolò Lucchesi 5ec47f3e48 [PD][PushConnector] Record last activity of remotes to allow clean up of stale ones (#50234) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-07 09:43:29 +00:00
Sebastian WooandClaude 4f76c8ad9d [Bugfix][Platform] Stop re-initializing NVML on every device-capability check (fixes #50381) (#50393) 
Signed-off-by: seewoo <seewoo@ucsc.edu>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-07 08:30:00 +00:00
Hank_andCyrus Leung b8db7f4abd [Bugfix][Quantization] Fix dynamic INT8 W8A8 MoE config being built as W8A16 (#50833) 
Signed-off-by: Hank <hcc.mayday@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-07 01:21:13 -07:00
Wentao Ye c84789c40b [Refactor] Remove kernel dead code (#51051) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-07 01:20:19 -07:00
jimmy-adamsAndreas KaratzashujiCursor
 		da788334bc Support DeepSeek-V4 AMD Quark NVFP4 with emulation kernel (#47972) 
Signed-off-by: Jimmy Adams <Adam_ji@outlook.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: huji <huidong.ji@amd.com>
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: huji <huidong.ji@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-07 01:19:31 -07:00
stefankoncarevicAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0de0362ea1 [ROCm][CI] Loosen block-FP8 fused MoE test tolerance for large-K shapes (#48847) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-07 01:19:26 -07:00
Guanyi ChenandClaude 58fcaa0baa [Feat][Core] Add disk offloading support to SimpleCPUOffloadConnector (#49644) 
Signed-off-by: Guanyi Chen <939416532@qq.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-07 15:37:47 +08:00
Aaron HaoandClaude Opus 5 21ea5b4fa1 [rl] Stateful Trainer Send: NCCL + Sparse NCCL [3/N] (#50902) 
Signed-off-by: haoaaron <ahao@anyscale.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-07 00:35:59 -07:00
kliuae 43d691ec6b [ROCm][Perf] Kimi-K3 Shard Latent MoE up-projection for ROCm path (#51253) 
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
 2026-08-07 06:11:18 +00:00
Chaojun Zhang b706fd1628 [CI][XPU] Work around intermittent segfault in Intel XPU CI with VLLM_DISABLE_COMPILE_CACHE=1 (#51337) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
 2026-08-07 05:18:25 +00:00
Qianxu Wangandqianxu c810e5ee99 [Bugfix] Fix Mamba all-mode CPU offload boundary alignment (#51100) 
Signed-off-by: qianxu <admin@picasso-lab.com>
Co-authored-by: qianxu <admin@picasso-lab.com>
 2026-08-07 07:37:48 +03:00
Tianyu Guo dd856e48bb [Bugfix][EPD][Model Runner V2] Skip gather mm embeddings for encoder only instance (#51222) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-08-06 20:40:47 -07:00
ChaunceyandNick Hill 72c0d67657 [Feat] Support thinking_token_budget in Model Runner V2 (#46727) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 03:36:18 +00:00
Artem Perevedentsev 5ac2684976 [CI] Re-enable FI autotune in GSM8K config for Qwen3.5-35B-A3B (#51293) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-08-07 02:42:34 +00:00
Kunshang Ji 0406ba22c4 fix pre-commit broken (#51341) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-06 18:17:39 -07:00
Bi TiekaiClaude Opus 5Yongye Zhu
 		e08111211b [Kernel] Support Nvfp4 Cutedsl Moe Swiglu-oai and Relu2(non-gated) Activation (#47106) 
Signed-off-by: Tiekai Bi <tiekaib@nvidia.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-08-06 17:42:45 -07:00