iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,003 Commits 3 Branches 0 Tags
1b2c591cd0c3bb5a85ac7f3d6cbaa2fa7df6bc7d
Commit Graph
21003 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
sychen52andCyrus Leung 1b2c591cd0 [Bugfix] speedup nvfp4 kv for FMHA (#55031) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-08 17:25:42 +00:00
Raya Elena Solanoandmgoin bcca76e7db [Test] Dequantize NVFP4 KV cache scales in the layout the kernel writes (#55908) 
Signed-off-by: Raya Elena Solano <raya.solano@mbinf.de>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-08 17:13:39 +00:00
Jiangyun ZhukhluujiahaoKimi Code
 		8ebc5b0a18 [Bugfix] Fix NVFP4 fused SiLU+mul scale allocation and global scale direction (#55643) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: jiahao <jxia77@terpmail.umd.edu>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-08 08:26:07 -07:00
Wentao Ye 6b15bea080 [Refactor] Remove utils dead code (#53941) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-08 11:22:09 -04:00
Qiu Chunshuo 7c2f1ff495 [Core] Scope PCP-DP validation to GPU manager (#54523) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
 2026-09-08 08:18:01 -07:00
Canlin GuoandOpenAI Codex 8e1f97e709 [Qwen3.8-Flash-Next] Tune FP8 TP2/TP4 Triton MoE on B200 (#55890) 
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-08 22:45:18 +08:00
linitra24 b105d8284f [LoRA] Clarify target module matching logic (#55865) 
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
 2026-09-08 07:30:15 -07:00
Wentao Ye 07950d4734 [Kimi Bug] Fix kimi k3 startup cuda graph issue with recoverSSM (#55774) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-08 10:29:53 -04:00
Roberto L. CastroandCodex 6ddbab03de [4/N][warmup][DSv4] Migrate common attention kernels (#50176) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-08 10:25:48 -04:00
Sherif WalyandNicolò Lucchesi f998862d46 [Bugfix] Fix Kimi K3 loading with interleaved weight streams (#53379) 
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
 2026-09-08 16:08:51 +02:00
Mohit SinglaandCodex 613ab20f7f [Model] Enable torch.compile for Sarvam MLA (#55817) 
Signed-off-by: mohit-sarvam <mohit@sarvam.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-08 22:01:30 +08:00
akshaverwzhao18Claude Opus 5roikoren755
 		263c4ff95f [Bugfix][Spec Decode] Cache the Mamba state at the block-grid position of EAGLE resume (#53945) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Adam Shaver <ashaver@nvidia.com>
Signed-off-by: akshaver <168006157+akshaver@users.noreply.github.com>
Co-authored-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: roikoren755 <26850796+roikoren755@users.noreply.github.com>
 2026-09-08 09:59:58 -04:00
Thomas ParnellClaudeHarry Mellor
 		414057a3d3 [Bugfix] Apply attention sinks in the Transformers backend (#52156) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-08 13:54:07 +00:00
SunCodexIsotr0py
 		472d8c9125 [Perf] Fuse DeepEncoder relative bias in Triton attention (#55629) 
Signed-off-by: levius <2114377220@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-08 13:22:04 +00:00
Andy LoandOpenAI Codex 5ebce2391e [Bugfix][KV Offload] Respect prefix-cache bypass in SimpleCPUOffload (#54998) 
Signed-off-by: Andy Lo <andy@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-08 13:14:19 +00:00
Stefano Castagnetta f6326f53bd [Perf][GDN] Enable the FlashInfer GDN prefill kernel on SM12x (#55715) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-09-08 12:26:03 +00:00
Shanshan Shen 25047604fe [ROCm][Perf] Remove AITER paged-MQA outputs guard for DeepSeek-V4 (#55808) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-09-08 06:58:26 -05:00
Kevin H. Luu 34b9899c8f [CI] Increase ColQwen3 pooling test memory budget on H200 MIG (#55878) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-09-08 11:39:01 +00:00
Stefano Castagnetta 13cf9e05c1 [Perf][Quant][NVFP4] Prefer W4A4 linear kernels over weight-only ones on SM120/121 (#55170) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-09-08 15:06:45 +04:00
jimmy-adams e41a17e606 [ROCm][Quantization] Support AMD Quark per-block FP8 for fused MoE layers (#52263) 
Signed-off-by: huji <huidong.ji@amd.com>
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
 2026-09-08 10:30:26 +00:00
Kevin H. LuuandCodex 5db652225f [Bugfix] Validate extension integers before engine serialization (#55606) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-08 16:30:06 +08:00
Kevin H. LuuandCodex 96eccb8f49 [CI] Restore clean GPU state before OAI Triton MoE tests (#55453) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-08 01:15:21 -07:00
Taoyu ZhuandShanshan Shen ce6c241ddc [ROCm][Perf][M3] Fused allreduce+GemmaRMSNorm fast path (#54787) 
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-08 07:21:49 +00:00
ahmed xijiaat 81dabe2e5d [Bugfix][KV Offload] Validate SWA coverage at unaligned cache-hit boundaries (#55712) 
Signed-off-by: xijiade.aihemaiti <3146335281@qq.com>
 2026-09-08 10:15:48 +03:00
Adababyandshaolila 5133e1d285 [Frontend] Migrate input-validation errors to VLLMValidationError (harmony/mistral/chat_utils/params) (#55735) 
Signed-off-by: shaolila <shaolila@buaa.edu.cn>
Co-authored-by: shaolila <shaolila@buaa.edu.cn>
 2026-09-08 06:45:40 +00:00
aoshen02 755541443f [Bugfix][ROCm] Mask paged attention V cache padding (#53856) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-09-08 14:39:23 +08:00
Subhash Polisetti 782f36cd0c [Bugfix][InternVL] Stop the video parser consuming image_embeds (#55779) 
Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
 2026-09-08 05:19:41 +00:00
Dev OskiiCursorCyrus Leung
 		6c73b08dec [Bugfix] reject string file on translation like transcription (#55618) 
Signed-off-by: oskii <oscarchambersweb@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-08 05:02:53 +00:00
weihan 9ea8f3ffc3 [Bugfix] Honor STEP token pooling in DispatchPooler.for_seq_cls (#55307) 
Signed-off-by: weihan <130119942+krsish@users.noreply.github.com>
 2026-09-08 04:13:09 +00:00
mingg26Kimi Codemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7fa2c63796 [Bugfix] Autotune FlashInfer deferred MoE decode kernels before CUDA graph capture (#55377) 
Signed-off-by: ming <ming@inferact.ai>
Co-authored-by: Kimi Code <kimi-code@moonshot.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-08 04:02:23 +00:00
He LiuandKimi eb6b619ab2 [Bugfix][KV Offload] Register mixed page sizes in one cache group (#54756) 
Signed-off-by: heliubj18 <chengfeng@ppio.com>
Co-authored-by: Kimi <noreply@moonshot.cn>
 2026-09-07 20:50:15 -07:00
Jee Jee LiandYongye Zhu 9c297e3b8b [Kernel] PDL enablement for fusedQKNormRopeKernel (#55755) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-09-07 20:49:03 -07:00
Jee Jee Li 607a6e48c5 [CI/Build] Fix unused fake implementation testing (#55797) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-09-08 03:36:18 +00:00
Wei Zhao a69402aaa8 [Perf] Integrate FlashInfer KDA kernels (#55364) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-09-07 20:29:30 -07:00
leo 12579c7f01 [Kernel][Perf] Tune H20 block-FP8 MoE low-batch configs (+21%) (#54668) 
Signed-off-by: liuyao0322 <yaoliu548926@gmail.com>
 2026-09-08 03:20:21 +00:00
Wentao Ye 869f78732b [Kimi Bug] Fix kimi k3 AssertionError assert 0 <= checkpoint_idx < len(blocks) (#55747) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-07 22:34:48 -04:00
Andreas Karatzas 6b5a12c0f8 [ROCm][CI] Enable HY-V4 model initialization on ROCm (#54405) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-09-07 20:55:44 -05:00
Artur FierkaandClaude 9dbdf8e9af [Bugfix][Quantization][MoE] Normalise an unset group_size on the compressed-tensors WNA16 MoE path (#53163) 
Signed-off-by: Artur Fierka <artur.fierka@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-08 09:42:46 +08:00
Woosuk KwonandOpenAI Codex 06334fae2d [Build] Remove obsolete TPU Dockerfile (#55376) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-07 18:07:27 -07:00
Mohit SinglaCursorCodex
 		54da70c1eb [Feature] Support EAGLE3 for Sarvam (#53052) 
Signed-off-by: mohit-sarvam <mohit@sarvam.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-08 00:08:35 +00:00
Taneem Ibrahim fdfa0a1659 [Pooling] Honor request_id from request bodies (#55665) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-07 19:55:20 -04:00
Taneem Ibrahim 252ed87621 [Bugfix] Restore Responses validation error boundary (#55761) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-07 22:12:18 +00:00
Kevin H. LuuandCodex 537af2c3a4 [CI] Recover empty multi-node Docker networks and finish partial cleanup (#55454) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-07 12:53:13 -07:00
Kevin H. LuuandCodex 58f09211dd [CI] Synchronize shared offload unlink test before observing pathname (#55604) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-07 12:52:36 -07:00
Sahil Patel 51da0ca66c [CI/Build] Unskip ColQwen3 multimodal pooling tests on Transformers v5 (#55588) 
Signed-off-by: Sahil <sip4818@gmail.com>
 2026-09-07 15:15:39 +00:00
Zeyu Yang ad2f18e8af [CI] reduce npu CI use time and add timeout (#55731) 
Signed-off-by: yangzeyu <yangzeyu7@huawei.com>
 2026-09-07 22:51:09 +08:00
am-cohereCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		70584f69b1 [Model] Add Cohere Compass model (#54774) 
Signed-off-by: am-cohere <312513976+am-cohere@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-07 07:36:49 -07:00
AbhiandAntigravity 7cc89a7dda [Tests] Update SarvamMLA transformers v5 compatibility reason to hf (#55728) 
Signed-off-by: Abhi <108084481+Aj2280@users.noreply.github.com>
Co-authored-by: Antigravity <antigravity@google.com>
 2026-09-07 14:27:07 +00:00
Nicolò Lucchesi cd64c2dea9 [Docs] Clarify admission control limits apply server-wide, not per DP rank (#55124) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-09-07 14:08:48 +00:00
Asaf Gardinminyichenaoshen02
 		f2d45f26bd [Bugfix][Gemma] Conditionally create KV projections/norms on KV-shared layers (#54917) 
Signed-off-by: Josephasafg <ajgard7@gmail.com>
Co-authored-by: minyichen <96753147+cyc00518@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-07 15:04:00 +01:00