iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
19,560 Commits 3 Branches 0 Tags
cd930c8e786bcfb84397e0aa424d789ebec58433
Commit Graph
5746 Commits
Author SHA1 Message Date
Jacob ZhangMatthew Bonannimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		cd930c8e78 [Bugfix] Fix MLA kv_b_proj activation dtype with Marlin FP8 (#38771) 
Signed-off-by: Jacob Zhang <jaco8123@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-05 07:25:41 -07:00
LucainOpenAI CodexClaude Opus 5Harry Mellor
 		397094da17 Resolve revision to commit_hash once per model load, via huggingface_hub's resolve_revision (#49990) 
Signed-off-by: Wauplin <lucainp@gmail.com>
Signed-off-by: Lucain <lucainp@gmail.com>
Signed-off-by: Lucain Pouget <lucain@huggingface.co>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-05 13:49:56 +00:00
qizixiandClaude Opus 5 4719a9b8f5 [Bugfix][Spec Decode] Fix EAGLE3 DeepSeek draft crash on non-YaRN rope configs (#51092) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-05 14:10:52 +08:00
Wei-Cheng (Wayne) ChiuandOpenAI Codex 7794b1e08b [Bugfix] Skip Qwen3 deepstack buffers without vision (#49397) 
Signed-off-by: WEI CHENG CHIU <waynehacking8@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-05 12:46:23 +08:00
TrainToGPBandClaude eb3dce9757 [Bugfix][Model] Gemma3n/Gemma4: pad variable-length audio batches (#50958) 
Signed-off-by: TrainToGPB <kaiba0514@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-05 12:46:00 +08:00
Jakub ZakrzewskiOpenAI CodexWentao Ye
 		c25093a305 [Kernel] Batch invariant NVFP4 MoE using cutlass (#40372) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-08-05 04:31:32 +00:00
aoshen02andyihengz 6153dbe036 [R3] Unify routed expert shape configuration (#50940) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: yihengz <yihengz@nvidia.com>
 2026-08-04 20:20:41 -07:00
Yejing Laiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2cb3ff88cc [BUGFIX][Quant]Fix test_kv_scale_reload failed (#50405) 
Signed-off-by: Lai, Yejing <yejing.lai@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-05 10:50:54 +08:00
Julian HuangOpenAI CodexTyler Michael Smith
 		05b7876f50 [MoE][Humming] Support SiTU activation for Kimi-K3 (#50510) 
Signed-off-by: 墨楼 <huangzhilin.hzl@antgroup.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-08-04 12:39:18 -07:00
Wentao Ye d31de3c421 [Kimi K3 Perf] option to shard the shared expert for non mega case, 16.98 GiB memory/GPU saved (#50912) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-04 14:00:44 -04:00
Linkunandlkchen 7cab4368f2 [MM][CG] Support ViT full CUDA graph for Kimi-K2.5 (#50929) 
Signed-off-by: lkchen <anthropic@lkchen.net>
Co-authored-by: lkchen <anthropic@lkchen.net>
 2026-08-04 17:15:56 +00:00
amitz-nvandOpenAI Codex 52c0e3cb08 [Kernel] Add support for Flashinfer Mamba SSU algorithm selection (#50157) 
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-04 09:40:23 -07:00
aoshen02andaoshen02 7153fd70f6 [Bugfix][MoE] Filter packed expert weights during EP loading (#49558) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-08-04 16:32:03 +00:00
Anuj BolewarAnuj BolewarHarry Mellor
 		166f4e2dc3 fix: fuse weightless RMSNorms at their declared width (#50867) 
Signed-off-by: Anuj Bolewar <anujbolewar@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Anuj Bolewar <anujbolewar@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-04 16:03:09 +00:00
Jiangyun Zhuandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> f9c74b4b9c [Mamba] enable prefix cache by default (#50991) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-04 15:20:51 +00:00
Nicolò Lucchesi 7b50d2c0bc [Misc] Avoid importing nixl_ep on every vllm serve config (#50879) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-04 07:56:54 -07:00
yimdevOpenAI CodexMatthew Bonanni
 		199644d410 [Bugfix][Attention] Guard sparse MLA masked MHA workspace (#50906) 
Signed-off-by: yimdev <5779256+yimdev@users.noreply.github.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-08-04 09:59:33 -04:00
Michał Ganczarenko 7c40d61eaf [Bugfix] Flatten >2D multimodal embeddings, not just 3D (#50250) 
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
 2026-08-04 13:34:58 +00:00
Andrii SkliarandAndrii Skliar 5789897aa4 [Spec Decode] Add top-k DSpark Markov projection (#49969) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
 2026-08-04 12:52:53 +00:00
Harry MellorandClaude Opus 5 1eb3694521 [Docs] Fix two docs build warnings (#51014) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-04 05:29:04 -07:00
Harry Mellor 59b2fdfc4e Support MLA properly in the Transformers modeling backend (#48250) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-04 13:24:11 +01:00
Lazurite 6a9fdf0dc7 [Bugfix] Resolve seq-cls num_labels from the top-level config for multimodal checkpoints (#50950) 
Signed-off-by: Lazurite <43494437+Rapisurazurite@users.noreply.github.com>
 2026-08-04 11:39:39 +00:00
fxmarty-amdandAndreas Karatzas 385d4c084e [ROCm][AITER] Hotfix for memory access fault errors in AITER triton MOE routing (#50859) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-03 21:07:46 -05:00
zzt 413e70d52c Perf/h20 moe config e256 n512 (#48825) 
Signed-off-by: zzt <zengzetang.zzt@antgroup.com>
 2026-08-03 17:47:58 -07:00
xiaozhoupyOpenAI CodexJee Jee Li
 		c810937573 [Kernel] Extend CuTe DSL skinny GEMM to GLM-5.2 (#49791) 
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-03 16:49:41 -07:00
Ryan HambyandIsotr0py 6a9109d865 [Bugfix] Fix Qwen3-Omni crash on video with no audio track when use_audio_in_video=True (#48420) 
Signed-off-by: RyanJHamby <ryanhamby22@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-03 16:22:00 -07:00
Chris FontesChris FontesChris Fontesmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0b37d8389f fix: NVFP4 quantization out_dtype should match model dtype, not torch default (#48861) 
Signed-off-by: Chris Fontes <2224082+fattchris@users.noreply.github.com>
Co-authored-by: Chris Fontes <chris@fontes.io>
Co-authored-by: Chris Fontes <chris@fontes.dev>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-03 22:47:37 +00:00
skysnow2001andDouglas Lehr f43e1d26e3 [ROCm] Enable AITER and FP8 inference on GFX120x (#43615) 
Signed-off-by: skysnow2001 <skysnow9285@gmail.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-08-03 16:45:37 -05:00
Raphaël Rialland 4a3447d200 [Bugfix][Model Runner V2] Restore multimodal draft capability detection (#50417) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
 2026-08-03 14:18:14 -07:00
aoshen02andNick Hill 42ab184ea7 [MRV2] Enable routed-experts capture (#50721) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-03 21:00:13 +00:00
Taneem IbrahimandWentao Ye 1c0d207915 [Bugfix] Default Gemma3 Model intermediate_tensors to None (#50777) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-08-03 15:07:59 -04:00
Hongxia Yang 8f50685c48 [ROCm][Kimi-K3] aiter moe environment variable cleanup (#50582) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
 2026-08-03 18:54:04 +00:00
wangxiyuan dd11df04f3 [Misc] Remove deprecated calculate_kv_scales runtime KV scale calculation (#49389) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-08-03 11:15:13 -04:00
Wentao Ye 0cf49a5d15 [Refactor] Remove multiple dead codes (#50285) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-03 10:59:08 -04:00
Kyungmin Leeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 9ae11a6b89 [Model] Add K-EXAONE-2.0-750B-A37B (#50524) 
Signed-off-by: lkm2835 <lkm2835@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-03 06:33:02 -07:00
Andrii SkliarAndrii SkliarBenjamin Chislett
 		b977407d8b Support quantized DSpark Markov heads (#50424) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-08-03 05:35:39 -07:00
Li, Jiang c8602c7906 [CPU] Refine CPU kernel dispatch (#50801) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-08-03 17:42:53 +08:00
Jee Jee Li b9d1e2437e K3: Move LatentMoERunner (#50678) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-08-03 09:36:17 +00:00
Jee Jee Li 4635cc3e8f Shard the K3 Latent-MoE up-projection on large batches (#50383) 2026-08-03 00:49:39 -07:00
Li, JiangClaude Opus 5Fadi Arafeh
 		0a6446005d [CPU] Migrate unquantized MoE to the modular-kernel experts structure (#50133) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-08-03 14:55:04 +08:00
zofiaandKunshang Ji e481da9508 [XPU] [Linear] add torch as xpu linear backend (#49664) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-03 14:30:21 +08:00
Qiming Zhang 2755489a26 [INC] fix w4a4 model (#50807) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
 2026-08-03 14:28:58 +08:00
Omkar Shewale 9a4fd57cac [Model] Support jina-embeddings-v5-text-nano (EuroBERT encoder backbone) (#50688) 
Signed-off-by: Omkar Shewale <omkarshewale2001@gmail.com>
 2026-08-03 01:37:08 +00:00
Rafli Surya Wijayaandloulanyue 0601850791 [Bugfix][Models] Accept Qwen3_5MoeTextConfig in Qwen3_5MoeProcessingInfo for transformers 5.x compatibility (#50704) 
Signed-off-by: loulanyue <loulanyue@users.noreply.github.com>
Co-authored-by: loulanyue <loulanyue@users.noreply.github.com>
 2026-08-02 05:50:40 +00:00
Isotr0pyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> e2fa28594f [1/N] Unify multiple-path encoder cuda graph support (#49934) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-02 10:33:23 +08:00
Ning Xie 03c782eb91 [model registry] some simple typos (#50673) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-08-01 15:37:29 +00:00
Michael GoinOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		fcdc7c2e9c [CI] Organize speculative decoding E2E tests by coverage (#50330) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-31 18:05:42 -07:00
Mike GMichael Goinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e3be89673d Enable ModelOpt FP8 emulation on SM80 (#50019) 
Signed-off-by: Mike G <180722391+mikekg@users.noreply.github.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-31 14:48:27 -07:00
bnellnmClaudeTaneem IbrahimAgata DobrzyniewiczTushar JainTushar JainWentao YeJiangyun Zhu
 		454ea5b526 [MoE Refactor] Combine CompressedTensorsWNA16MarlinMoEMethod with CompressedTensorsWNA16MoEMethod (#44570) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Dobrzyniewicz, Agata <agata.dobrzyniewicz@intel.com>
Signed-off-by: Tushar Jain <tushar00jain@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Agata Dobrzyniewicz <160237065+adobrzyn@users.noreply.github.com>
Co-authored-by: Tushar Jain <8455015+tushar00jain@users.noreply.github.com>
Co-authored-by: Tushar Jain <tushar00jain@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-07-31 14:13:03 -07:00
Michael GoinandOpenAI Codex 90329913e9 [UX] Reduce startup log noise (#50590) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-31 13:04:34 -07:00