iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,845 Commits 3 Branches 0 Tags
78300cdabffbaaffe7625ed34412ec2aec75535c
Commit Graph
7686 Commits
Author SHA1 Message Date
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> df71917cf1 [DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. (#49236) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-31 08:45:58 -07:00
Matthew BonanniandOpenAI Codex 82ae4164ee [2/N][Attention] Enable masked MHA for sparse MLA prefills (#48770) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-31 06:01:24 -07:00
Nicolò Lucchesi 864a87febb [Kernel][CI] --jit-monitor-mode error e2e tests for kernel warmup infra (#50109) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-07-31 08:53:16 -04:00
Oğuzhan KIR 7fdc1ab29e [CI] Remove default_torch_num_threads workaround from llava-onevision-transformers test (#50560) 
Signed-off-by: oguz <oguzhankir17@gmail.com>
 2026-07-31 12:29:31 +00:00
Recoordinateandlatent-9 0b5b49dde8 [Bugfix][Frontend] Raise VLLMValidationError for user-facing errors in chat_utils.py (#50491) 
Signed-off-by: latent-9 <296084221+latent-9@users.noreply.github.com>
Co-authored-by: latent-9 <296084221+latent-9@users.noreply.github.com>
 2026-07-31 11:54:43 +00:00
Itay EtelisItay EtelisOr OzeriItay Etelis
 		2c4d348848 [KV Connector] Add per-layer canonical KV page mappings for parallelism-agnostic offload (#48408) 
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Itay Etelis <Itay.etelis@gmail.com>
 2026-07-31 14:42:54 +03:00
Nicolò Lucchesi 17beffd55f [Misc] Clarify mono audio requirement (#50141) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-07-31 10:46:53 +00:00
bnellnmandHarry Mellor 6e311c6e20 [MoE Refactor] Rename FusedMoE to FusedMoEFactory (#44941) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-31 11:28:18 +01:00
Andreas Karatzas c911120a6f [ROCm][CI] Update Transformers AR+RMS fusion expectation (#50517) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-07-31 11:16:52 +01:00
Chaojun ZhangandKunshang Ji 482cfc20f0 [XPU] Unify XPU RMSNorm kernels with vllm_c and drop redundant XPU-specific implementation (#46981) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-07-31 17:28:18 +08:00
Qiming Zhang 5d7647a109 [UT] add skipif for rocm aiter sampler UT (#50530) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
 2026-07-31 16:32:49 +08:00
Thien TranandCodex 34bb795ff3 [CI] Add M3 MSA tests to CI (#49143) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <noreply@openai.com>
 2026-07-31 01:01:48 -07:00
Taneem IbrahimOpenAI CodexIsotr0py
 		f727951d3f [Bugfix] Re-land MiniMax M3 default video processor (#50305) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-07-31 07:46:18 +00:00
Fadi Arafehjiang1.liClaude
 		10e6b40015 [CPU][BugFix] Remove redundant kv cache write (#50437) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-31 15:28:44 +08:00
Isotr0py 1180b604f8 [Multimodal] Expose mm hash algothrim selection to cli args (#49686) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-07-31 05:25:19 +00:00
Taneem Ibrahim 0f17394564 [Model Runner V2] Enable encoder token classification (#50293) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-07-31 04:15:36 +00:00
Sebastian WooandClaude bebf918044 [Bugfix][Model] Reject encoder-backbone jina-embeddings-v5 checkpoints with a clear error (fixes #50337) (#50352) 
Signed-off-by: seewoo <seewoo@ucsc.edu>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-31 11:43:31 +08:00
stefankoncarevicandAndreas Karatzas 2773ec31b3 [ROCm][CI] Use explicit wvSplitKrc skinny-GEMM test tolerance for bf16 (gfx950) (#49309) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 22:25:42 -05:00
Chang GuoandCursor Agent 541128bdeb [KV Offload] Enable single-copy MLA layout for CPUOffloadingSpec (#50301) 
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: Cursor Agent <cursor-agent@cursor.com>
 2026-07-31 06:20:47 +03:00
music-dinoandAndreas Karatzas 5d5f22ed7a [ROCm][CI] Use larger atol value for INT3 in test_quick_all_reduce.py (#50450) 
Signed-off-by: Dino Music <Dino.Music@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 21:41:44 -05:00
MattandAndreas Karatzas d91f7af7d0 [Hardware][AMD][Kernel][CI][Bugfix] Fix ROCm DeepEP FP8 max (#50467) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 21:37:36 -05:00
Bugen Zhao ab98034d4c [Frontend][Bugfix] Use default tool call IDs for Kimi K3 for conversation-level uniqueness (#50420) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-31 10:02:17 +08:00
chaeminlim-mbEdwin LimlimewardQinPRClaude Opus 4.8
 		f1899b2ffd [Bugfix][ROCm] AITER MLA: size MTP verification decode metadata for real qlen/dtype (#45227) 
Signed-off-by: Edwin Lim <edwin.lim@mangoboost.io>
Signed-off-by: Chaemin Lim <chaemin.lim@mangoboost.io>
Signed-off-by: QinPR <1905873179@qq.com>
Co-authored-by: Edwin Lim <edwin.lim@mangoboost.io>
Co-authored-by: limeward <32970461+edwinlim0919@users.noreply.github.com>
Co-authored-by: QinPR <1905873179@qq.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-30 20:50:34 -05:00
+1
Juan VillamizarDouglas LehrClaude Opus 4.7DanielJaden MathiasjpvillamrootDouglas LehrdllehrDaniel ChanAndreas Karatzas
 		4f1da84eb5 Enable gfx1250 ROCm architecture (#46516) 
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com>
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com>
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: dllehr <douglehr@amd.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-31 01:41:12 +00:00
Andreas Karatzas 553fcb82d5 [CI] Retry Hugging Face processor loading (#49908) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-31 08:48:08 +08:00
Wentao Ye 0bff0ce5d3 [Kimi K3 Bug] Fix deepgemm support for kimi k3 (#50458) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-30 20:10:16 -04:00
Nicolò Lucchesi 8700f86a70 [CI] Fix tests/entrypoints/multimodal/openai/chat_completion/test_audio.py::test_chat_streaming_audio (#50451) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-07-30 16:18:53 -07:00
Ryan ClarkandClaude Fable 5 68fb303a41 [Bugfix] Preserve Marlin runtime tensor storage across weight reload (#48438) 
Signed-off-by: Ryan Clark <ryanclark2k@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-30 14:50:03 -07:00
Shangdi YuandOpenAI Codex 45b60e3914 [Kernel][Helion] Disable unsafe B200 RMS reduction warp specialization (#50345) 
Signed-off-by: Shangdi Yu <shangdiy@meta.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-30 20:37:52 +00:00
Michael Goin 0eec856cc3 Add Humming indexed-MoE regression test (#50468) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-30 16:26:24 -04:00
Netanel HaberandOpenAI Codex 70bd10930b [Quantization] Honor --linear-backend for ModelOpt W4A16 (#50273) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-30 13:19:56 -07:00
Richard ZouandCodex c27b080d26 [compile] Fix fake kernel return dtype (#50444) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-07-30 19:50:02 +00:00
Andreas Karatzas bdc98bf506 [CI] Initialize fused gated RMSNorm weights (#50377) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 15:34:08 -04:00
Andreas KaratzasandOpenAI Codex 30e333ca5f [Bugfix] Shut down private Tensorizer engines (#49840) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-30 12:33:22 -07:00
Bugen Zhao 629a938a92 [Frontend] Preserve bare Inkling text in Python and Rust parsers (#50403) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-30 12:25:10 -07:00
Wentao Ye 837eae6458 [DSv4 Perf] Remove redundant full kernel for dsv4, 1.88x kernel performance improvement (#50298) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-30 11:39:07 -04:00
Kunshang Ji f388dd6592 [XPU][CI] skip kimi-k3 test (#50447) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-07-30 08:27:28 -07:00
Andreas Karatzas 5b95890742 [FlexAttention] Avoid encoder block-mask compile explosion (#50339) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 11:24:13 -04:00
Aaron Hao 30b4e7f479 [rl] Stateful Trainer Send: IPC [2/N] (#48981) 
Signed-off-by: haoaaron <ahao@anyscale.com>
 2026-07-30 21:59:08 +08:00
BadrBasowid 59e831c09a [Compilation]Fuse Transformers Residual Add + RMSNorm (#48757) 
Signed-off-by: BadrBasowid <badr.basowid@gmail.com>
 2026-07-30 13:46:36 +00:00
Julien Denize 1a20d23dab [PARSER][Mistral] unified engine-based parser for reasoning and tool calls (#48947) 2026-07-30 15:38:00 +02:00
sroberts-amdElvir CrncevicAndreas Karatzas
 		e2efe79695 [ROCm]Migrating Deepseek V3.2 to vllm/models/deepseek_v32/ (#47207) 
Signed-off-by: sroberts-amd <sroberts@amd.com>
Signed-off-by: Stacy Roberts <sroberts@amd.com>
Co-authored-by: Elvir Crncevic <elvircrn@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 08:37:43 -05:00
Martin HickeyandHarry Mellor 38a267cdd5 [MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent (#49570) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-30 05:02:51 -07:00
Li, JiangandClaude 072a472782 [CPU] Bump up CPU kernels to latest version (#50387) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-30 19:18:43 +08:00
+13
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodexWei Zhao
 		aeeb36b1f1 [New model] Kimi K3 (#50000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-07-30 03:49:58 -07:00
Andreas KaratzasandOpenAI Codex 0c64be8873 [Test][ROCm] Account for gfx950 FP8 RMSNorm rounding (#49839) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-30 18:46:37 +08:00
Andreas Karatzas 61c1d098e5 [CI] Stabilize speculator memory teardown (#50284) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-07-30 18:42:55 +08:00
Andreas Karatzas 165ed33327 [CI][ROCm] Stabilize LLM GC teardown check (#50340) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-07-30 18:42:03 +08:00
Andreas Karatzas e04a30a77c [Frontend] Lazily initialize chat media connectors (#49914) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 14:56:37 +08:00
yinfengLiu f1e8fd27ae [ROCm] Add AITER FP8 ViT encoder attention (#49937) 
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
 2026-07-30 06:22:10 +00:00