iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
22,991 Commits 3 Branches 0 Tags
c19c263ac8ea3fd6e1d2d689f0e4e725855d89cf
Commit Graph
509 Commits
Author SHA1 Message Date
Andrii SkliarAndrii SkliarLuke Alonso
 		6849ca696a [Quantization] Add ModelOpt IQ2_XS, IQ2_XXS, and Q8_0 support via b12x (#59604) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Luke Alonso <lalonso@gmail.com>
 2026-10-13 09:06:33 -07:00
Yongye ZhuandClaude Opus 5.5 a6009ea85b [Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer (#61149) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-12 23:24:43 -07:00
haotiaa 6e6e86078a [Bugfix][Platform] Don't resolve current_platform during import vllm (#60766) 
Signed-off-by: Haotian An <haotiaa@amazon.com>
 2026-10-12 17:05:34 +00:00
Misha GoinandClaude Opus 5.5 29e4d2d42e [Frontend][Build] Add vllm download-kernels for FlashInfer precompiled kernels (#58765) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 21:22:35 -07:00
Dakai AnYifan QiaoOpenAI CodexOpenAI CodexLucas WilkinsonClaude Opus 5.5Lucas Wilkinson
 		02e55d2af3 [KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-09 15:48:37 -04:00
Misha GoinandClaude 3766a67a32 [WideEP] Explain how to fix a too-old NCCL for DeepEP v2 (#60449) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-09 12:34:58 -07:00
Haokai MaandClaude 0751f5272f [Bugfix] Scope deprecated CLI argument warnings to the parser that defines them (#59917) 
Signed-off-by: Tyler Ma <158780265+TylerMa-debugg@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-08 15:29:04 +00:00
Woosuk Kwon 194bd72be7 [Compile] Remove torch.compile-based sequence parallelism and AsyncTP (#60517) 2026-10-08 08:28:23 -07:00
Muhammad Usman MateenandMisha Goin e73895a5df [Bugfix] Make suppress_stdout redirect fd 1 instead of sys.stdout.fileno() (#59336) 
Signed-off-by: usmanmateen <usmanmateen255@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 09:52:50 +00:00
Harry MellorandClaude Opus 5.5 21d9758459 Remove Python 3.10 support now that it is EOL (#60402) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 02:30:52 -07:00
Bugen ZhaoOpenAI CodexClaude Opus 5.5
 		f226af0e11 [Bugfix] Eliminate frontend ZMQ port TOCTOU with inherited listeners (#54113) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 06:25:27 +00:00
Jiahan Chang (Cyrus) 816173ca9d [Kernel] Remove WAR for LL CuTeDSL BF16 GEMM (#60434) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
 2026-10-08 05:38:07 +00:00
Isotr0pyandShanshan Shen 8352b2427f [Multimodal] Optimize encoder cudagraph padding logic (#58925) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-10-08 03:38:09 +00:00
Nicolò Lucchesi a58bdd0d4e [Perf] JIT monitor: hook TileLang JITImpl.compile instead of __call__ (#58685) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-10-08 00:11:59 +00:00
aditi-amd 385d86a6cf [Attention] Add UltraQuant 4-bit KV cache backend (FlyDSL D=256) (#57057) 
Signed-off-by: Aditi Rana <aditi.rana@amd.com>
 2026-10-06 20:42:52 +00:00
aoshen02andClaude Opus 5.5 d09ff77637 [Bugfix] Give each data-parallel engine its own global RNG streams (#59788) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 17:38:18 +00:00
Julien Debache 111f71a6db [feat] FlashInfer CuteDSL MegaMoE integration (#54049) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-10-02 15:43:33 -04:00
Turner JabbourandClaude Sonnet 5 e2cb5ee46c [CI] Replace shellcheck-suppressed patterns flagged in #52572 with clean equivalents (#53341) 
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-10-02 11:30:04 +00:00
Guanxin Liaoshen524Claude Opus 5.5aoshen02
 		a810316c08 [Bugfix] Keep batch-invariance NCCL pins out of the weight-transfer group (#59500) 
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-02 11:19:28 +08:00
Duncan MossRoger Wangmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Thien Tran
 		766b4ae54c [Kimi-K3] Add FlashInfer speculative KDA backend (#54255) 
Signed-off-by: Duncan Moss <dmoss@nvidia.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-01 02:28:50 +00:00
Ning Xie 29773c722b [Observability] log sub process killing in process manager force kill (#52314) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-09-30 06:13:24 +00:00
Mia Ojeda e006d761a5 [Bugfix][CPU][DiffusionGemma] Support narrower canvas w/ sync scheduling (#59107) 
Signed-off-by: ojeda-e <estefania.b.ojeda@gmail.com>
 2026-09-29 23:28:50 -04:00
Tyler Michael SmithandCodex 50239dfc4d [WideEP] Change DeepEPv2 to auto select hybrid mode by default (#57991) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-29 18:36:30 -04:00
Bowen WangandClaude Opus 5.5 1fde8ff7da [Bugfix] Detect the CUDA toolkit the way FlashInfer does in has_flashinfer() (#58360) 
Signed-off-by: Bowen Wang <bowen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 04:14:49 +00:00
Chaojun Zhang 7230dfea50 [XPU] Preserve non-contiguous strides when pinning CPU tensors for UVA view (#54874) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
 2026-09-29 09:43:07 +08:00
Shanshan Shen 44af287ebe [ROCm][Perf] Enable layer-aware CSA2 multi-stream overlap for DeepSeek-V4.1-Flash (#57407) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-09-27 23:48:01 +08:00
Mark McLoughlin a44d7b5151 [Core][Logging] Fix JSON logging process decoration (#57957) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-09-25 10:12:28 -06:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
wangxiyuan 44257c2e35 [Feature] Triton kernel dispatcher (#43048) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-09-25 04:07:43 +00:00
Matt MastracciClaude Fable 5.1Lucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Misha Goin
 		ab3de6edf2 [Perf] DiffusionGemma: constrained reads over the request's logprob_token_ids (#58216) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-24 13:15:08 -07:00
Matt MastracciClaude Fable 5.1Lucas WilkinsonCodexRazorback16
 		1b3b88ec2b [Core] structured generation mode for DiffusionGemma model (Jev-like) (#57250) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Razorback16 <razorback16@protonmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Razorback16 <razorback16@protonmail.com>
 2026-09-22 20:51:17 +00:00
Tony Lin c42f5285ec [Bugfix] prioritize architecture capability before DeepGEMM availability check (#58073) 
Signed-off-by: Tony Lin <tony.lin@intel.com>
 2026-09-22 11:38:04 -04:00
Nils MattesonTaneem IbrahimelehayymAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		13058faa49 [Frontend] Add reusable TP1 initialized-engine snapshots (#51360) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-21 23:18:49 +00:00
Jinzhen LinCodexmgoin
 		7d99c2c4fd [Feature][Humming] Humming feature integration (#56685) 
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-20 08:11:01 -07:00
Kevin Liandkevinli 911548247a [Distributed][MoonEP] BF16 integration of MoonEP balanced EP backend (#52101) 
Signed-off-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com>
Co-authored-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com>
 2026-09-18 10:34:01 -04:00
Thien TranOpenAI Codexzjy0516
 		41f9104fa6 [DSv4.1] Integrate Mega-Gate from DeepGEMM (#56266) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-09-17 12:06:36 -07:00
Harry MellorandClaude Opus 5 0ef8366ba3 [Frontend] Only show the summary line of config docstrings in --help (#57357) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-17 12:53:11 +00:00
Tianmu LiOpenAI CodexLi, Jiang <jiang1.li@intel.com>
 		23cd0346ce [CPU] Align scheduler and NIXL CPU affinity per local rank (#53636) 
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-17 16:43:08 +08:00
Jiahan Chang (Cyrus)andOpenAI Codex 4e5ffda19d [Bugfix] Isolate supplemental FlashInfer BF16 autotuning (#57285) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-17 07:38:58 +00:00
Rohan PotdarClaude Opus 4.8Matt
 		3e267bae70 [ROCm] triton+triton_kernels 3.8 mxfp4 MoE support (gpt-oss + DeepSeek-V4) (#55934) 
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Matt <156021403+mawong-amd@users.noreply.github.com>
 2026-09-16 20:03:34 -05:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Harry MellorandClaude Opus 5 5f203baedb [Docs] Split slash-combined docstring parameters (#57141) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 02:16:32 -07:00
AlexHuangOpenAI CodexItay Etelis
 		0f8fa53acf [Bugfix][KV Offload] Check cgroup memory before SHM allocation (#54014) 
Signed-off-by: Alex <jihui.huang@daocloud.io>
Signed-off-by: AlexHuang <alex.tech.lab@outlook.com>
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
 2026-09-16 09:17:50 +03:00
Kevin H. LuuSherlockmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		af1c01499b [Bugfix] Make GPU sync checks safe under torch.compile (#56904) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Sherlock <sherlock@raft.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-15 21:37:08 -07:00
Jared WenandClaude Fable 5.1 1257512305 [DSA] Wire DeepGEMM sparse MQA logits into the DeepSeek V4.1 indexer (#56254) 
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-15 14:07:14 -07:00
Nick Hill f2aad6aa70 [MRV2] Buffer util simplifications (#56888) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-15 20:36:51 +00:00
Juntian LiuYongye ZhuClaude Opus 5Thien Tran
 		bb55077411 [Perf][Kernel] Integrate Mega-mHC from DeepGEMM for DeepSeek V4.1 (reopen of #56255) (#56962) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-09-15 04:01:06 -07:00
Yongye ZhuandClaude Opus 5 8263ea12bd [Build] Move DeepGEMM pin to the vLLM fork and update the Mega MoE call convention (#56876) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-15 00:57:18 -07:00
zhaoguochun1995 bbbd0a02c9 [Bugfix] Carry over queued work when materializing the dedicated stream (#56382) 
Signed-off-by: zhaoguochun1995 <zhaoguochun1995@163.com>
 2026-09-14 23:52:20 +00:00