iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,453 Commits 3 Branches 0 Tags
6ca2b23e22aab8534e00a85e8ec5222508a6ecf3
Commit Graph
468 Commits
Author SHA1 Message Date
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Harry MellorandClaude Opus 5 5f203baedb [Docs] Split slash-combined docstring parameters (#57141) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 02:16:32 -07:00
AlexHuangOpenAI CodexItay Etelis
 		0f8fa53acf [Bugfix][KV Offload] Check cgroup memory before SHM allocation (#54014) 
Signed-off-by: Alex <jihui.huang@daocloud.io>
Signed-off-by: AlexHuang <alex.tech.lab@outlook.com>
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
 2026-09-16 09:17:50 +03:00
Kevin H. LuuSherlockmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		af1c01499b [Bugfix] Make GPU sync checks safe under torch.compile (#56904) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Sherlock <sherlock@raft.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-15 21:37:08 -07:00
Jared WenandClaude Fable 5.1 1257512305 [DSA] Wire DeepGEMM sparse MQA logits into the DeepSeek V4.1 indexer (#56254) 
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-15 14:07:14 -07:00
Nick Hill f2aad6aa70 [MRV2] Buffer util simplifications (#56888) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-15 20:36:51 +00:00
Juntian LiuYongye ZhuClaude Opus 5Thien Tran
 		bb55077411 [Perf][Kernel] Integrate Mega-mHC from DeepGEMM for DeepSeek V4.1 (reopen of #56255) (#56962) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-09-15 04:01:06 -07:00
Yongye ZhuandClaude Opus 5 8263ea12bd [Build] Move DeepGEMM pin to the vLLM fork and update the Mega MoE call convention (#56876) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-15 00:57:18 -07:00
zhaoguochun1995 bbbd0a02c9 [Bugfix] Carry over queued work when materializing the dedicated stream (#56382) 
Signed-off-by: zhaoguochun1995 <zhaoguochun1995@163.com>
 2026-09-14 23:52:20 +00:00
siyuandIsotr0py 73fb19151f [Fast Start] Support fp4 (#55465) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-10 13:22:17 +00:00
Tianyu Guo 479611c0c6 [Frontend][EPD] Use JSON arrays for multimodal metadata (#56090) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-09-10 05:04:14 +00:00
Nick Hill 4aadb0f14d [Core] Enhance cpu<->gpu sync checking to include paged async copies (#53491) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-08 17:26:21 -07:00
Wentao Ye 6b15bea080 [Refactor] Remove utils dead code (#53941) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-08 11:22:09 -04:00
mingg26Kimi Codemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7fa2c63796 [Bugfix] Autotune FlashInfer deferred MoE decode kernels before CUDA graph capture (#55377) 
Signed-off-by: ming <ming@inferact.ai>
Co-authored-by: Kimi Code <kimi-code@moonshot.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-08 04:02:23 +00:00
Wei Zhao a69402aaa8 [Perf] Integrate FlashInfer KDA kernels (#55364) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-09-07 20:29:30 -07:00
Jee Jee Li 199cb9b964 [Kernel] Remove unused fake implementation (#55535) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-09-06 17:54:32 -07:00
Nick Hill 8277c42e4c [Perf] Ensure async h2d copies are pinned in more places (#55202) 2026-09-04 17:17:01 -07:00
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
Yusuke OdaandChauncey 798b557e06 [Frontend] Add site-packages support for reasoning/tool parser plugins (#45241) 
Signed-off-by: odashi <yusuke.oda@predicate.jp>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-02 07:41:13 +00:00
fxmarty-amdandAndreas Karatzas d1c15e589d [CI] Speed up quantization test group (#53291) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-01 13:15:24 -05:00
Luke AlonsoandLucas Wilkinson 0d4ad47981 [Kernel] Add B12X causal paged attention backend (#52017) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-09-01 10:42:02 -04:00
lzhan011andCyrus Leung ce7391712b [Bugfix][Security] Bound embedding densification before to_dense() (#54632) 
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-01 08:45:14 +00:00
sychen52 7c5dc571cb [Attention][DSA] Enable W4A16 DSA (#51724) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
 2026-09-01 12:37:47 +08:00
Lain 882ca8d696 [Kernel] add Flashinfer cutedsl w4a16 linear (#53014) 
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
 2026-09-01 02:49:14 +00:00
Nick Hill c28feab989 [Core][MRV2] Freeze gc during V2 CG capture; skip per-descriptor cleanup (#54646) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-31 19:37:46 -07:00
Xuan-1998andVladimir Aerov 026d5af7f4 [Bugfix] Fix ncclCommQueryProperties heap overflow with NCCL >= 2.31 (#53008) 
Signed-off-by: Xuan Jiang <xuanj@amazon.com>
Co-authored-by: Vladimir Aerov <vaerov@amazon.com>
 2026-08-29 00:25:00 +00:00
Jiahan Chang (Cyrus)OpenAI CodexJee Jee Li
 		7c877062ac [kernel] Integrate FlashInfer BF16 CuTeDSL Low Latency GEMM (#50572) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-27 22:29:12 +00:00
Luke Alonso 3454335f7b [Bugfix] Resolve B12X modules before Dynamo tracing (#53326) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
 2026-08-25 09:20:15 -07:00
Taneem Ibrahim 2014e47485 [Pooling UX] Improve serve --task error guidance (#53467) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-25 08:20:10 -04:00
Md Saidul Hoque AnikClaude Fable 5Anerudhan Gopal
 		8fe9317f2e [Model][MoE] DeepSeek-V4: add opt-in FlashInfer moe_ep expert backend (#49636) 
Signed-off-by: Md Saidul Hoque Anik <mhoqueanik@nvidia.com>
Signed-off-by: Anerudhan Gopal <agopal@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Anerudhan Gopal <agopal@nvidia.com>
 2026-08-25 03:35:01 +00:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00
Luke Alonsoandmgoin 2740c817ff [Kernel] Add b12x FP4 MoE backend (#52018) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-08-21 08:04:36 -07:00
SeonjinandMisha Goin bd8865a299 [Kernel] Add FlashInfer TRTLLM MXFP8 linear backend (#52204) 
Signed-off-by: seonjinn <sna@nvidia.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-08-20 10:07:22 -04:00
Prudhvi Vuda de216b6e64 [Bugfix] Skip MM processor cache inserts larger than capacity (#53016) 
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
 2026-08-20 13:42:50 +00:00
Turner JabbourandClaude Sonnet 5 1eab6fef01 [CI] replace shellcheck script with shellcheck-py hook (#52572) 
Signed-off-by: Turner <doubleujabbour@gmail.com>
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-08-20 10:09:59 +00:00
Misha Goin 4ab5e5012a [Refactor] Simplify B12X linear kernels and warmup (#52368) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-17 09:45:45 -07:00
Gabriel Wu 97388c44f9 [Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark (#51538) 2026-08-15 07:01:32 -07:00
Qiming ZhangandAndreas Karatzas 1f7427bc0a [UT][XPU] fix b12x UT (#52265) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-14 09:49:47 -04:00
Zhuobin Huangandnodeeeeee 63a9a5010a [Attention][DSA] Take the native decode path for MTP=3 on SM90 (#52164) 
Signed-off-by: zobinHuang <zobin1999@gmail.com>
Co-authored-by: nodeeeeee <zhangkai.nodeee@gmail.com>
 2026-08-14 10:09:51 +00:00
Luke Alonso 3c79b1a8bf [Kernel] Add B12X dense linear backends (#52016) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
 2026-08-13 19:48:33 -07:00
fxmarty-amdAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c5b7c069a4 [ROCm] Defer tilelang import through its import from vllm.tilelang_utils import tilelang and relaxed has_tilelang (#51159) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 13:12:40 -05:00
aoshen02OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7553aac77b [Frontend] Add routed-experts prompt offset (#51906) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 08:09:00 -07:00
Lucas WilkinsonOpenAI CodexClaude Opus 5Benjamin Chislettmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick HillOpenAI Codex
 		7f7a32cfec [Spec Decode] DSpark confidence-scheduled verification (#47808) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-12 12:27:48 -07:00
kiroxuCopilotWentao Yemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		19d61b1273 [Bugfix] Initialize DeepGemmQuantScaleFMT oracle lazily; bound QuantFP8 UE8M0 packed path to group_size 128 (#51359) 
Signed-off-by: BabyDrangoner <148877251+BabyDrangoner@users.noreply.github.com>
Signed-off-by: kiroxu <148877251+BabyDrangoner@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-12 14:27:50 -04:00
Andrii SkliarAndrii SkliarOpenAI Codex
 		1d2d83a07f [Attention] Add FlashInfer XQA decode support on SM12x (#49718) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 16:32:49 -07:00
Chaojun Zhang f863387250 [XPU] Fix UVA weight offloading (non-pinned-tensor views and static Triton launcher) (#51770) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
 2026-08-11 21:48:44 +08:00
vllmellmandtjtanaa 436be94e13 [ROCm][Bugfix] Use TCP store when AITER custom all-reduce is enabled (#51635) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-08-10 11:07:14 +00:00
Raj Vijay FirkeandHarry Mellor 3dafaef027 [Bugfix][Core] Emit --no-{key} for false BooleanOptionalAction flags in YAML config (#51573) 
Signed-off-by: Raj Firke <79653531+rajfirke@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-10 11:05:15 +00:00
aoshen02Parsa Mazaheriaoshen02Claude
 		11ba93f364 [BugFix] Use file:// rendezvous for single-node executors to eliminate startup port races (#50999) 
Signed-off-by: Parsa Mazaheri <p_mazaheri@apple.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Parsa Mazaheri <p_mazaheri@apple.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-09 22:00:04 -04:00
Nick Hill fbff187d59 [Core] Make the GPU sync check thread-local and fix its suppressors (#51455) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-08 17:13:40 -07:00