iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,280 Commits 3 Branches 0 Tags
663d7f679eda78a8d48613dfede8a4b4bcff2b74
Commit Graph
459 Commits
Author SHA1 Message Date
siyuandIsotr0py 73fb19151f [Fast Start] Support fp4 (#55465) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-10 13:22:17 +00:00
Tianyu Guo 479611c0c6 [Frontend][EPD] Use JSON arrays for multimodal metadata (#56090) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-09-10 05:04:14 +00:00
Nick Hill 4aadb0f14d [Core] Enhance cpu<->gpu sync checking to include paged async copies (#53491) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-08 17:26:21 -07:00
Wentao Ye 6b15bea080 [Refactor] Remove utils dead code (#53941) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-08 11:22:09 -04:00
mingg26Kimi Codemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7fa2c63796 [Bugfix] Autotune FlashInfer deferred MoE decode kernels before CUDA graph capture (#55377) 
Signed-off-by: ming <ming@inferact.ai>
Co-authored-by: Kimi Code <kimi-code@moonshot.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-08 04:02:23 +00:00
Wei Zhao a69402aaa8 [Perf] Integrate FlashInfer KDA kernels (#55364) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-09-07 20:29:30 -07:00
Jee Jee Li 199cb9b964 [Kernel] Remove unused fake implementation (#55535) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-09-06 17:54:32 -07:00
Nick Hill 8277c42e4c [Perf] Ensure async h2d copies are pinned in more places (#55202) 2026-09-04 17:17:01 -07:00
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
Yusuke OdaandChauncey 798b557e06 [Frontend] Add site-packages support for reasoning/tool parser plugins (#45241) 
Signed-off-by: odashi <yusuke.oda@predicate.jp>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-02 07:41:13 +00:00
fxmarty-amdandAndreas Karatzas d1c15e589d [CI] Speed up quantization test group (#53291) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-01 13:15:24 -05:00
Luke AlonsoandLucas Wilkinson 0d4ad47981 [Kernel] Add B12X causal paged attention backend (#52017) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-09-01 10:42:02 -04:00
lzhan011andCyrus Leung ce7391712b [Bugfix][Security] Bound embedding densification before to_dense() (#54632) 
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-01 08:45:14 +00:00
sychen52 7c5dc571cb [Attention][DSA] Enable W4A16 DSA (#51724) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
 2026-09-01 12:37:47 +08:00
Lain 882ca8d696 [Kernel] add Flashinfer cutedsl w4a16 linear (#53014) 
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
 2026-09-01 02:49:14 +00:00
Nick Hill c28feab989 [Core][MRV2] Freeze gc during V2 CG capture; skip per-descriptor cleanup (#54646) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-31 19:37:46 -07:00
Xuan-1998andVladimir Aerov 026d5af7f4 [Bugfix] Fix ncclCommQueryProperties heap overflow with NCCL >= 2.31 (#53008) 
Signed-off-by: Xuan Jiang <xuanj@amazon.com>
Co-authored-by: Vladimir Aerov <vaerov@amazon.com>
 2026-08-29 00:25:00 +00:00
Jiahan Chang (Cyrus)OpenAI CodexJee Jee Li
 		7c877062ac [kernel] Integrate FlashInfer BF16 CuTeDSL Low Latency GEMM (#50572) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-27 22:29:12 +00:00
Luke Alonso 3454335f7b [Bugfix] Resolve B12X modules before Dynamo tracing (#53326) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
 2026-08-25 09:20:15 -07:00
Taneem Ibrahim 2014e47485 [Pooling UX] Improve serve --task error guidance (#53467) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-25 08:20:10 -04:00
Md Saidul Hoque AnikClaude Fable 5Anerudhan Gopal
 		8fe9317f2e [Model][MoE] DeepSeek-V4: add opt-in FlashInfer moe_ep expert backend (#49636) 
Signed-off-by: Md Saidul Hoque Anik <mhoqueanik@nvidia.com>
Signed-off-by: Anerudhan Gopal <agopal@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Anerudhan Gopal <agopal@nvidia.com>
 2026-08-25 03:35:01 +00:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00
Luke Alonsoandmgoin 2740c817ff [Kernel] Add b12x FP4 MoE backend (#52018) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-08-21 08:04:36 -07:00
SeonjinandMisha Goin bd8865a299 [Kernel] Add FlashInfer TRTLLM MXFP8 linear backend (#52204) 
Signed-off-by: seonjinn <sna@nvidia.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-08-20 10:07:22 -04:00
Prudhvi Vuda de216b6e64 [Bugfix] Skip MM processor cache inserts larger than capacity (#53016) 
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
 2026-08-20 13:42:50 +00:00
Turner JabbourandClaude Sonnet 5 1eab6fef01 [CI] replace shellcheck script with shellcheck-py hook (#52572) 
Signed-off-by: Turner <doubleujabbour@gmail.com>
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-08-20 10:09:59 +00:00
Misha Goin 4ab5e5012a [Refactor] Simplify B12X linear kernels and warmup (#52368) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-17 09:45:45 -07:00
Gabriel Wu 97388c44f9 [Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark (#51538) 2026-08-15 07:01:32 -07:00
Qiming ZhangandAndreas Karatzas 1f7427bc0a [UT][XPU] fix b12x UT (#52265) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-14 09:49:47 -04:00
Zhuobin Huangandnodeeeeee 63a9a5010a [Attention][DSA] Take the native decode path for MTP=3 on SM90 (#52164) 
Signed-off-by: zobinHuang <zobin1999@gmail.com>
Co-authored-by: nodeeeeee <zhangkai.nodeee@gmail.com>
 2026-08-14 10:09:51 +00:00
Luke Alonso 3c79b1a8bf [Kernel] Add B12X dense linear backends (#52016) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
 2026-08-13 19:48:33 -07:00
fxmarty-amdAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c5b7c069a4 [ROCm] Defer tilelang import through its import from vllm.tilelang_utils import tilelang and relaxed has_tilelang (#51159) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 13:12:40 -05:00
aoshen02OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7553aac77b [Frontend] Add routed-experts prompt offset (#51906) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 08:09:00 -07:00
Lucas WilkinsonOpenAI CodexClaude Opus 5Benjamin Chislettmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick HillOpenAI Codex
 		7f7a32cfec [Spec Decode] DSpark confidence-scheduled verification (#47808) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-12 12:27:48 -07:00
kiroxuCopilotWentao Yemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		19d61b1273 [Bugfix] Initialize DeepGemmQuantScaleFMT oracle lazily; bound QuantFP8 UE8M0 packed path to group_size 128 (#51359) 
Signed-off-by: BabyDrangoner <148877251+BabyDrangoner@users.noreply.github.com>
Signed-off-by: kiroxu <148877251+BabyDrangoner@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-12 14:27:50 -04:00
Andrii SkliarAndrii SkliarOpenAI Codex
 		1d2d83a07f [Attention] Add FlashInfer XQA decode support on SM12x (#49718) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 16:32:49 -07:00
Chaojun Zhang f863387250 [XPU] Fix UVA weight offloading (non-pinned-tensor views and static Triton launcher) (#51770) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
 2026-08-11 21:48:44 +08:00
vllmellmandtjtanaa 436be94e13 [ROCm][Bugfix] Use TCP store when AITER custom all-reduce is enabled (#51635) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-08-10 11:07:14 +00:00
Raj Vijay FirkeandHarry Mellor 3dafaef027 [Bugfix][Core] Emit --no-{key} for false BooleanOptionalAction flags in YAML config (#51573) 
Signed-off-by: Raj Firke <79653531+rajfirke@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-10 11:05:15 +00:00
aoshen02Parsa Mazaheriaoshen02Claude
 		11ba93f364 [BugFix] Use file:// rendezvous for single-node executors to eliminate startup port races (#50999) 
Signed-off-by: Parsa Mazaheri <p_mazaheri@apple.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Parsa Mazaheri <p_mazaheri@apple.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-09 22:00:04 -04:00
Nick Hill fbff187d59 [Core] Make the GPU sync check thread-local and fix its suppressors (#51455) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-08 17:13:40 -07:00
Wei-Ming Chen 1c94e8dc7d Add NVFP4 KV 4-over-6 scale search (#45187) 
Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com>
 2026-08-07 13:22:27 -07:00
fxmarty-amd 3518110f2f [Online quantization] Add online MXFP4 quantization support (#49347) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-08-07 12:39:46 -07:00
Jinzhen LinandOpenAI Codex fcde8e1460 [Refactor] refactor humming linear and moe backends to use explicit layer configs (#49610) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-07 10:03:19 -07:00
aoshen02Chenglun Huferkans-amiraoshen02Claude Fable 5
 		448344c0e2 [Bugfix] Fix get_open_port() livelock on DP-reserved ports and cover get_open_ports_list (#50965) 
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Chenglun Hu <chenglunhu@gmail.com>
Co-authored-by: ferkans-amir <amir.rezaei@tu-berlin.de>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-07 22:39:33 +08:00
Nick HillandClaude 7635a9002b [Core] Explicitly manage torch CPU threads in workers (#49919) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-04 10:06:38 -07:00
Wentao Ye 0cf49a5d15 [Refactor] Remove multiple dead codes (#50285) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-03 10:59:08 -04:00
Michael GoinandOpenAI Codex 90329913e9 [UX] Reduce startup log noise (#50590) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-31 13:04:34 -07:00
+1
Juan VillamizarDouglas LehrClaude Opus 4.7DanielJaden MathiasjpvillamrootDouglas LehrdllehrDaniel ChanAndreas Karatzas
 		4f1da84eb5 Enable gfx1250 ROCm architecture (#46516) 
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com>
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com>
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: dllehr <douglehr@amd.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-31 01:41:12 +00:00
Julien Denize 1a20d23dab [PARSER][Mistral] unified engine-based parser for reasoning and tool calls (#48947) 2026-07-30 15:38:00 +02:00