iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
22,493 Commits 3 Branches 0 Tags
7e5f658d7574381acc4e7dacdb85cb59318aa56e
Commit Graph
22493 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
MOVZXandHinata 7e5f658d75 [Qwen4Exp] Pipeline-stage ownership for PLE, the MTP drafter and the vision tower 
Each pipeline stage builds only the parts it owns: PLE when every PLE layer
lives on stage 0, MTP under PP (port of vLLM PR #46994 minus its ROCm MLA file
and tests) with the drafter on the last stage, and the vision tower on the
first stage only.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 19:43:53 +07:00
MOVZXandHinata 461a8a0f82 [Qwen4Exp] FP8 E4M3 KV reader for QSA with a static scale sidecar 
FP8 E4M3 KV read path for the Qwen4Exp sparse attention (QSA), indexer and
pre-indexer kernels on sm_86, dequantised with a static per-layer scale loaded
from a JSON sidecar, validated against the model's QSA layer set and applied
per pipeline stage. Includes the offline absmax collector that produces the
sidecar and a CUDA-graph-safe clipping counter.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 19:43:52 +07:00
MOVZXandHinata 03f5313f0f [Worker] Capture-safe guard-telemetry infrastructure, default off 
Shared machinery for device-side bounds guards: persistent int64 counters per
site, written by guarded kernels and read by a background thread. The reader
starts only after warm-up and CUDA-graph capture and never waits on the
compute stream; a read failure arms the stop rather than popping the buffer.
Shutdown does one synchronous final drain so a hit on the last step is
reported before teardown. Release default: off.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 19:43:52 +07:00
MOVZX 4a3ebc6b5b Merge branch 'main' into patches 2026-10-05 08:21:46 +00:00
Harshal JanjaniHarry MellorClaude Opus 5
 		34051ad714 feat: Add video support for the Transformers backend (#57441)
pre-commit / pre-run-check (push) Canceled after 0s
Details
pre-commit / pre-commit (push) Canceled after 0s
Details 
Signed-off-by: Harshal Janjani <harshaljanjani@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-10-05 00:51:15 -07:00
Thien TranCodexnamgyu-youn
 		042ab0305c [Perf][Qwen4Exp] Merge QSA QKVG and indexer QK projections (#59533) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: namgyu-youn <namgyu.dev@gmail.com>
 2026-10-05 07:03:29 +00:00
Thiago Mota MartinsandFangzhou Ai 92044241a0 [Bugfix][ROCm] Preserve config during GPU memory profiling (#58014) 
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
 2026-10-04 22:28:33 -07:00
Gawr Gura 0721Gawr Gura 0721mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Wentao Ye
 		e8a53a55e6 [Bugfix][Attention] Support int4_per_token_head on non-power-of-two head sizes (#56198) 
Signed-off-by: Gawr Gura 0721 <LiRunGuo@users.noreply.github.com>
Co-authored-by: Gawr Gura 0721 <LiRunGuo@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-05 04:52:39 +00:00
Micah WilliamsonandFangzhou Ai ac8c2130a8 [ROCm] Bump AITER to 0.1.24.post1 (#59794) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
 2026-10-05 00:19:19 -04:00
Kam BasraandOpenAI Codex 0c16eee3f1 [Bugfix][KV Offload] Reset SimpleCPU eager-store placement state on MRV2 resume (#57816) 
Signed-off-by: Kam Basra <kameldipbasra@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-10-05 02:00:07 +00:00
MOVZX f83ecfac16 Merge remote-tracking branch 'upstream/main' into patches 2026-10-05 08:52:31 +07:00
Vadim GimpelsonandClaude 4f52fa35ef [Bugfix] Fix Qwen4Exp PLE embedding rejecting INC (AutoRound) checkpoints (#59990) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-05 01:52:06 +00:00
bluemelovbluemelovCursorYifan QiaoClaude Opus 5.5
 		49e0f47978 [Bugfix][DSv4.1] Keep the compressor ring out of the null block (#58560) 
Signed-off-by: bluemelov <hannes.b2411@eva.space>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: bluemelov <hannes.b2411@eva.space>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 01:49:41 +00:00
Shenglei FuClaude Opus 5.5Wentao Ye
 		1388100560 [Kernel][LoRA] Add deterministic split-K=8 LoRA shrink for batch invariance (#59377) 
Signed-off-by: Shenglei Fu <117230642+ShengleiFu@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-04 18:46:16 -04:00
TyroneandHarry Mellor 7867d6c52d [Frontend] Name the served models in the model-not-found 404 (#59889) 
Signed-off-by: TyroneNel <71038642+TyroneNel@users.noreply.github.com>
Signed-off-by: TyroneNel <Tyrone.Nel@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-10-04 20:45:51 +00:00
Prudhvi VudaWentao Yemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		155488d853 [Bugfix][Determinism] Preserve output dtype in batch-invariant mean (#59106) 
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-04 17:07:46 +00:00
Stefan WangandRobert Shaw b0eb87fe49 [Bugfix][Frontend] Return streaming errors before the first token (#40986) 
Signed-off-by: 1fanwang <1fannnw@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-04 16:42:49 +00:00
Lio EinaudiYifan DaCodeBuddyClaude Fable 5.1Wentao Ye
 		89439db727 [CI][Docs] Fix decode/prefill consistency test prefix construction; add GLM-4-9B and Phi-4 to batch-invariant tested models (#53692) 
Signed-off-by: Yifan Da <1363818765@qq.com>
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Yifan Da <1363818765@qq.com>
Co-authored-by: CodeBuddy <noreply@codebuddy.dev>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-04 11:55:18 -04:00
DivyClaude Opus 5.5Wentao Ye
 		bd42276e1e [Misc][Platform] Check aligned KV block sizes against every attention backend (#58457) 
Signed-off-by: Divy <divy@coralbricks.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-04 11:13:20 -04:00
Aarushi JainandAndreas Karatzas bdd31c31de [CI] Stop requiring both API servers to record weight sync metrics (#59521) 
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-04 14:14:27 +00:00
00CC f98d1fc493 [ROCm][Bugfix] Fix ROCM_ATTN sliding-window boundary (#59550) 
Signed-off-by: tangzzycc <3081129260@qq.com>
 2026-10-04 13:48:46 +00:00
Taneem IbrahimandLucas Wilkinson d64f6cd08a [CI] Auto-label pooling PRs and issues (#59913) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-04 13:26:10 +00:00
Harry MellorandClaude Opus 5.5 d61081dc3d [CI] Bump peft to satisfy transformers 5.18 minimum (#59932) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 10:53:07 +00:00
Yongqi WangandKunshang Ji 155d23cb00 [Bugfix][XPU] Make DeepSeek V4 FP8 sparse decode graph-capturable (#59159) 
Signed-off-by: Yongqi Wang <yongqi.wang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-04 18:31:15 +08:00
YleanCodexCyrus Leung
 		0872ddf4b9 [Bugfix][Multimodal] Normalize single-channel audio to 1D (#56691) 
Signed-off-by: Ylean <61880605+Woolgathererer@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-04 06:11:41 +00:00
Zupeng Wang 18f8f96025 [Bugfix][KV Offload] Release pending CPU lookup pins on cache reset (#59862) 
Signed-off-by: Wang Zupeng <zupenwang@gmail.com>
 2026-10-04 04:30:17 +00:00
Yufeng HeandFlora Feng 6f75cc7d5d [Bugfix][Responses API] Reuse streamed item ids in final harmony response (#59859) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-04 03:01:04 +00:00
Ting SUNandFlora Feng 50b404e71e [Bugfix][Frontend] Preserve abort finish_reason for scale-out token streams (#47933) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-04 02:57:26 +00:00
Elvir Crnčević ce49174247 [Bugfix] Fix FlashInfer all_reduce backend selection (#56891) 2026-10-03 22:18:58 -04:00
aoshen02andClaude Opus 5.5 5e56e9af2a [Feature] Release the CUDA graph pool on sleep (#59160) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 01:13:10 +00:00
Harry Mellor 9e0b6fcf77 [CI] Bump Transformers version to 5.18.0 (#59621) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-10-04 00:42:35 +00:00
Wei Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 84bcbc6264 [DCP] Enable TokenSpeed MLA with block-interleaved DCP (#59462) 
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-03 17:26:07 +00:00
AIwork4meAIwork4meJartX
 		4ac0d0eac2 [ROCm][RDNA3] Fix W4A16 split-K accuracy and determinism (#54706) 
Signed-off-by: AIwork4me <AIwork4me@users.noreply.github.com>
Co-authored-by: AIwork4me <AIwork4me@users.noreply.github.com>
Co-authored-by: JartX <sagformas@epdcenter.es>
 2026-10-03 23:44:17 +08:00
MOVZX 85c31d47ad Merge remote-tracking branch 'upstream/main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-03 22:06:10 +07:00
MOVZX 2877121b2d Merge remote-tracking branch 'upstream/main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-03 21:59:03 +07:00
Hexiang WangandCodex f03026a548 [Bugfix][MoRIIO] Keep discovery heartbeats running while workers hold the GIL (#59441) 
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-03 14:55:18 +00:00
aoshen02aoshen02Claude Opus 5.5
 		e319f86f15 [Feature] Per-row candidate IDs for prefill token scoring (M2 of #56860) (#56984) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-03 14:17:09 +00:00
Martin Hickey bc21cba967 [Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) (#58588) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-03 19:38:25 +08:00
Yicong Wang 5f30fc7031 [Bugfix][KV Connector][Mooncake] Suppress completion for empty pulls (#59347) 
Signed-off-by: wangyicong <wangyicong@bytedance.com>
 2026-10-03 15:28:00 +08:00
Markus Hartikainen b0e21b3083 [ROCm][Kimi-K3] Add opt-in gfx942 MXFP4-to-int4 conversion (#51274) 
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Signed-off-by: Hartikainen <mahartik@amd.com>
 2026-10-03 14:28:02 +08:00
Yifan QiaoClaude Opus 5.5Codex
 		ca0df1f122 [Bugfix][Core] Exempt exactly the blocks an async KV load writes from zeroing (#59504) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-03 06:26:12 +00:00
Kevin H. LuuandClaude Opus 5.5 1a001d5842 [CI] Drop duplicate bf16 skinny GEMM test from Kimi K3 B200 job (#59850) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 21:43:05 -07:00
Andreas KaratzasOpenAI CodexStefan Koncarevic
 		c28834df00 [ROCm][CI] Extend AMD coverage for distributed, model, and eval tests (#56679) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
 2026-10-02 23:20:51 -05:00
Nils Matteson 28556f8600 [Bugfix] Log CRIU failure details before snapshot cleanup (#59661) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
 2026-10-03 04:02:27 +00:00
Nils Matteson 9fdb174678 [Bugfix] Avoid InfiniBand state in TP1 snapshots (#59699) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
 2026-10-03 03:59:17 +00:00
Divakar Verma faa9860dbd [ROCm][CI][AiterExperts] Add test coverage for hidden/intermediate padding correctness (#59333) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-10-03 01:53:37 +00:00
C 44198f577f [Bugfix][Model] Fix M-RoPE offset double-count in Qwen3-Omni (#58890) 
Signed-off-by: C1ves <3121985758@qq.com>
 2026-10-03 09:00:51 +08:00
Miłosz GrunwaldandKunshang Ji 8c3b720a4d [TEST][XPU][CI] disable xpu tests for nonexistent input norm kernels (#59568) 
Signed-off-by: Milosz Grunwald <milosz.grunwald@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-03 08:27:29 +08:00
DOCCA0andRobert Shaw 7dfe3338d5 [EPLB] Add contention-aware expert migration batching (#52641) 
Signed-off-by: DOCCA0 <2947323341@qq.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-03 00:13:50 +00:00
Shang WangClaude Opus 5.5Tyler Michael SmithShengqi Chen
 		8e443871f0 [CI/Build][NVIDIA] Build Rubin images on the public nvidia/cuda base image (#59288) 
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-10-03 00:12:33 +00:00