iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,341 Commits 3 Branches 0 Tags
patches
Commit Graph
23341 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
MOVZX 49d708e158 Merge branch 'main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 18:09:11 +00:00
MOVZXandHinata 6cf86cd95f [Bugfix] Load checkpoint KV-cache scales in the Qwen3.5 family 
Checkpoints that ship calibrated k_scale/v_scale tensors (e.g.
unsloth/Qwen3.8-27B-NVFP4, which carries one pair per full-attention
layer) name them ...self_attn.{k,v}_scale, but the runtime parameters
live on the inner attention layer (...self_attn.attn.{k,v}_scale).
Qwen3_5Model.load_weights hands the iterable straight to
AutoWeightsLoader, which performs no scale-name remapping, so the
scales are silently dropped and --kv-cache-dtype fp8 runs at scale 1.0
with only a warning_once that fires even on success (the MTP draft
layer legitimately has no scales).

Route scale-suffixed names through maybe_remap_kv_scale_name first,
mirroring gemma4/gpt_oss. No behavior change for checkpoints without
scales.

Measured on RTX 5090 / Qwen3.8-27B NVFP4 / fp8 KV / MTP k=3 (GSM8K
greedy, n=300): 92.3% at scale 1.0 vs 93.7% with scales loaded
(bf16-KV control 94.5%).

Fixes #54623

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 17:18:47 +00:00
alpandmottopanikeiku ff01a6b14f [Bugfix][Frontend] Stop beam search on generation config EOS tokens (#62072)
pre-commit / pre-run-check (push) Canceled after 0s
Details
pre-commit / pre-commit (push) Canceled after 0s
Details 
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
 2026-10-18 16:49:42 +00:00
aoshen02andClaude Opus 5.5 fe8f2140ee [Doc] Document /inference/v1/generate request/response fields and DP rank header (#62305) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 15:26:36 +00:00
Taoyu ZhuClaude Opus 5.5vllm-agent
 		e80b6c642f [ROCm][DSv4] Overlap the unfused shared expert with AITER MegaMoE (#61819) 
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-18 22:39:38 +08:00
aoshen02andClaude Opus 5.5 371745ebd6 [Doc][RL] Document routed experts return for MoE routing replay (R3) (#62302) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 21:11:45 +08:00
André Lustosamergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Shengqi Chen
 		072c0886d5 [Build] Keep plain CUDA arch when its suffixed variant is also requested (#56549) 
Signed-off-by: Andre Lustosa <alustosa@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-10-18 20:13:17 +08:00
Muhammad IbrahimMuhammad IbrahimShengqi Chen
 		6b24947628 [CI/Build] Build DeepEP for SM103 (B300/GB300) in the Docker image (#61829) 
Signed-off-by: Muhammad Ibrahim <muhammad.ibrahim@multiversecomputing.com>
Co-authored-by: Muhammad Ibrahim <muhammad.ibrahim@multiversecomputing.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-10-18 20:12:34 +08:00
Harry HuangandThien Tran 494663940b [Qwen4Exp] Clean up HC and PLE implementations for Qwen3.8-Flash-Next (#61454) 
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-18 19:38:21 +08:00
Rémi Delhaye 3c13653738 [Frontend] Score with the chat template declared by Sentence Transformers CrossEncoders (#60877) 
Signed-off-by: rdlhio <contact@rdlh.io>
 2026-10-18 11:30:12 +00:00
Kevin H. LuuandClaude Opus 5.5 953fa442fa [CI] Per-test timeout for Language Models (Extended Pooling) (#62279) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 04:00:16 -07:00
Isotr0py 3f1b04e045 [Bugfix] Fix Opencv video backend frames count missing for WebM video inputs (#61759) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-18 10:37:39 +00:00
aoshen02andClaude Opus 5.5 2365177ddd [Doc][RL] Document the weight-transfer HTTP protocol for trainers without vLLM (#62267) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 09:26:56 +00:00
MOVZX c5ab42bc0f Merge branch 'main' into patches 2026-10-18 16:04:36 +07:00
Louie Tsai 1b2a666391 [Doc][Recipe] Fix for redhat fp8 xeon support issue (#62245) 2026-10-18 16:46:56 +08:00
Kevin H. LuuClaude Opus 5.5vllm-agent
 		83c9a547bf [CI][Release] Fix release wheel builds: stop excluding tracked recipes/ from Docker context (#62221) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-18 00:48:00 -07:00
trienebutdieneClaude Opus 5.5Shengqi Chenkimseunghyun
 		b510857d37 [Bugfix][Build] Build fused GDN MTP decode for SM80-SM90 with CUDA < 13 (#60896) 
Signed-off-by: kimseunghyun <incongnito12@gmail.com>
Signed-off-by: kimseunghyun <hellonypcteam@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com>
 2026-10-18 15:14:06 +08:00
Junru ChenNick Hillvllm-agent
 		7c9042c05a [Bugfix][MRV2] Fix speculative warmup deadlock with pipeline parallelism (#62129) 
Signed-off-by: chenjr97 <chenjr97@mail2.sysu.edu.cn>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-17 23:51:32 -07:00
Andreas KaratzasAndreas KaratzasClaude
 		da9134bc89 [ROCm][CI] Give the MoE parallel-launch watchdog room for AITER JIT builds (#62161) 
Signed-off-by: Andreas Karatzas <AndreasKaratzas@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <AndreasKaratzas@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-17 21:49:27 -05:00
Rohan PotdarClaude Opus 5.5Micah WilliamsonKevin H. LuuAndreas Karatzas
 		b2456c622b [ROCm][CI] Make ROCm 10.0 the default image; keep ROCm 7.2 as -rocm72 (#58761) 
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-17 21:43:59 -05:00
Kevin H. LuuClaude Opus 5.5Andreas Karatzas
 		03ee2e134d [CI][ROCm] Fix order-dependent test_rocm_aiter_sampler_defers_import (CPU tensors hit Triton) (#62213) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-18 01:10:08 +00:00
djramicandAndreas Karatzas 9c778e9ff8 [Bugfix][EPLB] Wait for pending expert buffer copies before NIXL READs (#62081) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-17 18:18:02 -05:00
David CheungClaude Opus 5.5vllm-agent
 		7879f3687b [NIXL] Drop unreachable producer-side notif counting in push mode (#62210) 
Signed-off-by: David Cheung <d7cheung@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-17 21:24:35 +00:00
Kevin H. LuuandClaude Opus 5.5 255f14599e [Bugfix][Rust Frontend] Fix macOS build: read inherited listener family from its address (#62177) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-17 11:56:28 -07:00
yifanFenggClaude Opus 5.5Wentao YeKevin H. Luu
 		03e8300d2c [Bugfix] Enable inductor batch-invariant mode under VLLM_BATCH_INVARIANT (#61118) 
Signed-off-by: Yifan Feng <evanfengff@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-17 14:53:06 -04:00
Afonso RibeiroandFlora Feng 5c1c903501 [Perf][Parser] Stream Qwen3 tool argument values incrementally (12 KB call: 2.1 s to 11 ms) (#62027) 
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-17 18:49:06 +00:00
Tyrone bafb263664 [Bugfix][Benchmarks] Name the real cause when the /tokenize alignment probe fails (#59888) 2026-10-17 14:47:52 -04:00
Nick HillandClaude Opus 5.5 0c5dd18e7a [MRV2] Collect KV transfer results before the drafter runs (#62041) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-17 10:27:57 -07:00
Ting SUNmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Wentao Ye
 		763e3f7c91 [Bugfix][Spec Decode] Disable dynamic speculative decoding for fixed-K proposers instead of crashing (#46399) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-17 12:41:04 -04:00
AlecandClaude Opus 5.5 e744eacf68 [Rust Frontend] Validate bad_words limits during request lowering (#61933) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-17 09:10:40 -07:00
Woosuk KwonClaude Opus 5.5vllm-agent
 		c0b6f74bee [Benchmark] Add per-case kernel profiles to vllm bench forward (#61692) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-17 08:27:11 -07:00
siyuandXun Sun c61a1e73c6 [Feature] Skip MLA weight absorption when weights are pre-processed (#57320) 
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
 2026-10-17 23:10:04 +08:00
aoshen02andClaude Opus 5.5 237d99b4af Revert "[Docs] Add train-inference consistency agent skill" (#61551) (#62186) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-17 13:54:30 +00:00
aoshen02andClaude Opus 5.5 2d8efee9a7 [Docs] Add train-inference consistency agent skill (#61551) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-17 20:49:50 +08:00
Roy WangandClaude Opus 5.5 21e09ea959 [Doc] Sync DeepSeek-V4.1-Flash recipe with vllm-project/recipes#1136, #1137 (#62179) 
Signed-off-by: esmeetu <jasonailu87@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-17 12:27:52 +00:00
aoshen02aoshen524Claude Opus 5.5
 		837e87e1c5 [Feature][LoRA][MoE] Expert LoRA for FlashInfer TRTLLM MXFP4 experts (#61979) 
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-17 20:17:56 +08:00
liuzhenweiandKunshang Ji 4e964b2623 [XPU] Fix Event init failure w/ external (#61743) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-17 19:10:39 +08:00
Vadim GimpelsonandClaude ec8b20fbf0 [Bugfix] Fix Gemma 4 encoder CUDA graph capture OOM at startup (#61365) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-17 11:04:49 +00:00
Kevin H. LuuClaude Opus 5.5Wei Zhao
 		2b85d9cb65 [Perf][Bugfix] Prefer MNNVL for single-node FlashInfer allreduce again (#61989) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-10-17 02:55:28 -07:00
gitbisectorClaudeYifan Qiao
 		026f6ab268 [Model] DeepSeek-V4/V4.1 VL: stream language-model weights instead of sorting the checkpoint (#58857) 
Signed-off-by: git bisector <gitbisector@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-10-17 09:37:56 +00:00
Roy WangandClaude Opus 5.5 b515648641 [Doc] Host model recipes in recipes/ (#61302) 
Signed-off-by: esmeetu <jasonailu87@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-17 09:14:36 +00:00
David CheungClaude Opus 5.5Yifan Zong
 		f98ae6a9fd [Bugfix][Core] Keep resumable sessions that stop while preempted (#58778) 
Signed-off-by: David Cheung <d7cheung@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Zong <yzong@redhat.com>
 2026-10-17 09:08:14 +00:00
afriedriSimon DanielssonKevin H. LuuAndreas Karatzas
 		2067297b2f [ROCm][Perf] Use AITER tuned configs to select a4w4 gemm kernel instead of env var (#61278) 
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Signed-off-by: afriedri <afriedri@amd.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-17 04:00:28 -05:00
Aarushi JainandAndreas Karatzas 3bf25c5097 [CI][Bugfix] Ignore AMD CI webhook no-op builds in /amd-ci run and retry (#62087) 
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-17 03:54:31 -05:00
Jee Jee Li e56b1603af [Kernel]Add norm fused kernel for Step-5 Preview and Muse Spark Open (#62132) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-10-17 01:53:00 -07:00
Ting SUN eb8de8d1a1 [Bugfix][OpenAI] Respect allowed_token_ids in beam search (#47630) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-10-17 08:50:40 +00:00
dskwe b05ede5616 [Bugfix][Frontend] Preserve token IDs in completion derender responses (#62114) 
Signed-off-by: Yuan Li <dskwelmcy@163.com>
 2026-10-17 08:27:48 +00:00
Cheng RuiCodexGiancarlo Delfin
 		61bb590877 [Bugfix][Spec Decode] Clear stale shared block-table rows (#61174) 
Signed-off-by: Cheng Rui <286040359@qq.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
 2026-10-17 01:25:02 -07:00
Dakai AnYifan QiaoClaude Opus 5.5
 		c3de0b37e2 [Core][Spec Decode] Replace EAGLE group heuristics with explicit draft-layer tags (#61455) 
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-17 08:22:33 +00:00
MOVZX e0fe5df731 Merge branch 'main' into patches 2026-10-17 08:06:56 +00:00