iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,547 Commits 3 Branches 0 Tags
eb68fb9105eab33888ce252001a269a8c52c19a5
Commit Graph
3252 Commits
Author SHA1 Message Date
MOVZXandHinata eb68fb9105 recoverssm: read the GDN RecoverSSM flag defensively in _forward_core (port of upstream #58863, c1d2da8773) 
Port of the upstream vLLM change #58863 (Apache-2.0). One check read use_gdn_recoverssm directly; it now uses getattr with a False
default like its neighbours, so layers built without __init__ in tests work.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 20:08:55 +07:00
MOVZXandHinata 7f6055caa2 ple, tests: debug-gated pull-flag check; diagnostics report that they ran 
The pull-transport flag check is wired in (rank-gated, off by default, warns when enabled). Both
diagnostic checks log that they ran. Test repairs for helpers v0.30.0 renamed.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 20:01:26 +07:00
MOVZXandHinata dec8be19de [PLE] The pull registry refuses unbound or clashing names; test fixes 
A schema walk fails on any pointer-like int parameter of a custom op on the
PLE path. The FULL_K test child calls the Marlin MoE GEMM by keyword against
its v0.30.0 signature.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 19:56:01 +07:00
MOVZXandHinata 7f4385a422 [PLE] The pull op resolves live pointers by layer name 
The wait/copy op took host pointers as Python-int arguments, which were baked
into the AOT compile cache as the tracing process's addresses; the first
forward after a cache-load boot then failed. The op now carries only
process-stable values and resolves pointers from a per-process registry.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 19:55:23 +07:00
MOVZXandHinata 23349e43b2 [PLE] Integrate the pull transport on v0.30.0 with upstream UVA refused 
- VLLM_E2_PLE_PULL_TRANSPORT replaces the fork's old switch name, which
  v0.30.0 reuses for its UVA backend with the opposite default; strict parser;
  compile/capture factor.
- The pull transport is reachable on v0.30.0 through a pull embedding class
  that inherits upstream's without editing it; the upstream UVA backend is
  provably off while the pull switch is on.
- resolve_ple_mode()/assert_ple_mode() log and assert the effective PLE mode
  for target and drafter, and refuse a mixed configuration.
- Producer log lines for the PLE backend and weight dtype.
- CPU tests for mode resolution and the mixed-config refusal.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 19:52:54 +07:00
MOVZXandHinata 025fd08603 [PLE] Carry the host-mapped pull transport onto v0.30.0 
Restores the fork's PLE offload package (not upstream) and its pull
transport: the offload process holds no CUDA context; the captured op waits on
a device-mapped host flag and copies host->GPU on the model stream; publication
is fenced; releases are synchronous across dummy/capture work; failures close
the rank. Includes the test-only fault-injection cuts for the transport.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 19:49:16 +07:00
MOVZXandHinata 41f6691c26 [Qwen4Exp] Host-resident token embedding tables (VLLM_HOST_EMBED_TABLE) 
Keeps the token embedding table in pinned host memory behind a UVA view for the
stage-0 target and the last-stage MTP drafter, freeing GPU memory for KV. TP
sharding, masking and the all-reduce are retained. The knob is registered as a
compile-cache factor.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 19:43:53 +07:00
MOVZX 85c31d47ad Merge remote-tracking branch 'upstream/main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-03 22:06:10 +07:00
MOVZX 2877121b2d Merge remote-tracking branch 'upstream/main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-03 21:59:03 +07:00
Markus Hartikainen b0e21b3083 [ROCm][Kimi-K3] Add opt-in gfx942 MXFP4-to-int4 conversion (#51274) 
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Signed-off-by: Hartikainen <mahartik@amd.com>
 2026-10-03 14:28:02 +08:00
Julien Debache 111f71a6db [feat] FlashInfer CuteDSL MegaMoE integration (#54049) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-10-02 15:43:33 -04:00
jackLeiandMisha Goin 10f6ab01a9 [Perf] Use value-only reduction for native per-token FP8 quantization (#59800) 
Signed-off-by: jackLei0901 <841747318@qq.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-02 18:37:17 +00:00
Jinzhen LinCodexMisha Goin
 		9cf087fd14 [Perf] Tune MoE weighted-sum kernel launch configuration (#59731) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-02 17:09:29 +00:00
Eric Tangaoshen02Claude Opus 5.5
 		4c4003bb5f [Bugfix] Bind routed-experts capture to the MoE layer, not the kernel (#59455) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 15:50:38 +00:00
aoshen02andClaude Opus 5.5 c91dccc080 [Bugfix][Quark] Pass grouped-routing arguments to OCP MX monolithic kernels (#59752) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 13:46:48 +00:00
DimensionSTPThien TranKimi Code
 		e4294144a8 [Perf] Keep non-speculative GDN decode on the standard path (#59735) 
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
 2026-10-02 13:44:53 +00:00
Rajeev Warrierandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 30e956fad3 [Docs] Fix stale W4A16 NVFP4 default kernel in ModelOpt docs (#59135) 
Signed-off-by: Rajeev Warrier <warrier.rajeev.95@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-02 13:21:43 +00:00
pmanczakandPawel Manczak bdfb02674f [XPU][Kernel] Tune Triton W8A8 block-FP8 GEMM for Intel B70 (#56063) 
Signed-off-by: Pawel Manczak <203671809+pmanczak@users.noreply.github.com>
Co-authored-by: Pawel Manczak <203671809+pmanczak@users.noreply.github.com>
 2026-10-02 12:56:11 +01:00
Simon Danielssonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 98b29aa99a [ROCm][Perf][GLM-5.3-Flash] Add AITER topk backend for decodes (#58167) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-02 11:47:41 +01:00
Lucas WilkinsonClaude Opus 5.5Matthew Bonanni
 		4056c8ac1f [Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool (#59309) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-01 23:04:21 -07:00
Qiu ChunshuoandLucas Wilkinson 8486f1ebfe [Perf][PCP] Shard decode requests across PCP ranks (#52162) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-01 23:50:13 -04:00
ai-jzJingqiao Zhangzjy0516
 		b538d807ff [Bugfix][MRV2] Keep GDN prefill checkpoint metadata local to each cache group (#59536) 
Signed-off-by: Jingqiao Zhang <ai-jz@users.noreply.github.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jingqiao Zhang <ai-jz@users.noreply.github.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-10-01 22:59:15 +08:00
Jiangyun ZhuClaude Opus 5.5Thomas Parnell
 		cb6458c5ce [Core] Remove deprecated mamba_cache_mode "all" (#58997) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
 2026-10-01 02:59:37 -07:00
Ganesh RCursorDouglas Lehr
 		5addde2a0a [CPU][Zen] Pass f32 weight scales to the zentorch INT8 MoE (#59434) 
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-10-01 15:51:53 +08:00
Ganesh RCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e12291d733 [CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers (#54024) 
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 15:18:25 +08:00
Andrii SkliarandAndrii Skliar deb148e730 [Mamba] Add FlashInfer ReplaySSM support for MTP (#52928) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
 2026-09-30 21:41:17 +00:00
Aarushi JainandAndreas Karatzas b33e1cb172 [ROCm]Transpose compressed-tensors MoE weights on device (#59253) 
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-30 12:44:40 -05:00
Sergey Shlyapnikov 4e1182a3a6 [ROCm][MoE] Pad the AITER MoE intermediate size at allocation time, and round the expert-group count to a kernel that exists (#55368) 
Signed-off-by: Sergei Shliapnikov <sergei.shliapnikov@amd.com>
 2026-09-30 23:08:21 +08:00
Varun Sundar Rabindranath 5463fe4962 [KV-Offloading][TP] : Expand replicated_layout detection to multi-group MLA (#57652) 2026-09-30 09:11:03 -04:00
almayneandLi, Jiang <jiang1.li@intel.com> 93dd80f636 [CPU] Conv1d optimised kernel for aarch64 (#54093) 
Signed-off-by: Anna Mayne <anna.mayne@arm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-30 20:23:42 +08:00
Michał GanczarenkoClaude Sonnet 5Harry Mellor
 		df5668e5c9 [XPU] Dispatch nn.LayerNorm to fused SYCL kernel via CustomOp (#57172) 
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-30 12:19:33 +00:00
Jakub ZakrzewskiandCodex 5a4351ab09 [MM] Keep device input normalization fused when encoder compilation is enabled (#59195) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-30 11:08:24 +00:00
Matej Sirovatkaaoshen02OpenAI Codexaoshen02
 		0103a9b96f [Quantization] Add per-token NVFP4 CuTe-DSL MoE backend (#50030) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-09-30 10:42:03 +00:00
Akash kaothalkarAkash kaothalkarAntigravityLi, Jiang <jiang1.li@intel.com>
 		d71f662606 [Bugfix][Model][CPU] Fix Mamba2 quantized in_proj weight and scale loading for TP >1 (#58083) 
Signed-off-by: Akash kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Antigravity <antigravity@google.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-30 17:59:02 +08:00
yhcheong (vllmellm) 16d4ac4ba3 [ROCm][MoE] Support MiMo-V2.6 MXFP4 on gfx942 (#58262) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2026-09-30 17:23:00 +08:00
Fangzhou AiandCursor Agent 77e8b02c8d [ROCm][Perf] Add opt-in a4w4 (FP4 activation) MoE for DeepSeek V4.1 on AITER (#58819) 
Signed-off-by: fai <fangzhouai@gmail.com>
Signed-off-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
Co-authored-by: Cursor Agent (Claude Sonnet 5) <cursoragent@cursor.com>
 2026-09-30 15:07:44 +08:00
Matthew BonanniandClaude Opus 5.5 2798f66860 [Bugfix][HiSparse] Resolve MTP verification rows with a union residency kernel (#59235) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 03:01:45 +00:00
Chaitanya Sri Krishna Lollashikamd123Andreas Karatzas
 		467d81d9a0 [ROCm] Upgrade MoRI version on rocm dockers required for WideEP DP16 DI CI enablement and fixes for combine API (#56073) 
Signed-off-by: Shiksha Patel <shiksha.patel@amd.com>
Signed-off-by: lcskrishna <lollachaitanya@gmail.com>
Co-authored-by: shikamd123 <shikpate@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-29 18:07:31 +00:00
AlanFokCoClaude Opus 5.5aoshen02
 		f4917dadc8 [Bugfix][Quantization] Stop sleep(level=2) from zeroing compressed-tensors KV scales (#57163) 
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-29 15:34:15 +00:00
Chauncey 3e2a7e74a5 [Feat][Model] Enable KDA prefill checkpoints for GLM-5.3-Flash (#56960) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-09-29 09:14:30 +00:00
Jakub ZakrzewskiOpenAI CodexOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		d86cf91240 [MM][Mistral3] Image preprocessing optimization (#57531) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-29 06:53:38 +00:00
MOVZXandHinata ae0350c0de [Spec Decode][MRV2] RecoverSSM for Qwen GDN and PLE short conv (PR #58863) 
Intake of open upstream PR vllm-project/vllm#58863. Active only with
VLLM_USE_V2_MODEL_RUNNER=1; inert otherwise.

Speculative decoding on Gated DeltaNet snapshotted the full recurrent
state after every draft position and reserved one Mamba block per draft
position. On a 24 GB card that is reserved VRAM we never use, plus
per-step copy traffic.

RecoverSSM, the protocol Kimi-K3 KDA already uses, replaces this:
verify runs from a read-only checkpoint and stores a small per-token
replay record [correction (V), normalized key (K), gate (1)] in fp32;
commit replays only the accepted tokens and writes the recurrent state
once, plus the block-boundary state in align mode. The PLE dilated
short conv compacts its accepted window with the same plan.

Conflict in vllm/models/qwen4_exp/nvidia/mtp.py: our get_top_tokens()
work put LocalArgmaxMixin on Qwen4ExpMTP, the PR adds SupportsReplaySSM
to the same base list. Kept both.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-09-29 13:39:09 +07:00
Mehmet CagriClaude Opus 5.5Shanshan Shen
 		63b9931b73 [ROCm][DSv4.1] Paged MXFP4 sparse indexer on aiter's MQA-logits kernel (#58671) 
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-29 14:37:57 +08:00
Isotr0pyandcjackal 07f1ba2ec6 [Multimodal] Remove processsor fallback for fused input norm params resolve (#59073) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-09-29 03:33:43 +00:00
Muhammad Ahmed 0af34418e9 [ROCm][DSv4.1][Perf] Run the delayed mHC seams through aiter's fused Triton kernel (#58655) 
Signed-off-by: Ahmed <Muhammad.Ahmed@amd.com>
 2026-09-29 10:19:10 +08:00
Spandan TiwariClaudeAndreas Karatzas
 		9b29370524 [Bugfix] Fix moe_wna16 w13 zero-point shard split for 8-bit asym GPTQ MoE (#58950) 
Signed-off-by: Spandan Tiwari <sptiwari@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-28 23:38:28 +00:00
stefankoncarevicandAndreas Karatzas 6c300ddb9e [CI][Bugfix] Relax packed_qk_rope_ correctness test to one ULP (#59008) 
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-28 17:29:59 +00:00
HDCharlesClaude Opus 4.6HDCharles <{"message":"Not Found","documentation_url":"https://docs.github.com/rest/users/emails#list-email-addresses-for-the-authenticated-user","status":"404"}>OpenAI CodexOpenAI Codex
 		c01368fd64 [Quant] Use canonical N-first weight format for CT WNA16 MoE (#52798) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: HDCharles <{"message":"Not Found","documentation_url":"https://docs.github.com/rest/users/emails#list-email-addresses-for-the-authenticated-user","status":"404"}>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-28 11:22:44 -06:00
linitra24andJee Jee Li 953f90d25f [LoRA] Support variable num_labels for sequence classification (#57766) 
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-09-28 14:58:35 +00:00
Isotr0py 522101f6ed [Multimodal] Avoid extra d2d for encoder cudagraph with fused input norm (#56711) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-28 21:04:01 +08:00