iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,534 Commits 3 Branches 0 Tags
fba504d5bb5fb973922d0ae4ee8d7a3bed18a76f
Commit Graph
6540 Commits
Author SHA1 Message Date
MOVZXandHinata dec8be19de [PLE] The pull registry refuses unbound or clashing names; test fixes 
A schema walk fails on any pointer-like int parameter of a custom op on the
PLE path. The FULL_K test child calls the Marlin MoE GEMM by keyword against
its v0.30.0 signature.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 19:56:01 +07:00
MOVZXandHinata 7f4385a422 [PLE] The pull op resolves live pointers by layer name 
The wait/copy op took host pointers as Python-int arguments, which were baked
into the AOT compile cache as the tracing process's addresses; the first
forward after a cache-load boot then failed. The op now carries only
process-stable values and resolves pointers from a per-process registry.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 19:55:23 +07:00
MOVZXandHinata 23349e43b2 [PLE] Integrate the pull transport on v0.30.0 with upstream UVA refused 
- VLLM_E2_PLE_PULL_TRANSPORT replaces the fork's old switch name, which
  v0.30.0 reuses for its UVA backend with the opposite default; strict parser;
  compile/capture factor.
- The pull transport is reachable on v0.30.0 through a pull embedding class
  that inherits upstream's without editing it; the upstream UVA backend is
  provably off while the pull switch is on.
- resolve_ple_mode()/assert_ple_mode() log and assert the effective PLE mode
  for target and drafter, and refuse a mixed configuration.
- Producer log lines for the PLE backend and weight dtype.
- CPU tests for mode resolution and the mixed-config refusal.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 19:52:54 +07:00
MOVZXandHinata 025fd08603 [PLE] Carry the host-mapped pull transport onto v0.30.0 
Restores the fork's PLE offload package (not upstream) and its pull
transport: the offload process holds no CUDA context; the captured op waits on
a device-mapped host flag and copies host->GPU on the model stream; publication
is fenced; releases are synchronous across dummy/capture work; failures close
the rank. Includes the test-only fault-injection cuts for the transport.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 19:49:16 +07:00
MOVZXandHinata 41f6691c26 [Qwen4Exp] Host-resident token embedding tables (VLLM_HOST_EMBED_TABLE) 
Keeps the token embedding table in pinned host memory behind a UVA view for the
stage-0 target and the last-stage MTP drafter, freeing GPU memory for KV. TP
sharding, masking and the all-reduce are retained. The knob is registered as a
compile-cache factor.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 19:43:53 +07:00
MOVZXandHinata 7e5f658d75 [Qwen4Exp] Pipeline-stage ownership for PLE, the MTP drafter and the vision tower 
Each pipeline stage builds only the parts it owns: PLE when every PLE layer
lives on stage 0, MTP under PP (port of vLLM PR #46994 minus its ROCm MLA file
and tests) with the drafter on the last stage, and the vision tower on the
first stage only.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 19:43:53 +07:00
MOVZX 4a3ebc6b5b Merge branch 'main' into patches 2026-10-05 08:21:46 +00:00
Harshal JanjaniHarry MellorClaude Opus 5
 		34051ad714 feat: Add video support for the Transformers backend (#57441)
pre-commit / pre-run-check (push) Canceled after 0s
Details
pre-commit / pre-commit (push) Canceled after 0s
Details 
Signed-off-by: Harshal Janjani <harshaljanjani@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-10-05 00:51:15 -07:00
MOVZX f83ecfac16 Merge remote-tracking branch 'upstream/main' into patches 2026-10-05 08:52:31 +07:00
Prudhvi VudaWentao Yemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		155488d853 [Bugfix][Determinism] Preserve output dtype in batch-invariant mean (#59106) 
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-04 17:07:46 +00:00
Harry Mellor 9e0b6fcf77 [CI] Bump Transformers version to 5.18.0 (#59621) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-10-04 00:42:35 +00:00
MOVZX 85c31d47ad Merge remote-tracking branch 'upstream/main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-03 22:06:10 +07:00
MOVZX 2877121b2d Merge remote-tracking branch 'upstream/main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-03 21:59:03 +07:00
Markus Hartikainen b0e21b3083 [ROCm][Kimi-K3] Add opt-in gfx942 MXFP4-to-int4 conversion (#51274) 
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Signed-off-by: Hartikainen <mahartik@amd.com>
 2026-10-03 14:28:02 +08:00
C 44198f577f [Bugfix][Model] Fix M-RoPE offset double-count in Qwen3-Omni (#58890) 
Signed-off-by: C1ves <3121985758@qq.com>
 2026-10-03 09:00:51 +08:00
Julien Debache 111f71a6db [feat] FlashInfer CuteDSL MegaMoE integration (#54049) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-10-02 15:43:33 -04:00
jackLeiandMisha Goin 10f6ab01a9 [Perf] Use value-only reduction for native per-token FP8 quantization (#59800) 
Signed-off-by: jackLei0901 <841747318@qq.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-02 18:37:17 +00:00
Thang NguyenandClaude Opus 5.5 7cfd233538 Revert "[Bugfix] Tie lm_head.weight for Nemotron Parse when checkpoint omits it" (#53020) (#59805) 
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 10:19:25 -07:00
Jinzhen LinCodexMisha Goin
 		9cf087fd14 [Perf] Tune MoE weighted-sum kernel launch configuration (#59731) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-02 17:09:29 +00:00
Eric Tangaoshen02Claude Opus 5.5
 		4c4003bb5f [Bugfix] Bind routed-experts capture to the MoE layer, not the kernel (#59455) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 15:50:38 +00:00
aoshen02andClaude Opus 5.5 c91dccc080 [Bugfix][Quark] Pass grouped-routing arguments to OCP MX monolithic kernels (#59752) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 13:46:48 +00:00
DimensionSTPThien TranKimi Code
 		e4294144a8 [Perf] Keep non-speculative GDN decode on the standard path (#59735) 
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
 2026-10-02 13:44:53 +00:00
SeongJun Leelesj0610Claude Opus 5Thien Tran
 		24df2d3fc7 [Model] Add Qwen4Exp to the Qwen GDN Triton warmup (#56742) 
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-02 21:38:15 +08:00
Rajeev Warrierandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 30e956fad3 [Docs] Fix stale W4A16 NVFP4 default kernel in ModelOpt docs (#59135) 
Signed-off-by: Rajeev Warrier <warrier.rajeev.95@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-02 13:21:43 +00:00
pmanczakandPawel Manczak bdfb02674f [XPU][Kernel] Tune Triton W8A8 block-FP8 GEMM for Intel B70 (#56063) 
Signed-off-by: Pawel Manczak <203671809+pmanczak@users.noreply.github.com>
Co-authored-by: Pawel Manczak <203671809+pmanczak@users.noreply.github.com>
 2026-10-02 12:56:11 +01:00
Harry MellorandClaude Opus 5.5 6e4efafc6e [Model] Migrate GPT-NeoX, Phi, Seed-OSS and Jais2 to the Transformers modeling backend (#59701) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 11:18:35 +00:00
Harry MellorandClaude Opus 5.5 5d20979fc9 [Misc] Remove code paths for Transformers < 5.16.1 (#59762) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 11:17:24 +00:00
Simon Danielssonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 98b29aa99a [ROCm][Perf][GLM-5.3-Flash] Add AITER topk backend for decodes (#58167) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-02 11:47:41 +01:00
Farhad Al-AminClaude Opus 5.5Harry Mellor
 		af2a582d5c [Bugfix][Spec Decode] Qualify Transformers backend attention layer names with the model prefix (#59293) 
Signed-off-by: Farhad Al-Amin <99006654+Legend-2727@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-10-02 11:21:20 +01:00
Juan Pérez de Algaba c15672cfb4 [Security] Accept zero-sum DeepSeek-OCR pixel tensors (#59417) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
 2026-10-02 09:17:12 +00:00
Lucas WilkinsonClaude Opus 5.5Matthew Bonanni
 		4056c8ac1f [Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool (#59309) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-01 23:04:21 -07:00
Mohit SinglaCodexThien Tran
 		0cbac6cd13 [Model][Spec Decode] Enable EAGLE3/DSpark pipeline parallelism for Sarvam MLA (#55902) 
Signed-off-by: mohit-sarvam <mohit@sarvam.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-02 04:28:19 +00:00
Qiu ChunshuoandLucas Wilkinson 8486f1ebfe [Perf][PCP] Shard decode requests across PCP ranks (#52162) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-01 23:50:13 -04:00
Guanxin Liaoshen524Claude Opus 5.5aoshen02
 		a810316c08 [Bugfix] Keep batch-invariance NCCL pins out of the weight-transfer group (#59500) 
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-02 11:19:28 +08:00
Harry MellorandClaude Opus 5.5 8faacd707f [Model] Migrate Glm, Arcee, CWM and Mellum to the Transformers modeling backend (#59679) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 03:15:14 +00:00
Flora Feng 785b990116 [Distributed] Enable custom all-reduce under VLLM_BATCH_INVARIANT (#58623) 
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-10-02 02:25:48 +00:00
Harry MellorandClaude Opus 5.5 7b44aad5db [MyPy] Fix mypy errors in vllm/model_executor/models/[kK]* (#59402) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 19:32:27 +00:00
Mehmet Cagri 08e03df940 [Bugfix][ROCm] Use a zero default for masked scales in the MXFP8 GEMM (#59454) 
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com>
 2026-10-01 11:32:25 -05:00
aniskumar-nvandCyrus Leung 83cadd65d9 [Bugfix] Tie lm_head.weight for Nemotron Parse when checkpoint omits it (#53020) 
Signed-off-by: Anish Kumar <aniskumar@nvidia.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-01 09:26:33 -07:00
ai-jzJingqiao Zhangzjy0516
 		b538d807ff [Bugfix][MRV2] Keep GDN prefill checkpoint metadata local to each cache group (#59536) 
Signed-off-by: Jingqiao Zhang <ai-jz@users.noreply.github.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jingqiao Zhang <ai-jz@users.noreply.github.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-10-01 22:59:15 +08:00
Jiangyun ZhuClaude Opus 5.5Thomas Parnell
 		cb6458c5ce [Core] Remove deprecated mamba_cache_mode "all" (#58997) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
 2026-10-01 02:59:37 -07:00
YueshenZYueshenZClaude CodeCodexzjy0516
 		08d77cad7b [Bugfix] Gate Kimi-K3 KDA warmup on sys.modules to skip Kimi import for non-Kimi models (#59257) 
Signed-off-by: YueshenZ <YueshenZ@users.noreply.github.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: YueshenZ <YueshenZ@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-10-01 09:25:45 +00:00
Jiangyun ZhuchanhClaude Opus 5.5
 		c4fc3c7d32 [Bugfix][Spec Decode] Per-module LM heads for multi-layer MTP on Model Runner V2 (#58921) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: chanh <channguyen@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 01:09:14 -07:00
Ganesh RCursorDouglas Lehr
 		5addde2a0a [CPU][Zen] Pass f32 weight scales to the zentorch INT8 MoE (#59434) 
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-10-01 15:51:53 +08:00
Ganesh RCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e12291d733 [CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers (#54024) 
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 15:18:25 +08:00
Ning Xie a37c402f32 [weight loader] add dtype equality validation between parameter and weight (#51792) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-10-01 03:07:31 +00:00
Ning Xie bfbcde26d5 [Observability] add model initializing duration log (#52141) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-10-01 02:54:33 +00:00
Andrii SkliarandAndrii Skliar deb148e730 [Mamba] Add FlashInfer ReplaySSM support for MTP (#52928) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
 2026-09-30 21:41:17 +00:00
VenkyandClaude Opus 5.5 42f0c17ea7 [Bugfix][PP][Spec Decode] MiniMax-M3 EAGLE3 aux-state relay at PP > 1 and per-stage FlashInfer autotune (#57197) 
Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-01 01:14:24 +04:00
Aarushi JainandAndreas Karatzas b33e1cb172 [ROCm]Transpose compressed-tensors MoE weights on device (#59253) 
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-30 12:44:40 -05:00