MOVZX
76df77c04e
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-15 21:16:39 +07:00
Jiangyun Zhu and Claude Opus 5.5
badfcdbf91
[Perf][GDN] Internal prefill checkpoints for the FlashInfer GDN prefill backend ( #60050 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 14:49:46 +08:00
Jiangyun Zhu
ff305934f1
[Tests][Mamba] Wait for accepted counts in fake MTP proposer ( #61342 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-13 16:15:47 +08:00
2cc2e68d1c
[Core][Mamba] Remove leftovers of mamba_cache_mode="all" ( #60043 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 00:58:59 -07:00
MOVZX
85c31d47ad
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-03 22:06:10 +07:00
ca0df1f122
[Bugfix][Core] Exempt exactly the blocks an async KV load writes from zeroing ( #59504 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-03 06:26:12 +00:00
MOVZX
a63dca4afb
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 12:54:18 +07:00
Fanli Lin and Kunshang Ji
6a6c2952fa
[XPU][CI] Make test_mamba_prefix_cache block-size agnostic ( #58269 )
...
Signed-off-by: Lin, Fanli <fanli.lin@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-29 07:12:43 +08:00
MOVZX
5da84df6b7
[BugFix] Honor drop_eagle_block in MambaManager
...
On a hybrid model (full-attention + Mamba/GDN layers, e.g. Qwen3.6-35B-A3B),
enabling MTP/EAGLE speculative decoding together with prefix caching corrupts
the recurrent state, and the server serves wrong answers. #43559 reports
tool-eval accuracy dropping from ~90% to ~50%. Nothing raises an exception
and nothing shows up in the logs, so an operator has no signal that anything
is wrong, and prefix caching then spreads it, because the corrupted state is
hash-reachable and every later request sharing that prefix restores from it.
The root cause is the find_longest_cache_hit method on MambaManager,
at vllm/v1/core/single_type_kv_cache_manager.py:1049 on main, which takes
drop_eagle_block in its signature and never acts on it. Its body searches
right-to-left for the last cached block, builds [null]*i + [cached], and
breaks, referencing neither drop_eagle_block nor .pop(.
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-27 10:45:10 +07:00
0b7f11a1ee
[AuxOutput] Add block-keyed storage for routed-expert outputs ( #45635 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-21 10:25:05 +00:00
Nick Hill
7dbc3d62c3
[Cleanup] Remove vestigial tpu_input_batch.py ( #56898 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-09-14 21:29:15 -07:00
fadfe1c7d4
[Bugfix][Core] Retire Mamba states across null gaps ( #55450 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-11 15:50:47 +08:00
+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00
Nick Hill and Claude Opus 4.8
615d4cfade
[Core] Check for GPU<->CPU syncs during CI ( #43107 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-14 17:42:05 -07:00
Nick Hill
d6f17f3d53
[MRV2] Support attention-free models ( #52374 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-14 23:54:50 +00:00
Nick Hill
9b0afeb4f6
[Perf] Avoid some more unnecessary GPU<->CPU syncs ( #51458 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-08 17:13:26 -07:00
f51193b9ae
[Kernel][Mamba] Fused-kernel support for align-mode DS-conv state migration with num_accepted_tokens > 1 ( #49291 )
...
Signed-off-by: Sungsoo Ha <sungsooh@nvidia.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-07-29 18:54:52 +08:00
cd0de48d08
[Bugfix][V1] Free out-of-window blocks on the processed-token basis under async scheduling ( #47728 )
...
Signed-off-by: Saddss <28726669061@qq.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Saddss <28726669061@qq.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-08 13:34:19 +01:00
Harry Mellor
e7d0fcbc09
[CI] Fix various failures on main ( #47197 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-01 10:35:34 +01:00
zhrrr
61ab70ec3b
[Model Runner V2] support mamba hybrid models align prefix cache ( #42406 )
...
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com >
2026-06-29 14:09:16 -07:00
Nick Hill
2263f8a3de
[CI][BugFix] Fix broken test_mamba_prefix_cache.py due to stale mock ( #45345 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-12 03:26:17 +00:00
Nick Hill
3b3d5287fa
[BugFix] Resolve multiple async kv load deadlock ( #44560 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-06 23:05:47 +00:00
Francesco Fusco and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
b730c46352
[Perf] [Hybrid] Fused Triton kernel for GPU-side Mamba state postprocessing ( #40172 )
...
Signed-off-by: Francesco Fusco <ffu@zurich.ibm.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-21 04:50:54 -07:00
roikoren755
737bfa3a43
[Bugfix][Hybrid][NemotronH] Fix mamba_cache_mode=all + speculative decoding crash ( #41233 )
...
Signed-off-by: Roi Koren <roik@nvidia.com >
2026-05-18 14:54:00 +03:00
Nick Hill
dd5506a157
[Core] Simplify handling of scheduler_reserve_full_isl option ( #41064 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-04-30 18:10:00 -07:00
wliao2
32e0c0bfa2
refactor hard coded device string in test files under tests/v1 and tests/lora ( #37566 )
...
Signed-off-by: Liao, Wei <wei.liao@intel.com >
2026-04-03 11:21:47 +08:00
Wentao Ye
c59a132f96
[V0 Deprecation] Refactor kv cache from list to element ( #37487 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-03-23 20:10:11 -07:00
Kevin H. Luu and Claude Opus 4.6
f1816fb192
[CI] Split V1 e2e + engine (1 GPU) into separate jobs ( #36945 )
...
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-13 14:16:02 -07:00