 Andreas Karatzas
|
61c1d098e5
|
[CI] Stabilize speculator memory teardown (#50284)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
|
2026-07-30 18:42:55 +08:00 |
|
|
|
f51193b9ae
|
[Kernel][Mamba] Fused-kernel support for align-mode DS-conv state migration with num_accepted_tokens > 1 (#49291)
Signed-off-by: Sungsoo Ha <sungsooh@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-07-29 18:54:52 +08:00 |
|
|
Zach Zhu
|
58f9659397
|
[Frontend][Core] Standardize request error handling with VLLMError hierarchy (#49665)
Signed-off-by: Zach Zhu <zzqshu@126.com>
|
2026-07-29 04:20:40 +00:00 |
|
|
Andreas Karatzas
|
7398a30d79
|
[ROCm][CI] Stabilize ngram and suffix correctness test (#50190)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
|
2026-07-28 20:29:00 -06:00 |
|
|
Giancarlo Delfin
|
272abd5f48
|
[Tests][Spec Decode] Add gemma4 MTP acceptance rates test (#47920)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-07-27 16:15:54 -07:00 |
|
|
|
866fea2b99
|
[Kernel] ReplaySSM: cache SSM inputs for faster Mamba2 standard decode (#48018)
Signed-off-by: Johnny-Liou <a897111@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-07-24 09:39:49 -07:00 |
|
|
Stefano CastagnettaandBenjamin Chislett
|
5812e1a66b
|
[Test] Add DeepSeek MTP parallel-load tests (#41653)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-07-21 10:38:20 -04:00 |
|
|
Andreas Karatzas
|
df362b2d6d
|
[ROCm][CI] Ensure sliding window tests release GPU memory (#49055)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-07-18 20:44:05 +00:00 |
|
|
Nicolò LucchesiandClaude Opus 4.8
|
5784507da4
|
[Attention] Allow selecting a different attention backend per KV-cache group (#48012)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-07-17 15:19:02 -04:00 |
|
|
Hangrui CaoandDiegoCao
|
4a394bfcda
|
[Spec Decode][DSpark] Add Gemma4-12B DSpark draft model (#47216)
Signed-off-by: DiegoCao <DiegoCao@users.noreply.github.com>
Co-authored-by: DiegoCao <DiegoCao@users.noreply.github.com>
|
2026-07-16 21:51:47 +00:00 |
|
|
|
88e5e2c57b
|
[CI/Build][AMD] Fix ROCm OOM in eagle_correctness_heavy by reserving CUDA graph memory (#47366)
Signed-off-by: pei.zhang <pei.zhang@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-07-10 02:14:38 +00:00 |
|
|
|
cd0de48d08
|
[Bugfix][V1] Free out-of-window blocks on the processed-token basis under async scheduling (#47728)
Signed-off-by: Saddss <28726669061@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Saddss <28726669061@qq.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-08 13:34:19 +01:00 |
|
|
BadrBasowid
|
740f379fae
|
[ROCm][AITER] Directly Implement AITER Custom All-reduce in CudaCommunicator (#46065)
Signed-off-by: BadrBasowid <badr.basowid@gmail.com>
|
2026-07-06 12:16:32 +00:00 |
|
|
Liangliang MaandKunshang Ji
|
69715823df
|
[Test][XPU] Skip fork in kv_sharing_fast_prefill test on XPU (#47406)
Signed-off-by: Ma, Liangliang <liangliang.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-07-06 11:32:26 +08:00 |
|
|
adamkbaranowski
|
4c3c64fcf7
|
Add Laguna XS.2.1 DFlash drafter support (#46853)
Signed-off-by: Adam Baranowski <adam.baranowski@poolside.ai>
|
2026-07-02 18:09:27 -07:00 |
|
|
|
f5a8d73377
|
[Spec Decode] DSpark (#46995)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-07-01 08:30:24 -07:00 |
|
|
Harry Mellor
|
e7d0fcbc09
|
[CI] Fix various failures on main (#47197)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-01 10:35:34 +01:00 |
|
|
|
727971f1c1
|
Add Medusa speculative decoding e2e test (#41396)
Signed-off-by: Anshika Ojha <anshikao@nvidia.com>
Signed-off-by: Rishi Puri <riship@nvidia.com>
Signed-off-by: Rishi Puri <puririshi98@berkeley.edu>
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Anshika Ojha <215760622+ojhaanshika@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Stefano Castagnetta <scastagnetta@nvidia.com>
|
2026-06-30 18:02:22 +00:00 |
|
|
zhrrr
|
61ab70ec3b
|
[Model Runner V2] support mamba hybrid models align prefix cache (#42406)
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
|
2026-06-29 14:09:16 -07:00 |
|
|
|
1aad125815
|
[CPU] Enable chunked prefill and prefix caching for qwen3.5 (#46202)
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-06-25 03:49:21 +00:00 |
|
|
Liangliang Maandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2801b11156
|
[Test] Pin block_size in auto-fit max_model_len test (#45914)
Signed-off-by: Ma, Liangliang <liangliang.ma@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-24 22:56:21 +08:00 |
|
|
Matt
|
a19ff2218a
|
[Hardware][AMD][CI] Fix Spec Decode Eagle test group (#46018)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
|
2026-06-21 17:40:02 -05:00 |
|
|
Mattandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
dced290769
|
[Hardware][AMD][CI] Fix e2e core test group (#46024)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-20 02:04:35 -05:00 |
|
|
Yejing Lai
|
9872921c5f
|
[XPU] skip UT test_with_ngram_gpu_spec_decoding (#44423)
Signed-off-by: Lai, Yejing <yejing.lai@intel.com>
|
2026-06-15 08:46:30 +00:00 |
|
|
Nick Hill
|
2263f8a3de
|
[CI][BugFix] Fix broken test_mamba_prefix_cache.py due to stale mock (#45345)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-12 03:26:17 +00:00 |
|
|
Benjamin ChislettandGiancarlo Delfin
|
0bae1d3848
|
[MRV2][Spec Decode] DFlash (#44586)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-06-10 08:47:46 -07:00 |
|
|
Nick Hill
|
3b3d5287fa
|
[BugFix] Resolve multiple async kv load deadlock (#44560)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-06 23:05:47 +00:00 |
|
|
haosdentandWentao Ye
|
5ea76fa89a
|
[CI] Fix test_lora_with_spec_decode on V2 model runner (#43314)
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-05-22 14:24:18 +08:00 |
|
|
Francesco Fuscoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
b730c46352
|
[Perf] [Hybrid] Fused Triton kernel for GPU-side Mamba state postprocessing (#40172)
Signed-off-by: Francesco Fusco <ffu@zurich.ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-21 04:50:54 -07:00 |
|
|
Benjamin Chislett
|
c628a93a64
|
[Perf][Bugfix] Update dflash aux layer indexing (#40727)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-05-19 20:15:57 -07:00 |
|
|
roikoren755
|
737bfa3a43
|
[Bugfix][Hybrid][NemotronH] Fix mamba_cache_mode=all + speculative decoding crash (#41233)
Signed-off-by: Roi Koren <roik@nvidia.com>
|
2026-05-18 14:54:00 +03:00 |
|
|
Nick Hill
|
fe8b42e80c
|
[CI] Fix test_async_scheduling.py flakiness (#42455)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-12 21:38:32 +00:00 |
|
|
Kevin H. Luu
|
1ff9d33535
|
[CI] Migrate remaining B200 jobs to b200-k8s with test fixes (#42387)
Signed-off-by: khluu <khluu000@gmail.com>
|
2026-05-12 02:00:37 -07:00 |
|
|
Lucas Wilkinson
|
b1728c1e66
|
[Attention][Cleanup] Remove tree attention (#42121)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-05-08 18:36:19 -07:00 |
|
|
Luciano MartinsandLuciano Martins
|
27e0057aed
|
[Spec Decode] Add Gemma4 MTP speculative decoding support (#41745)
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
|
2026-05-06 22:39:29 +08:00 |
|
|
Joachim StudniaandClaude Opus 4.6
|
422dd02598
|
[bugfix] Fix prompt logprobs on request eviction during chunked prefill (#41411)
Signed-off-by: Joachim Studnia <joachim@mistral.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-05-04 11:46:00 -07:00 |
|
|
Nick Hill
|
dd5506a157
|
[Core] Simplify handling of scheduler_reserve_full_isl option (#41064)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-04-30 18:10:00 -07:00 |
|
|
Stefano Castagnettaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
92a7c121b6
|
[CI] Add MTP coverage: Qwen3.5 correctness + no-sync spec decode (#40472)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-04-30 12:24:09 -07:00 |
|
|
Nick Hill
|
e68fa1b90a
|
[Core] Account for num_gpu_blocks_override in max_model_len checks (#41069)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-04-28 15:44:09 -07:00 |
|
|
Benjamin Chislettandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
626daa2076
|
[Feat] Unified Synthetic Acceptance Rate for V1 and V2 (#40662)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-04-24 00:48:08 +00:00 |
|
|
Matthew BonanniandTyler Michael Smith
|
96a85c5750
|
[Startup][UX] Enable CUDAGraph memory profiling by default (#38284)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-04-21 18:16:59 -04:00 |
|
|
|
03f8d3a548
|
Update to transformers v5 (#30566)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: khluu <khluu000@gmail.com>
Signed-off-by: Kevin H. Luu <khluu000@gmail.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
|
2026-04-15 16:29:15 -07:00 |
|
|
Yan Ma
|
ec68d53b2b
|
Add platform manual_seed_all API (#38468)
Signed-off-by: Yan Ma <yan.ma@intel.com>
|
2026-04-10 13:43:50 +08:00 |
|
|
Wentao Ye
|
3352bf8b03
|
[CI Bug] Fix pre-commit issue in main (#39347)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-04-08 14:10:05 -07:00 |
|
|
triangleXIVandmgoin
|
7c94ae16c6
|
[BugFix] --max-model-len=-1 causes over-limit requests to hang and starve the entire service (#39102)
Signed-off-by: triangle14 <y1019026570@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-04-08 14:03:17 -07:00 |
|
|
|
ad05edfbca
|
tests/v1/e2e/spec_decode: assert async scheduling is used (#39206)
Signed-off-by: Rishi Puri <riship@nvidia.com>
Signed-off-by: Rishi Puri <puririshi98@berkeley.edu>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-04-08 20:30:03 +00:00 |
|
|
wliao2
|
32e0c0bfa2
|
refactor hard coded device string in test files under tests/v1 and tests/lora (#37566)
Signed-off-by: Liao, Wei <wei.liao@intel.com>
|
2026-04-03 11:21:47 +08:00 |
|
|
Benjamin Chislett
|
494636b29d
|
[Feat][Spec Decode] DFlash (#36847)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-03-30 15:03:15 -04:00 |
|
|
Andreas Karatzas
|
4f2ed5fddb
|
[ROCm][CI] Enable hybrid chunked prefill test (#38317)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-03-30 10:30:26 +08:00 |
|
|
Andrii SkliarandAndrii Skliar
|
cd7643015e
|
[Feature] Support per-draft-model MoE backend via --speculative-config (#37880)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: [Andrii Skliar] <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
|
2026-03-25 14:31:52 +00:00 |
|