 Isotr0py
|
1c26e57d3c
|
[Bugfix] Raise for unavailable piecewise CUDA graphs (#54782)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-02 08:52:24 +00:00 |
|
|
Canlin Guoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
b383e16396
|
[Bugfix] Reset cached Mamba align metadata on profiling teardown (#54044)
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-30 00:43:20 -07:00 |
|
|
|
d3d79ffc1e
|
[Bugfix][Spec Decode] Capture the widest uniform decode batch by default (#50488)
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Rahul Chalamala <rahul@modal.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin <devin@cognition.ai>
|
2026-08-28 22:38:13 -07:00 |
|
|
Nick HillandKimi Code
|
96242aa50d
|
[Bugfix][MRV2] Release layer-bound KV cache memory in shutdown() (#54246)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
|
2026-08-28 21:27:09 +00:00 |
|
|
Qiu Chunshuo
|
b3af042abd
|
Bugfix: use PCP slot mappings for PIECEWISE capture (#53869)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
|
2026-08-27 05:39:36 +00:00 |
|
|
Wei Zhao
|
94d96e2446
|
[Bugfix] Release CUDA graph profiling memory before KV cache allocation (#53955)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
|
2026-08-27 01:35:59 +00:00 |
|
|
fanxingran
|
080a66a69c
|
[Bugfix][ROCm] Capture CUDA graphs on the current stream (#53818)
Signed-off-by: fanxingran <xingran.fan@amd.com>
|
2026-08-26 23:16:45 +08:00 |
|
|
Wentao Ye
|
a447955aca
|
[Kimi Bug] Fix k3 torch.AcceleratorError: CUDA error: an illegal memory access was encountered (#53773)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-08-25 17:38:48 -07:00 |
|
|
Nick Hill
|
e6cc0899b6
|
[Bugfix][MRV2] Run cudagraph memory profiling in a throwaway graph pool (#53682)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-25 18:52:15 +00:00 |
|
|
|
c74dd2d208
|
[Bugfix][MRV2] Dispatch uniform decode to a padded FULL cudagraph (#53407)
Signed-off-by: Xiaohu Guo <Xiaohu.Guo@amd.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: xiaohuguo2023 <149615094+xiaohuguo2023@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-08-25 06:59:38 +00:00 |
|
|
|
a4d70bef37
|
[Model Runner V2] Reserve CUDA graph memory (#53306)
Signed-off-by: Anh Tran <anh.tran.3889@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Anh Tran <anh.tran.3889@gmail.com>
Co-authored-by: anhtra3889 <anhtra@nvidia.com>
|
2026-08-23 23:11:19 -07:00 |
|
|
|
7f7a32cfec
|
[Spec Decode] DSpark confidence-scheduled verification (#47808)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-12 12:27:48 -07:00 |
|
|
Lucas WilkinsonandOpenAI Codex
|
02ac17851d
|
[Refactor][MRV2] Unify uniform decode token count helper (#51917)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-12 05:58:43 +00:00 |
|
|
|
0a94d85a66
|
[Bugfix][MRV2] Require all requests to be decoding for uniform-decode dispatch (#51865)
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
|
2026-08-11 17:15:06 -07:00 |
|
|
Nick HillandWoosuk Kwon
|
8112b6c997
|
[MRV2] Always build attn metadata at capture time (#49364) (#49995)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-07-27 17:25:17 +00:00 |
|
|
vllm-agentandvllm-agent CI bot
|
494845e79f
|
Revert "[MRV2] Always build attn metadata at capture time" (#49364) (#49451)
Co-authored-by: vllm-agent CI bot <ci-bot@vllm-agent.local>
|
2026-07-23 09:51:33 -07:00 |
|
|
Woosuk Kwon
|
6049424b7e
|
[MRV2] Always build attn metadata at capture time (#49364)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-07-21 19:27:16 -07:00 |
|
|
Nick HillandClaude Opus 4.8
|
de6ec294ef
|
[Bugfix] Fix DSA crash under breakable piecewise cudagraphs (#49302)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-07-21 18:28:13 +01:00 |
|
|
Ilya MarkovandMarkov Ilya
|
f890e1dbe2
|
[BugFix] Set graph_pool_id before FULL CUDA graph capture in ModelRunner V2 (#48843)
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>
|
2026-07-21 08:59:26 -04:00 |
|
|
|
251f7e478e
|
[Model] Add PW CUDA graph support for Inkling [2/N] (#48822)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-16 10:28:56 -07:00 |
|
|
Nick Hill
|
26587f9519
|
[BugFix][ModelRunner V2] Fix stale attn metadata in speculator prefill cudagraph capture (#48261)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-13 09:39:15 -07:00 |
|
|
|
07516fda67
|
[MRV2][SD] Make Dynamic SD comatible with Full Cuda Graphs (#45953)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-03 23:58:27 -04:00 |
|
|
Woosuk Kwonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
0a3e2dbc09
|
[Optimization] Skip DP padding tokens in MoE (#46428)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-23 14:54:46 -07:00 |
|
|
|
eacff17c8d
|
[Model Runner V2][Bugfix] Fix MRV2 LoRA warmup (#35536)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-06-15 13:17:23 -07:00 |
|
|
Giancarlo DelfinandWoosuk Kwon
|
91945b6e4a
|
[Bug Fix][Model Runner V2][Spec Decode] Warmup & capture with different attention states for speculator prefill (#44253)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-06-03 13:32:40 -07:00 |
|
|
Woosuk Kwon
|
e4a2e584e5
|
[MRV2] Remove assignment of graph_pool in cudagraph_utils (#44338)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-06-02 11:50:27 -07:00 |
|
|
Woosuk Kwon
|
27fa5aa3b9
|
[MRV2] Support breakable CUDA graph (#44050)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-05-30 09:40:52 -07:00 |
|
|
Nick Hill
|
39bba710be
|
[MRV2][BugFix] Fix default-stream CG capture in P/W LoRA case (#43160)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-19 19:19:05 -07:00 |
|
|
Giancarlo Delfin
|
3b6a204789
|
[Model Runner V2][Bug Fix][DSV4] Ensure lazy attention state initializations happen during cudagraph capture (#42444)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-05-14 16:16:17 -07:00 |
|
|
zhrrr
|
7a08b34fbf
|
[Model Runner V2] support qwen35 / mamba hybrid model (#35520)
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
|
2026-05-07 09:31:05 -07:00 |
|
|
Wentao Ye
|
526927be94
|
[Model Runner v2] Fix v2 compile counter num_gpu_runner_capture_triggers and num_cudagraph_captured (#41285)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-04-30 15:20:11 -07:00 |
|
|
Giancarlo Delfin
|
4c7c69b4e0
|
[Model Runner V2] Skip attention metadata rebuild before draft prefill (#40410)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-04-27 15:38:05 -07:00 |
|
|
Nick Hill
|
4e824d1c83
|
[Model Runner V2][Minor] Simplify PP logic (#38031)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-03-24 13:57:17 -07:00 |
|
|
zhanqiuhuandWoosuk Kwon
|
63f49b8bd4
|
[Model Runner V2] Enable piecewise CUDA graphs for pipeline parallelism (#35162)
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-22 20:48:25 +00:00 |
|
|
Woosuk Kwon
|
ce9b1d76cf
|
[MRV2] Skip hidden states allocation for PW CUDA graphs (#37818)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-22 11:47:21 -07:00 |
|
|
Santino Ramos
|
3ed46f374b
|
[Model Runner V2] Add Support for XD-RoPE (#36817)
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com>
|
2026-03-14 09:27:55 -07:00 |
|
|
Woosuk Kwon
|
55eed6b7a5
|
[Model Runner V2] Add WhisperModelState [6/N] (#35790)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-11 14:20:38 -07:00 |
|
|
Woosuk Kwon
|
f088a831dd
|
[Model Runner V2] Use unpadded num_tokens for PW CUDA graph attn metadata (#36626)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-10 09:30:56 -07:00 |
|
|
Woosuk Kwon
|
2a194ddd72
|
[Model Runner V2] Add model_state inputs to CUDA graph capture (#36544)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-09 15:14:51 -07:00 |
|
|
Lucas Wilkinson
|
483463f735
|
[MRV2] Extensible CG dispatch rework (#35959)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-03-09 13:58:45 -07:00 |
|
|
Woosuk Kwon
|
a0a5178ab4
|
[Model Runner V2] Use ModelState.prepare_attn() for cuda graph capture [5/N] (#35774)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-02 20:06:27 -08:00 |
|
|
Woosuk Kwon
|
72f4d16262
|
[Model Runner V2] Use block table apis for capture inputs (#35671)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-01 10:31:13 -08:00 |
|
|
Woosuk Kwon
|
e3eb146f7a
|
[Model Runner V2] Add ModelStateInterface [4/N] (#35621)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-02-28 13:19:45 -08:00 |
|
|
Woosuk Kwon
|
1a014a0a93
|
[Model Runner V2] Move MM encoder to Model States [3/N] (#35564)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-02-27 18:32:38 -08:00 |
|
|
Woosuk Kwon
|
c66aa48e99
|
[Model Runner V2] Add model states [1/N] (#35350)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-02-26 11:20:35 -08:00 |
|
|
Ming YangandMichael Goin
|
6831650c40
|
[offloader] v2: Hide weight onloading latency via prefetching (#29941)
Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-02-25 17:20:59 -08:00 |
|
|
Woosuk Kwon
|
2cbf9656ce
|
[Model Runner V2] Enable CUDA graph for Eagle3 (#35040)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-02-21 21:42:50 -08:00 |
|
|
Woosuk Kwon
|
b71fbd06e2
|
[Model Runner V2] Support attention group (#35036)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-02-21 16:42:53 -08:00 |
|
|
Woosuk Kwon
|
c878b43b64
|
[Model Runner V2] Remove unnecessary copies in PW CUDA graph capture (#34849)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-02-18 15:52:50 -08:00 |
|
|
zhrrr
|
11d3976b88
|
[Model Runner V2] support piecewise & mixed cudagraph (#32771)
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
|
2026-02-18 15:03:17 -08:00 |
|