 Wentao Ye
|
526927be94
|
[Model Runner v2] Fix v2 compile counter num_gpu_runner_capture_triggers and num_cudagraph_captured (#41285)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-04-30 15:20:11 -07:00 |
|
|
Wentao YeandNick Hill
|
51fda1ba44
|
[Model Runner v2] Fix block table IMA issue (#40648)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-04-29 08:30:33 -07:00 |
|
|
Giancarlo Delfin
|
4c7c69b4e0
|
[Model Runner V2] Skip attention metadata rebuild before draft prefill (#40410)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-04-27 15:38:05 -07:00 |
|
|
Giancarlo Delfin
|
f5f9878514
|
[Model Runner V2] Fix rejection sampling acceptance rate gap vs MRV1 (#40651)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-04-26 19:12:08 -07:00 |
|
|
+7
|
4d51588e23
|
[Feat] DeepSeek V4 Rebased (#40860)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
|
2026-04-26 18:31:08 -07:00 |
|
|
Wentao Ye
|
a474da2813
|
[Refactor] Remove unused dead code (#40640)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-04-25 07:28:18 +08:00 |
|
|
Nick Hill
|
e9f331d72e
|
[MRV2] Ensure warmup covers prefill path (#40746)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-04-24 01:33:26 +00:00 |
|
|
Benjamin Chislettandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
626daa2076
|
[Feat] Unified Synthetic Acceptance Rate for V1 and V2 (#40662)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-04-24 00:48:08 +00:00 |
|
|
Nick Hill
|
fe85a92e86
|
[Core] Avoid seq_lens_cpu GPU->CPU sync (#40654)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-04-24 00:35:55 +00:00 |
|
|
Wentao Yeandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
8f87eb4622
|
[Refactor] Clean up log once scope="local" (#40540)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-04-22 16:42:43 -04:00 |
|
|
Zijing Liu
|
9a6a66f3b8
|
[MRv2]fix: model accuracy regression caused by reusing the stale last_sampled_tokens and draft_tokens (#39833)
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
|
2026-04-21 16:30:32 +00:00 |
|
|
Wentao YeandNick Hill
|
66cc3fa559
|
[Model Runner V2] Multiple prompt logprobs support (#39937)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-04-21 15:49:05 +00:00 |
|
|
roikoren755
|
b897f00c9c
|
Gate SSU dispatch setup (#40039)
Signed-off-by: Roi Koren <roik@nvidia.com>
|
2026-04-16 13:06:01 -07:00 |
|
|
Giancarlo Delfin
|
343f65234b
|
[Model Runner V2][BugFix] fix num_sampled dtype for probabilistic rej… (#39951)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-04-15 18:09:11 -07:00 |
|
|
Giancarlo Delfin
|
3bfe55a037
|
[Model Runner V2] Disable piecewise cudagraph mode fallback for eagle draft decodes (#39773)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-04-14 17:47:57 -07:00 |
|
|
roikoren755
|
ecf5ff7ce3
|
[Mamba] Flashinfer selective_state_update (#36162)
Signed-off-by: Roi Koren <roik@nvidia.com>
|
2026-04-14 15:10:58 -04:00 |
|
|
Giancarlo Delfin
|
ccf90ba784
|
[Model Runner V2] Add full cuda graph support for eagle prefill (#37588)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-04-13 16:01:24 -07:00 |
|
|
zhrrr
|
8f121f7879
|
[Model Runner V2] support auto resolve cudagraph mode/sizes based on attn backend (#32936)
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
|
2026-04-10 08:27:15 -07:00 |
|
|
Giancarlo Delfin
|
5daf62271d
|
[Model Runner V2] Fuse probabilistic rejection sample kernels (#38496)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-04-07 17:37:37 -07:00 |
|
|
Nick Hill
|
62095e82c1
|
[BugFix][MRV2] Fix cuda event reuse race (#39115)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-04-07 00:21:09 +00:00 |
|
|
Woosuk Kwon
|
f186cfe75e
|
[MRV2] Fix hanging issue with DeepSeek V3.2 by setting skip_attn=False (#39098)
Signed-off-by: WoosukKwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-04-06 12:55:13 -07:00 |
|
|
Yifan Qiao
|
311c981647
|
[MRV2][KVConnector] Fix missing build_connector_worker_meta (#38698)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-04-03 08:42:52 +03:00 |
|
|
wang.yuqi
|
a9b4f07ba2
|
[Frontend] Re-enable running MaxSim on GPU (#38620)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-04-03 00:03:13 +08:00 |
|
|
Lukas Geiger
|
4f6eed3bd4
|
[Core] Simplify multimodal masking (#34246)
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
|
2026-04-01 01:18:22 -07:00 |
|
|
Wentao Ye
|
856589ed9a
|
[Refactor] Remove dead code in kv connector and model runner (#38383)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-03-31 17:05:23 -04:00 |
|
|
Giancarlo Delfin
|
384e4d5f48
|
[Model Runner V2] Rebuild attention metadata before eagle decode full… (#38311)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-03-27 13:46:42 -07:00 |
|
|
Giancarlo Delfin
|
c32e97602d
|
[Model Runner V2] Enable forcing a specific acceptance rate during rejection sampling (#38045)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-03-26 13:38:12 -07:00 |
|
|
Woosuk KwonandNick Hill
|
144030c84e
|
Relocate Encoder CUDA graph manager (#38116)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-03-25 20:52:12 -07:00 |
|
|
Wentao YeandWoosuk Kwon
|
d7e93e13fb
|
[Feature] EPLB Support for GPU Model Runner v2 (#37488)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-25 08:16:39 -07:00 |
|
|
Woosuk Kwon
|
4b53740d7f
|
[MRV2] Fix for DS v3.2 (#38030)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-24 14:03:24 -07:00 |
|
|
Nick Hill
|
4e824d1c83
|
[Model Runner V2][Minor] Simplify PP logic (#38031)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-03-24 13:57:17 -07:00 |
|
|
Giancarlo Delfin
|
8f4824b664
|
[Model Runner V2] Gather multimodal embeddings before draft model postprocess (#37932)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-03-23 18:14:13 -07:00 |
|
|
Woosuk KwonandNick Hill
|
ffb5b32b5f
|
[MRV2] Consider spec decoding in warmup (#37812)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-03-23 17:45:43 +00:00 |
|
|
Baorun (Lauren) Mu
|
f85e479e66
|
[Feature] ViT Full CUDA Graph (#35963)
Signed-off-by: Baorun Mu <bmu@nvidia.com>
|
2026-03-23 13:01:10 +08:00 |
|
|
zhanqiuhuandWoosuk Kwon
|
63f49b8bd4
|
[Model Runner V2] Enable piecewise CUDA graphs for pipeline parallelism (#35162)
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-22 20:48:25 +00:00 |
|
|
Woosuk Kwon
|
a5e9d511de
|
[MRV2] Use FP64 for Gumbel noise (#37798)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-22 12:28:10 -07:00 |
|
|
Woosuk Kwon
|
ce9b1d76cf
|
[MRV2] Skip hidden states allocation for PW CUDA graphs (#37818)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-22 11:47:21 -07:00 |
|
|
Giancarlo DelfinandWoosuk Kwon
|
b3e846017d
|
[Model Runner V2] Support multi-modal embeddings for spec decode model (#36097)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-22 02:48:43 -07:00 |
|
|
Santino Ramos
|
85f671b8e1
|
[Model Runner V2] Support Streaming Inputs (#37028)
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com>
|
2026-03-20 20:42:25 +00:00 |
|
|
Woosuk Kwon
|
e80cfe575d
|
[MRV2] Avoid recompilation of _gather_block_tables_kernel (#37645)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-20 10:31:45 -07:00 |
|
|
Giancarlo Delfin
|
dcee9be95a
|
[Model Runner V2] Fix draft logits not populated during cudagraph replay (#37639)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-03-20 07:43:47 +00:00 |
|
|
Giancarlo Delfin
|
39474513f6
|
[Model Runner V2] fix draft attention metadata generation (#37364)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-03-19 21:05:15 -07:00 |
|
|
Woosuk Kwon
|
40b8363b45
|
[MRV2] Use fp32 for draft logits (#37526)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-19 08:41:21 -07:00 |
|
|
Giancarlo Delfin
|
053f3b6309
|
[Model Runner V2] Spec decode rejection sampler logprobs support (#37237)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-03-19 01:36:27 +00:00 |
|
|
Giancarlo Delfin
|
04244fd0e1
|
[Model Runner V2] Spec decode rejection sampler greedy support (#37238)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-03-18 15:59:03 -07:00 |
|
|
Or OzeriandNicolò Lucchesi
|
fcf0687b27
|
[kv_offload+HMA][0/N]: Support block-level preemption handling (#34805)
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
|
2026-03-18 08:49:53 +02:00 |
|
|
Woosuk Kwon
|
96efb91480
|
[Model Runner V2] Fix processed logits in sample() (#37144)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-16 00:35:49 -07:00 |
|
|
Santino Ramos
|
3ed46f374b
|
[Model Runner V2] Add Support for XD-RoPE (#36817)
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com>
|
2026-03-14 09:27:55 -07:00 |
|
|
sbeurnier
|
a116f96930
|
[V1] Remove pin_memory() in async_copy_to_gpu to fix sporadic stalls (#37006)
Signed-off-by: Sebastien Beurnier <sbeurnier@together.ai>
|
2026-03-14 01:37:32 +00:00 |
|
|
Benjamin Chislett
|
8b346309a5
|
[Refactor] Consolidate SupportsEagle (#36063)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-03-13 23:22:40 +00:00 |
|