 Jiangyun Zhu
|
98ed0856f3
|
[Model] add GLM-5.3-Flash support (#53906)
|
2026-09-03 09:40:35 -07:00 |
|
|
Derek WeitzelandClaude Fable 5.1
|
859dd39512
|
[Bugfix][Core] Wait for the previous PP tensor sends before the next forward pass (#54962)
Signed-off-by: Derek Weitzel <djw8605@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-03 08:01:36 +00:00 |
|
|
Andrii SkliarandAndrii Skliar
|
adebc41b7e
|
[Mamba] Add FlashInfer ReplaySSM backend (#52506)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
|
2026-09-01 10:26:43 -04:00 |
|
|
|
4ac452ad98
|
[Core] Release NCCL communicator memory in sleep mode (#51485)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-01 14:31:16 +08:00 |
|
|
Roberto L. CastroandCodex
|
e16b5e518d
|
[1/N][warmup][DSv4] Migrate generic MLA metadata and indexing kernels (#50175)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-08-31 23:26:05 -04:00 |
|
|
AusarandClaude
|
6bafc049aa
|
[Bugfix][PP] Never drop a decoding request from the sampled-token broadcast (#54436)
Signed-off-by: ArcheyChen <45146080+ArcheyChen@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-31 23:49:54 +00:00 |
|
|
Artem Perevedentsev
|
39e276eaeb
|
[Structured Output] Let terminal grammars stop under min_tokens (II) (#54218)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
|
2026-08-31 20:39:27 +04:00 |
|
|
+5
|
e126687a9a
|
[Model] Support Qwen3.8-Flash-Next (#53896)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: KungYork <firelikest@163.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-31 05:57:56 +00:00 |
|
|
Canlin Guoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
b383e16396
|
[Bugfix] Reset cached Mamba align metadata on profiling teardown (#54044)
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-30 00:43:20 -07:00 |
|
|
Giancarlo Delfin
|
fe755c8899
|
[Bugfix][Model Runner V2][Spec Decode] Decouple the draft's gumbel noise stream from the target's (#54282)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-08-29 09:58:29 -07:00 |
|
|
Nicolò Lucchesi
|
7f4793eaa3
|
[Nixl][PD] DCP support for MLA models (#50611)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-08-29 16:03:53 +00:00 |
|
|
Summer YangandNick Hill
|
2aac565cae
|
[Core][KV Connector] Start async KV loads after the forward launch when no sync loads are scheduled (#53333)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-28 21:58:00 +00:00 |
|
|
stefankoncarevic
|
9236159bb6
|
[ROCm][CI] Keep startup profiling from aborting when free memory grows (#53591)
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
|
2026-08-28 00:02:40 -07:00 |
|
|
Ronaldandaoshen02
|
479eeb32d2
|
[Bugfix][MRV2] Isolate sleep-mode KV allocations (#53508)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-08-27 15:08:28 -07:00 |
|
|
Giancarlo Delfin
|
5acc1c4e4b
|
[Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft prefill (#53694)
|
2026-08-26 20:59:35 -07:00 |
|
|
Wei Zhao
|
94d96e2446
|
[Bugfix] Release CUDA graph profiling memory before KV cache allocation (#53955)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
|
2026-08-27 01:35:59 +00:00 |
|
|
|
ee5ae2ae70
|
[RL] Add rank-local IPC weight updates (#52497)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-08-26 15:36:57 +08:00 |
|
|
|
b1fbbc2ade
|
BugFix(PCP): use persistent input buffers for PIECEWISE CUDA graphs (#53515)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-26 05:58:13 +00:00 |
|
|
Nick Hill
|
e6cc0899b6
|
[Bugfix][MRV2] Run cudagraph memory profiling in a throwaway graph pool (#53682)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-25 18:52:15 +00:00 |
|
|
Misha Goin
|
7de96050c5
|
[Spec Decode] Enable adaptive DSpark on SM100 sparse MLA (#52783)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-08-25 13:31:15 -04:00 |
|
|
Naveenraj KamalakannanandBenjamin Chislett
|
d5cadcee86
|
[Feature][DSpark]: Logprobs adaptive verification (#52242)
Signed-off-by: Naveenraj Kamalakannan <therealnaveenkamal@gmail.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-08-25 10:22:33 -04:00 |
|
|
Elvir Crnčević
|
afc91fa0d2
|
[Profiler] Fix start_profile permanently no-op after max_iterations auto-stop (#51839)
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
|
2026-08-25 08:13:29 -04:00 |
|
|
Giancarlo Delfin
|
d154d90d6c
|
[Model Runner V2] batch-sharded sample (#50465)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-08-24 15:49:57 -07:00 |
|
|
Taneem Ibrahim
|
79bb395eea
|
[Pooling] Improve BGE-M3 sync pooling throughput by up to 3.13% (#53464)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-08-24 08:33:32 -04:00 |
|
|
Nicolò Lucchesi
|
0ecc284790
|
[Bugfix][Kernel] Handle kernel block sizes in V2 DCP slot mapping (#51031)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-08-24 07:28:38 +00:00 |
|
|
|
a4d70bef37
|
[Model Runner V2] Reserve CUDA graph memory (#53306)
Signed-off-by: Anh Tran <anh.tran.3889@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Anh Tran <anh.tran.3889@gmail.com>
Co-authored-by: anhtra3889 <anhtra@nvidia.com>
|
2026-08-23 23:11:19 -07:00 |
|
|
Tianyu Guo
|
7ca49fbe4b
|
[Refactor][Model Runner V2][Multimodal] Move the encoder-only path out of the shared runner (#53176)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
|
2026-08-22 05:11:44 +00:00 |
|
|
Lucas Wilkinson
|
8bdc70ec7b
|
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-08-21 17:47:49 -07:00 |
|
|
|
01af92e175
|
[Feature][Model Runner V2] Support extract_hidden_states speculation (#49811)
Signed-off-by: Zupeng Wang <71580390+zupengwang@users.noreply.github.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI <support@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-20 11:13:43 -04:00 |
|
|
|
a1c5b1fd9f
|
[Core][V1] Support trace_decode_token_ids for deterministic decode replay (#46701)
Signed-off-by: ianzxliu <ianzxliu@tencent.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Zhixia Liu <liuzhixia.nju@gmail.com>
Co-authored-by: ianzxliu <ianzxliu@tencent.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-20 15:12:43 +08:00 |
|
|
Giancarlo Delfin
|
d4f4d3f40f
|
[Model Runner V2][Spec Decode] Fix draft logits cache column stride in gumbel_sample (#53017)
|
2026-08-19 17:39:30 -07:00 |
|
|
Willow Lopez
|
f936a267f9
|
[Bugfix][XPU] Fix Mamba state pointer overflow (#48109)
Signed-off-by: Oxygen56 <jiangth99@163.com>
|
2026-08-19 10:45:41 +08:00 |
|
|
Itay Alroy
|
a2257f95b7
|
[Elastic EP] Reduce eager-mode reconfiguration downtime (#51885)
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
|
2026-08-19 02:13:06 +00:00 |
|
|
|
0db502c8d8
|
[Kimi-K3] support DCP partial prefix cache hit (#50493)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-08-17 17:05:51 -07:00 |
|
|
Canlin GuoandNick Hill
|
455edc022b
|
[ModelRunnerV2] Support prompt embeds (#42963)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Canlin Guo <961750412@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-17 21:58:50 +00:00 |
|
|
Roberto L. CastroandCodex
|
49905ad94d
|
[3/N][Feat][Perf] Add new warmup infrastructure for JITs. Add provider registry and orchestration for JIT warmup (#50174)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-08-17 10:11:14 -07:00 |
|
|
|
70afdedc10
|
[K3] support recoverssm for K3 (#51855)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-17 19:34:14 +08:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
a02cfccbc6
|
[Bugfix][Mamba] Fix overlapping state copy race (#50729)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-17 08:29:41 +02:00 |
|
|
Cheng RuiandRui Rui4 Cheng
|
6664d397bf
|
[Performance][MRV2] Cache logits-processing request state (#52329)
Signed-off-by: Rui Rui4 Cheng <chengrui4@lenovo.com>
Co-authored-by: Rui Rui4 Cheng <chengrui4@lenovo.com>
|
2026-08-17 02:36:41 +00:00 |
|
|
|
1b079c40ff
|
[Bugfix][Model Runner V2][Spec Decode] Fix off-by-one in bad_words draft-prefix matching (#52311)
Signed-off-by: jyan <r_02213@sjtu.edu.cn>
Co-authored-by: jyan <r_02213@sjtu.edu.cn>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-16 09:02:20 +00:00 |
|
|
|
fa9d67f782
|
[EC Connector] Added Build Connector Worker Meta for EC Connector (#49585)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-08-15 15:40:38 -07:00 |
|
|
Gabriel Wu
|
97388c44f9
|
[Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark (#51538)
|
2026-08-15 07:01:32 -07:00 |
|
|
James E T SmithandAndreas Karatzas
|
9b0ab5dd53
|
[ROCm][AMD] Enable preshuffled sparse indexing for 16-token blocks (#51216)
Signed-off-by: jamesETsmith <james.smith9113@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-08-14 12:38:14 -05:00 |
|
|
Lucas WilkinsonandClaude
|
57bd0ed441
|
[5/N][KV-Cache Layout Refactor] Backend-published KV packing via customize_spec (#51704)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-14 09:47:55 -04:00 |
|
|
|
95c9144424
|
[LoRA][Gemma4] Support vision tower LoRA (#42662)
Signed-off-by: bk-201 <joy25810@foxmail.com>
Signed-off-by: linitra24 <Joy25810@foxmail.com>
Signed-off-by: linitra24 <joy25810@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: linitra24 <joy25810@gmail.com>
|
2026-08-13 22:07:43 +08:00 |
|
|
Lucas WilkinsonandOpenAI Codex
|
79f3183f86
|
[Bugfix] Bound KV block zeroing launch geometry (#52058)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-13 03:42:26 +00:00 |
|
|
|
7f7a32cfec
|
[Spec Decode] DSpark confidence-scheduled verification (#47808)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-12 12:27:48 -07:00 |
|
|
|
0a94d85a66
|
[Bugfix][MRV2] Require all requests to be decoding for uniform-decode dispatch (#51865)
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
|
2026-08-11 17:15:06 -07:00 |
|
|
鐘天楽
|
a367cdbca0
|
[Profiler] Add minimal Triton Proton profiling backend (#48789)
Signed-off-by: Luossu <zhluosuu@outlook.com>
|
2026-08-11 14:51:57 -07:00 |
|
|
stefankoncarevic
|
3e372c5ff2
|
[Bugfix][ROCm] Give KV-first attention blocks their own page in hybrid models (#51837)
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
|
2026-08-11 16:52:30 -04:00 |
|