 ![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)
|
763e3f7c91
|
[Bugfix][Spec Decode] Disable dynamic speculative decoding for fixed-K proposers instead of crashing (#46399)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-10-17 12:41:04 -04:00 |
|
Wei Zhao
|
90d6d871d6
|
[Attention] Add CuTe DSL backend selection for MLA context prefill (#62069)
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
|
2026-10-17 01:21:55 -04:00 |
|
 
|
b544587408
|
[Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning (#53558)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-16 23:16:47 +00:00 |
|
 maithilijoshi20andCyrus Leung
|
92dc3dd393
|
[Bugfix] Honor OPT token embedding width for prompt embeddings (#61736)
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-16 04:41:50 +00:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)       
|
dd9e07a25f
|
[Feature] Add External LB Mode Support for Elastic EP Scaling (#43202)
Signed-off-by: lqh-hh <liuqihang6@huawei.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: lqh-hh <liuqihang6@huawei.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: 2017403603 <705328646@qq.com>
Co-authored-by: Julien Mancuso <jmancuso@nvidia.com>
Co-authored-by: Itay Alroy <75032521+itayalroy@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-15 21:29:02 -04:00 |
|
 Andreas KaratzasandClaude
|
29345f42a6
|
[ROCm][Qwen4Exp] Support HC and MoE sequence parallelism on ROCm (#61471)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-15 20:11:17 -05:00 |
|
 
|
de7d70676d
|
[Perf] Low-latency CPU all-reduce for DP metadata syncs (#61767)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: hjjq <50634613+hjjq@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-15 15:29:13 -07:00 |
|
 Stefano CastagnettaandKevin H. Luu
|
784d1fff0c
|
[MRV2][Spec Decode] Quantized draft lm_head for drafters that share the target head (#59973)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 22:06:40 +00:00 |
|
   
|
d92a4f25f2
|
[watermarking] Add DFlash support to dual-key gumbel-max watermarking (#59554)
Signed-off-by: Shern Shiou Tan <shernshiou@gmail.com>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-10-15 22:06:06 +00:00 |
|
     
|
e711723352
|
[ROCm] gfx1250 optimizations and model enablement (#58800)
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: Juan Villamizar <jpvillam@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
|
2026-10-15 14:51:57 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
7cd7982dfd
|
[Spec Decode] Don't reserve DFlash/DSpark draft slots in the scheduler token budget (#59468)
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-16 01:23:58 +08:00 |
|
  
|
ed3050add4
|
[Core] Apply prefill_schedule_interval outside data parallelism (#54627)
Signed-off-by: lifulu <fululi12@amd.com>
Signed-off-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-10-15 11:03:13 -05:00 |
|
 
|
24f632ca7e
|
[Bugfix] Engage indexer TP row sharding on total rows, not per-rank rows (#61647)
Signed-off-by: Zheng Cai <caizheng1993@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
|
2026-10-15 13:54:14 +00:00 |
|
 TimofeyandCodex
|
716c5225ae
|
[Model] Add AliceAI support (#57963)
Signed-off-by: timka <timka.byzov@yandex.ru>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-15 10:15:03 +00:00 |
|
 Misha GoinandClaude Opus 5.5
|
1c4c319792
|
[Frontend] Advertise --enable-x/--disable-x for boolean CLI flags (#61479)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 01:08:02 -07:00 |
|
 
|
48f17b77bb
|
[Bugfix] Reject mismatched float16/bfloat16 KV cache dtype (#60885)
Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 08:06:59 +00:00 |
|
Shanshan Shen
|
f46021a7fa
|
[ROCm][Perf] Enable DPA+ETP prefill compute/comm dual-batch-overlap for DeepSeek-V4 (#57773)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
Signed-off-by: shen-shanshan <467638484@qq.com>
|
2026-10-15 11:16:48 +08:00 |
|
 Woosuk KwonandClaude Opus 5.5
|
8530aed2b7
|
[Spec Decode] Remove Medusa and MLP speculator support (#61701)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 17:29:23 -07:00 |
|
 Luca MotzandClaude Opus 5.5
|
2d09fdcdc2
|
[Quantization] Auto-disable DeepGEMM for Kolibri 1 on SM12x (#61274)
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 12:59:50 +00:00 |
|
Sherif Waly
|
be4ce2e505
|
[Bugfix] Share PCP-aware local GPU indexing for NUMA binding (#61237)
|
2026-10-14 10:54:03 +02:00 |
|
  
|
3ccc0a90b6
|
[KV Connector] Support sleep mode with MooncakeConnector over RDMA (#59625)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Dakai An <77474977+andakai@users.noreply.github.com>
|
2026-10-14 04:18:47 +00:00 |
|
 
|
c068db0e1d
|
[Bugfix][Compile] Replace Transformers AOT caching with graph caching (#58918)
Signed-off-by: fangyao fan <2281693949@qq.com>
Signed-off-by: Taimys <ffy2281693949@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Taimys <ffy2281693949@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-10-13 19:17:29 +00:00 |
|
 
|
de8291e24f
|
[Core] Add lifecycle hooks for worker extensions (#59463)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-13 21:51:05 +08:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
2cc2e68d1c
|
[Core][Mamba] Remove leftovers of mamba_cache_mode="all" (#60043)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-13 00:58:59 -07:00 |
|
 Fangzhou AiandClaude Opus 5.5
|
cc2a6eea1a
|
[ROCm][Perf][DSv4.1] Mono decode on every backbone layer (#61082)
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-13 10:16:50 +08:00 |
|
 Maroon AyoubandClaude
|
d6aca837c3
|
[Attention] Make the sparse MLA prefill workspace size configurable (#61023)
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-12 14:56:51 -04:00 |
|
 Woosuk KwonandClaude Opus 5.5
|
e1a5086256
|
[Attention] Remove cascade attention (#61126)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-12 10:41:07 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Harry Huangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
c75cb63cd2
|
[Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next (#56322)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-12 20:56:44 +08:00 |
|
Tianyu Guo
|
258b97ff6b
|
[Bugfix][EC Connector] Size transfers by measured encoder output widths (#57904)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
|
2026-10-12 12:32:39 +00:00 |
|
 Canlin GuoandClaude Opus 5.5
|
fe4df3df0e
|
[Bugfix][GLM-5.3-Flash] Restore MTP index sharing across draft iterations (#61079)
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-12 05:41:52 +00:00 |
|
 aoshen02andCodex
|
239a3fa3a8
|
[Sleep] Retain draft-private weights with existing parameter selector (#61046)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-11 23:30:56 +08:00 |
|
 SeongJun Leeandlesj0610
|
a98247ab4d
|
[Model] Drop the 4x M-RoPE cache headroom for Qwen3-VL, Qwen3.5 and Qwen3.8 (#59853)
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
Signed-off-by: lesj0610 <lesj0610@gmail.com>
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com>
|
2026-10-10 06:51:52 +00:00 |
|
Nick Hill
|
7d0b4e57aa
|
[ROCm] Remove MRV1 model exceptions (#56360)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-10 05:09:19 +00:00 |
|
 Wei ZhaoandCodex
|
03a7d3776f
|
[Perf] Support minimum KV splits configuration for TokenSpeed MLA decode (#60345)
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-09 23:33:47 +00:00 |
|
 
|
04b02fc8a5
|
[Core] shm tensor arena for efficient cpu->gpu worker broadcast (#51207)
Signed-off-by: Bolin Sun <bolins@nvidia.com>
Signed-off-by: <bolins@nvidia.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-09 16:13:24 -07:00 |
|
 
|
04462732a4
|
[Bugfix] Reject max_num_scheduled_tokens=0 at config time instead of hanging (#60714)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Hui Li <40770106+untrall@users.noreply.github.com>
|
2026-10-09 19:15:10 +00:00 |
|
 Mikko TukiainenandCursor
|
81c993658e
|
[ROCm][Perf] Fuse main QK-norm/RoPE/gate and KV-cache write into the AMD QSA prepare launch (#59732)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-10-09 21:14:49 +08:00 |
|
 
|
a6f43e57d6
|
[ROCm] Enable the cuMem CUDA-graph pool offload on ROCm (#59523)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: indianspeedster <cspandey016@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: indianspeedster <cspandey016@gmail.com>
|
2026-10-09 08:29:55 +00:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)    
|
d8fa463e2b
|
[Bugfix][Prefix caching] Fix inconsistent hybrid Mamba prefix cache boundaries (#60533)
Signed-off-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-08 18:33:48 +00:00 |
|
 
|
e41d5d5543
|
[Bugfix] Don't warn about missing watermark config for default requests (#60412)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-10-08 15:29:20 +00:00 |
|
Woosuk Kwon
|
194bd72be7
|
[Compile] Remove torch.compile-based sequence parallelism and AsyncTP (#60517)
|
2026-10-08 08:28:23 -07:00 |
|
 Harry MellorandClaude Opus 5.5
|
21d9758459
|
Remove Python 3.10 support now that it is EOL (#60402)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-08 02:30:52 -07:00 |
|
 ARAVINDHAN Tandlinitra24
|
e58f8dde7f
|
[Bugfix] Use FusedMoEWithLoRA for shared MoE LoRAs on 3D models (#60098)
Signed-off-by: ARAVINDHAN T <arvindhant01@gmail.com>
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io>
|
2026-10-08 15:04:32 +08:00 |
|
 
|
f38c6778ce
|
[quantization] Add per-token NVFP4 quantization MoE support for ReLU2 (#56740)
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-10-07 22:01:09 +00:00 |
|
 
|
b9767299db
|
[Bugfix] Respect Inductor deterministic mode in combo-kernel defaults (#59074)
Signed-off-by: cheesecake <farzanaman99@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 20:08:11 +01:00 |
|
 Micah WilliamsonandFangzhou Ai
|
f6e3944e31
|
[ROCm][DSv4] AITER MegaMoEV2 Integration For DeepSeek V4 (#59685)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
|
2026-10-07 09:35:53 -05:00 |
|
aditi-amd
|
385d86a6cf
|
[Attention] Add UltraQuant 4-bit KV cache backend (FlyDSL D=256) (#57057)
Signed-off-by: Aditi Rana <aditi.rana@amd.com>
|
2026-10-06 20:42:52 +00:00 |
|
   
|
819852df6b
|
[ModelRunner V2] Speculative Decoding NGram GPU Implementations (#40704)
Signed-off-by: PatchouliTaisa <patchychen@tencent.com>
Signed-off-by: PatchouliTaisa <pyramkar@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Xuanan Chen <xuananchenc@nvidia.com>
Co-authored-by: PatchouliTaisa <patchychen@tencent.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Xuanan Chen <xuananchenc@nvidia.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
|
2026-10-06 13:32:59 -07:00 |
|
 Nishant RevurandClaude
|
ad0f67a7ec
|
[Bugfix][Spec Decode] Keep heterogeneous-vocab draft models on Model Runner V1 (#59541)
Signed-off-by: Nishant Revur <kumrfni@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-06 18:03:44 +00:00 |
|
SeongJun Lee
|
4ea0c28bc4
|
[Bugfix] Warn only on per-request do_normalize/do_rescale overrides (#60240)
Signed-off-by: lesj0610 <lesj0610@gmail.com>
|
2026-10-06 15:55:27 +00:00 |
|