Ronald and aoshen02
d12d5460a5
[Bugfix][MoE] Keep routing rows during DeepSeek V4.1 decoder replay ( #61643 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-15 14:02:06 +00:00
littlecircle0730 and Shanshan Shen
71536bd5f8
[MM][CG] Support ViT full CUDA graph for DeepSeek-VL2 ( #46005 )
...
Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com >
Signed-off-by: littlecircle0730 <43994952+littlecircle0730@users.noreply.github.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-15 21:38:52 +08:00
0951fc183c
[ROCm][DSv4.1] Disambiguate ROCm attention class names ( #58689 )
...
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 21:38:39 +08:00
Xunzhuo
766c837bd5
[Model] Support every Decision 3.0 size (d3-flash, d3-mini, d3-nano, d3-lite, d3-edge) ( #61778 )
...
Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com >
2026-10-15 13:29:24 +00:00
Micah Williamson
e8e1e96196
[ROCm][DSv4] AITER MegaMoE Fixed-Slot Dispatch For DeepSeek-V4 ( #61262 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
2026-10-15 19:10:59 +08:00
Timofey and Codex
716c5225ae
[Model] Add AliceAI support ( #57963 )
...
Signed-off-by: timka <timka.byzov@yandex.ru >
Co-authored-by: Codex <noreply@openai.com >
2026-10-15 10:15:03 +00:00
b965ee564d
[Chore] Use upstream Transformers DeepseekV4Config ( #61392 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 01:08:08 -07:00
Haiyu Zuo and Kevin H. Luu
ff5a91a456
[Feature][Model Loader] Apply model-owned MTP shard selection to RunAI ModelStreamer ( #60759 )
...
Signed-off-by: 浩韵 <zuohaiyu.zhy@alibaba-inc.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 12:23:11 +08:00
Woosuk Kwon and Claude Opus 5.5
8530aed2b7
[Spec Decode] Remove Medusa and MLP speculator support ( #61701 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 17:29:23 -07:00
josh-schell-amd and Claude Opus 5.5
273a04964e
[ROCm] Bump AITER to v0.1.25 and relax the DSv4.1 mono MoE tolerance ( #61697 )
...
Signed-off-by: josh schell <josh.schell@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-14 23:50:24 +00:00
d1d8e6c096
[Perf][Nemotron-Parse] Do image rescale/normalize on the device ( #60500 )
...
Signed-off-by: DiegoCao <DiegoCao@users.noreply.github.com >
Co-authored-by: DiegoCao <DiegoCao@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 18:33:22 +00:00
Rohan Potdar and Claude Opus 5.5
52a8cc5316
[ROCm][CI] Add dtype to the KDA metadata test's fake model config ( #61650 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 18:04:14 +00:00
Xunzhuo
89cab55086
[Model] Support Decision 3.0 decision models (vllm-sr/d3) ( #61565 )
...
Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com >
2026-10-14 17:10:55 +00:00
3723a1ea96
[Bugfix][Transformers backend] Fix CUDA graph capture for dynamic-rope models ( #58064 )
...
Signed-off-by: YongfengX <erkang040804@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 05:56:54 -07:00
Lazurite
23fbc2ec2f
[Model] Support precomputed image embeddings for LightOnOCR ( #61585 )
...
Signed-off-by: Rapisurazurite <43494437+Rapisurazurite@users.noreply.github.com >
2026-10-14 10:54:09 +00:00
Ronald and aoshen02
19aba3e4d5
[Bugfix][RL] Load attention sinks through weight_loader for layerwise reload ( #57797 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-14 10:17:58 +00:00
30e79c3a76
[Core] Update PyTorch to 2.14.0, torchvision to 0.29.0, triton to 3.8.0 (release channel) ( #52183 )
...
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com >
Signed-off-by: Andrey Talman <atalman@fb.com >
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 00:17:59 -07:00
Stefano Castagnetta and Kartikey Vyas
4e0681ed56
[Quantization] Support packed NVFP4 Qwen4Exp PLE embeddings ( #56273 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Kartikey Vyas <kartikey@artificialanalysis.ai >
2026-10-14 08:24:55 +08:00
698e79f3bd
[Model] Kolibri 1 via the Transformers modeling backend ( #60827 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-13 22:17:52 +01:00
Andreas Karatzas and Claude
67516b6c42
[Bugfix][ROCm][Kimi-K3] Stage fused KDA decode weights after loading ( #61207 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-13 14:55:57 -05:00
c5bda658d2
[Model][MoE] Load MegaMoE experts directly into the DeepGEMM layout ( #60096 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Aaron Hao <hao-aaron@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Aaron Hao <hao-aaron@users.noreply.github.com >
2026-10-13 11:52:08 -07:00
wang.yuqi
669f85e686
[Misc] Break chat_utils into more maintainable modules. ( #60972 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-13 15:08:07 +00:00
Jiangyun Zhu and Claude Opus 5.5
61d9c777bf
[CI] Run the Clef pooler test on GPU ( #61409 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 14:42:53 +00:00
Nathan Sala and Shaun Kotek - Nvidia
0838cd6593
[Bugfix] Load and apply Nemotron Super final vision LayerNorm ( #61401 )
...
Signed-off-by: Shaun Kotek <skotek@nvidia.com >
Co-authored-by: Shaun Kotek - Nvidia <skotek@nvidia.com >
2026-10-13 12:42:29 +00:00
Harry Mellor and Claude Opus 5
6f11c220b5
[Transformers backend] Fuse MoE routers that score after the top-k ( #57410 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-13 08:41:23 +00:00
380638afbb
[Model] Add Clef-Flash support ( #61308 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-13 05:46:37 +00:00
df6c7ad031
[Core][Refactor] Move KV cache planning logic to kv_cache_planning.py ( #61183 )
...
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 03:05:39 +00:00
frida-andersson
c077dc663d
[ROCm][Perf][DSv4.1] Enable decoder SWA bounded replay ( #59039 )
...
Signed-off-by: Frida Andersson <fanderss@amd.com >
2026-10-13 02:19:14 +00:00
Fangzhou Ai and Claude Opus 5.5
cc2a6eea1a
[ROCm][Perf][DSv4.1] Mono decode on every backbone layer ( #61082 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 10:16:50 +08:00
189bbfb9f4
Add MoNe experts pruning support ( #50415 )
...
Signed-off-by: Ilya Boytsov <ieboytsov@nebius.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Ilya Boytsov <ieboytsov@nebius.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 23:18:43 +00:00
Shang Wang and Claude Opus 5.5
ffec221276
[Bugfix][DSv4.1] Match decoder-replay input_ids dtype for FlashInfer MegaMoE ( #61041 )
...
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 14:23:00 +00:00
Ilya Markov
a281ad32f0
[EPLB] Add EPLB support to dspark draft models (DeepSeekV4) ( #57549 )
...
Signed-off-by: Markov Ilya <markovilya197@gmail.com >
2026-10-12 10:03:09 -04:00
092c35dd15
[Model] Support Cloudflare/clef-omni on /v1/systemone ( #61064 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-12 21:27:34 +08:00
Harry Huang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
c75cb63cd2
[Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next ( #56322 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-12 20:56:44 +08:00
MatthewCYM
f7a3a7265f
[Bugfix][Multimodal] Fix Qwen3-ASR audio preprocessing parity ( #57146 )
...
Signed-off-by: MatthewCYM <chenyiming0v0@gmail.com >
2026-10-12 10:42:51 +00:00
AMD-yanfeiwang and aoshen02
e3bbaf2369
[ROCm][DSv4] Fuse the shared expert into AITER MegaMoE ( #61004 )
...
Signed-off-by: Yanfei Wang <yanfei.wang@amd.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-12 15:53:16 +08:00
dingdangmao and Codex
124b89d5b7
[Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries ( #59962 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-12 02:08:14 +00:00
f26d9f432c
[Bugfix][GLM-5.3-Flash] Honor --mamba-ssm-cache-dtype for the KDA recurrent state ( #59147 )
...
Signed-off-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-11 17:55:00 +00:00
Nick Hill
c059bee812
[MRV2] Switch CI tests still running on MRV1 unnecessarily ( #61014 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-11 09:57:07 -07:00
tc-mb and Cyrus Leung
c85616efc6
[Model] Support MiniCPM-V 4.7 ( #58674 )
...
Signed-off-by: tc-mb <tianchi_cai@icloud.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-11 10:21:10 +00:00
Bruno Volpato and Codex
61f084b5f3
[Bugfix][Multimodal] Fix per-video cache option reuse ( #55203 )
...
Signed-off-by: bvolpato <brunocvcunha@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 06:21:55 +00:00
Mook
bf16c9c139
[Bugfix][Model] K-EXAONE: restore loading of pre-2.0 configs and apply RoPE to the MTP layer ( #60187 )
...
Signed-off-by: godmook <cmoh4135@naver.com >
Signed-off-by: Mook <68294499+Godmook@users.noreply.github.com >
2026-10-10 15:05:55 +00:00
wang.yuqi
c1692c85f1
[Misc] mv chat_utils.py to renderers. ( #60522 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-10 06:50:34 +00:00
Fangzhou Ai and Claude Opus 5.5
193922d6ff
[ROCm][Perf][DSv4.1] Mono decode layer for DeepSeek-V4.1-Flash on MI355X ( #60397 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 20:53:18 -05:00
f8e6e365ae
[Bugfix][Mooncake] Save the EAGLE attention proof with hybrid Mamba partial tails ( #60695 )
...
Signed-off-by: Dao Le <Dao007forever@gmail.com >
Signed-off-by: Dao007forever <daole@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-10 00:08:08 +00:00
1a3a7a8f01
[Bugfix] Treat max_tokens as output upper bound, not input-space reservation ( #57035 )
...
Signed-off-by: YuvrajSinghBhadoria2 <245782664+YuvrajSinghBhadoria2@users.noreply.github.com >
Signed-off-by: YuvrajSinghBhadoria2 <yuvrajsinghbhado2030@gmail.com >
Signed-off-by: Yuvraj Singh Bhadoria <yuvrajsinghbhado2030@gmail.com >
Co-authored-by: YuvrajSinghBhadoria2 <245782664+YuvrajSinghBhadoria2@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-10-10 01:15:22 +08:00
Mikko Tukiainen and Cursor
81c993658e
[ROCm][Perf] Fuse main QK-norm/RoPE/gate and KV-cache write into the AMD QSA prepare launch ( #59732 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-09 21:14:49 +08:00
Damien Laine and Cyrus Leung
2186b89a9a
[Bugfix][Model] Voxtral realtime: fix boot OOM and engine crash at max_model_len ( #45022 )
...
Signed-off-by: Damien Laine <damien.laine@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-09 17:40:29 +08:00
Harry Mellor and Claude Opus 5.5
2c99ee9333
[Chore] Remove vendored code made redundant by Transformers 5.16.1 ( #60623 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 09:29:03 +00:00
ad73a4740d
[Perf][DSv4.1] Trim the last KV-source layer in decoder replay ( #59894 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Juntian Liu <Juntianl777@gmail.com >
2026-10-09 06:07:49 +00:00
