|
e80b6c642f
|
[ROCm][DSv4] Overlap the unfused shared expert with AITER MegaMoE (#61819)
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-18 22:39:38 +08:00 |
|
|
Harry HuangandThien Tran
|
494663940b
|
[Qwen4Exp] Clean up HC and PLE implementations for Qwen3.8-Flash-Next (#61454)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-10-18 19:38:21 +08:00 |
|
|
Rémi Delhaye
|
3c13653738
|
[Frontend] Score with the chat template declared by Sentence Transformers CrossEncoders (#60877)
Signed-off-by: rdlhio <contact@rdlh.io>
|
2026-10-18 11:30:12 +00:00 |
|
|
|
b2456c622b
|
[ROCm][CI] Make ROCm 10.0 the default image; keep ROCm 7.2 as -rocm72 (#58761)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-10-17 21:43:59 -05:00 |
|
|
Vadim GimpelsonandClaude
|
ec8b20fbf0
|
[Bugfix] Fix Gemma 4 encoder CUDA graph capture OOM at startup (#61365)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-17 11:04:49 +00:00 |
|
|
|
026f6ab268
|
[Model] DeepSeek-V4/V4.1 VL: stream language-model weights instead of sorting the checkpoint (#58857)
Signed-off-by: git bisector <gitbisector@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-17 09:37:56 +00:00 |
|
|
|
f019e3e105
|
[ROCm] attn res + sigmoid_mul + conv fusions (#52968)
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-17 00:54:54 +00:00 |
|
|
|
28e92eeff2
|
[ROCm][Perf][Kimi-K3] Use AITER Gluon chunk_kda for KDA prefill (#59552)
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-16 22:58:56 +00:00 |
|
|
Oxana KorzhandClaude Opus 5.5
|
5bf3d9f665
|
[ROCm][Kimi-K3] Support DSpark adaptive verification on MI355X (#59824)
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-16 13:53:49 -05:00 |
|
|
|
fa75d3456d
|
[Feature][PCP] Add DeepSeek-V4.1-Flash support (#59857)
Signed-off-by: wangyicong <wangyicong@bytedance.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 23:17:51 -07:00 |
|
|
maithilijoshi20andCyrus Leung
|
92dc3dd393
|
[Bugfix] Honor OPT token embedding width for prompt embeddings (#61736)
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-16 04:41:50 +00:00 |
|
|
Andreas KaratzasandClaude
|
29345f42a6
|
[ROCm][Qwen4Exp] Support HC and MoE sequence parallelism on ROCm (#61471)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-15 20:11:17 -05:00 |
|
|
|
a8be1500ba
|
[Bugfix][MoE] Restore TP-sharded routed-expert capture for DP=1 (#61388)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: Ao Shen <281812371+aoshen02@users.noreply.github.com>
Signed-off-by: Ao Shen <aoshen@inferact.ai>
Co-authored-by: Ao Shen <281812371+aoshen02@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Ao Shen <aoshen@inferact.ai>
|
2026-10-16 00:21:24 +00:00 |
|
|
|
3f1e57225a
|
[ROCm][Perf][Kimi-K3] Use AITER Gluon KDA kernels for decode and spec decode (#57392)
Signed-off-by: omuhamma <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
|
2026-10-15 17:54:24 -05:00 |
|
|
|
4d58fa254a
|
[MoE] Unify DeepGEMM MegaMoE experts behind one feature-matrix layer; add NVFP4 (#61538)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 19:02:01 +00:00 |
|
|
Ronaldandaoshen02
|
d12d5460a5
|
[Bugfix][MoE] Keep routing rows during DeepSeek V4.1 decoder replay (#61643)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-15 14:02:06 +00:00 |
|
|
littlecircle0730andShanshan Shen
|
71536bd5f8
|
[MM][CG] Support ViT full CUDA graph for DeepSeek-VL2 (#46005)
Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com>
Signed-off-by: littlecircle0730 <43994952+littlecircle0730@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
|
2026-10-15 21:38:52 +08:00 |
|
|
|
0951fc183c
|
[ROCm][DSv4.1] Disambiguate ROCm attention class names (#58689)
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 21:38:39 +08:00 |
|
|
Xunzhuo
|
766c837bd5
|
[Model] Support every Decision 3.0 size (d3-flash, d3-mini, d3-nano, d3-lite, d3-edge) (#61778)
Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com>
|
2026-10-15 13:29:24 +00:00 |
|
|
Micah Williamson
|
e8e1e96196
|
[ROCm][DSv4] AITER MegaMoE Fixed-Slot Dispatch For DeepSeek-V4 (#61262)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
|
2026-10-15 19:10:59 +08:00 |
|
|
TimofeyandCodex
|
716c5225ae
|
[Model] Add AliceAI support (#57963)
Signed-off-by: timka <timka.byzov@yandex.ru>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-15 10:15:03 +00:00 |
|
|
|
b965ee564d
|
[Chore] Use upstream Transformers DeepseekV4Config (#61392)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 01:08:08 -07:00 |
|
|
Haiyu ZuoandKevin H. Luu
|
ff5a91a456
|
[Feature][Model Loader] Apply model-owned MTP shard selection to RunAI ModelStreamer (#60759)
Signed-off-by: 浩韵 <zuohaiyu.zhy@alibaba-inc.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 12:23:11 +08:00 |
|
|
Woosuk KwonandClaude Opus 5.5
|
8530aed2b7
|
[Spec Decode] Remove Medusa and MLP speculator support (#61701)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 17:29:23 -07:00 |
|
|
josh-schell-amdandClaude Opus 5.5
|
273a04964e
|
[ROCm] Bump AITER to v0.1.25 and relax the DSv4.1 mono MoE tolerance (#61697)
Signed-off-by: josh schell <josh.schell@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-14 23:50:24 +00:00 |
|
|
|
d1d8e6c096
|
[Perf][Nemotron-Parse] Do image rescale/normalize on the device (#60500)
Signed-off-by: DiegoCao <DiegoCao@users.noreply.github.com>
Co-authored-by: DiegoCao <DiegoCao@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 18:33:22 +00:00 |
|
|
Rohan PotdarandClaude Opus 5.5
|
52a8cc5316
|
[ROCm][CI] Add dtype to the KDA metadata test's fake model config (#61650)
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 18:04:14 +00:00 |
|
|
Xunzhuo
|
89cab55086
|
[Model] Support Decision 3.0 decision models (vllm-sr/d3) (#61565)
Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com>
|
2026-10-14 17:10:55 +00:00 |
|
|
|
3723a1ea96
|
[Bugfix][Transformers backend] Fix CUDA graph capture for dynamic-rope models (#58064)
Signed-off-by: YongfengX <erkang040804@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-14 05:56:54 -07:00 |
|
|
Lazurite
|
23fbc2ec2f
|
[Model] Support precomputed image embeddings for LightOnOCR (#61585)
Signed-off-by: Rapisurazurite <43494437+Rapisurazurite@users.noreply.github.com>
|
2026-10-14 10:54:09 +00:00 |
|
|
Ronaldandaoshen02
|
19aba3e4d5
|
[Bugfix][RL] Load attention sinks through weight_loader for layerwise reload (#57797)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-14 10:17:58 +00:00 |
|
|
|
30e79c3a76
|
[Core] Update PyTorch to 2.14.0, torchvision to 0.29.0, triton to 3.8.0 (release channel) (#52183)
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com>
Signed-off-by: Andrey Talman <atalman@fb.com>
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 00:17:59 -07:00 |
|
|
Stefano CastagnettaandKartikey Vyas
|
4e0681ed56
|
[Quantization] Support packed NVFP4 Qwen4Exp PLE embeddings (#56273)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Kartikey Vyas <kartikey@artificialanalysis.ai>
|
2026-10-14 08:24:55 +08:00 |
|
|
|
698e79f3bd
|
[Model] Kolibri 1 via the Transformers modeling backend (#60827)
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-10-13 22:17:52 +01:00 |
|
|
Andreas KaratzasandClaude
|
67516b6c42
|
[Bugfix][ROCm][Kimi-K3] Stage fused KDA decode weights after loading (#61207)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-13 14:55:57 -05:00 |
|
|
|
c5bda658d2
|
[Model][MoE] Load MegaMoE experts directly into the DeepGEMM layout (#60096)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Aaron Hao <hao-aaron@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Aaron Hao <hao-aaron@users.noreply.github.com>
|
2026-10-13 11:52:08 -07:00 |
|
|
wang.yuqi
|
669f85e686
|
[Misc] Break chat_utils into more maintainable modules. (#60972)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-10-13 15:08:07 +00:00 |
|
|
Jiangyun ZhuandClaude Opus 5.5
|
61d9c777bf
|
[CI] Run the Clef pooler test on GPU (#61409)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-13 14:42:53 +00:00 |
|
|
Nathan SalaandShaun Kotek - Nvidia
|
0838cd6593
|
[Bugfix] Load and apply Nemotron Super final vision LayerNorm (#61401)
Signed-off-by: Shaun Kotek <skotek@nvidia.com>
Co-authored-by: Shaun Kotek - Nvidia <skotek@nvidia.com>
|
2026-10-13 12:42:29 +00:00 |
|
|
Harry MellorandClaude Opus 5
|
6f11c220b5
|
[Transformers backend] Fuse MoE routers that score after the top-k (#57410)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-10-13 08:41:23 +00:00 |
|
|
|
380638afbb
|
[Model] Add Clef-Flash support (#61308)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-10-13 05:46:37 +00:00 |
|
|
|
df6c7ad031
|
[Core][Refactor] Move KV cache planning logic to kv_cache_planning.py (#61183)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-13 03:05:39 +00:00 |
|
|
frida-andersson
|
c077dc663d
|
[ROCm][Perf][DSv4.1] Enable decoder SWA bounded replay (#59039)
Signed-off-by: Frida Andersson <fanderss@amd.com>
|
2026-10-13 02:19:14 +00:00 |
|
|
Fangzhou AiandClaude Opus 5.5
|
cc2a6eea1a
|
[ROCm][Perf][DSv4.1] Mono decode on every backbone layer (#61082)
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-13 10:16:50 +08:00 |
|
|
|
189bbfb9f4
|
Add MoNe experts pruning support (#50415)
Signed-off-by: Ilya Boytsov <ieboytsov@nebius.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Ilya Boytsov <ieboytsov@nebius.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-12 23:18:43 +00:00 |
|
|
Shang WangandClaude Opus 5.5
|
ffec221276
|
[Bugfix][DSv4.1] Match decoder-replay input_ids dtype for FlashInfer MegaMoE (#61041)
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-12 14:23:00 +00:00 |
|
|
Ilya Markov
|
a281ad32f0
|
[EPLB] Add EPLB support to dspark draft models (DeepSeekV4) (#57549)
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
|
2026-10-12 10:03:09 -04:00 |
|
|
|
092c35dd15
|
[Model] Support Cloudflare/clef-omni on /v1/systemone (#61064)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-10-12 21:27:34 +08:00 |
|
|
Harry Huangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
c75cb63cd2
|
[Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next (#56322)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-12 20:56:44 +08:00 |
|
|
MatthewCYM
|
f7a3a7265f
|
[Bugfix][Multimodal] Fix Qwen3-ASR audio preprocessing parity (#57146)
Signed-off-by: MatthewCYM <chenyiming0v0@gmail.com>
|
2026-10-12 10:42:51 +00:00 |
|