Julien Debache
111f71a6db
[feat] FlashInfer CuteDSL MegaMoE integration ( #54049 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
2026-10-02 15:43:33 -04:00
Thang Nguyen and Claude Opus 5.5
7cfd233538
Revert "[Bugfix] Tie lm_head.weight for Nemotron Parse when checkpoint omits it" ( #53020 ) ( #59805 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 10:19:25 -07:00
Jiangyun Zhu and Claude Opus 5.5
d648847fe4
[CI][Kimi-K3] Test prefix cache reuse with KV offload, P/D and DCP ( #59229 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 20:58:35 +08:00
Abhay Shukla and Claude Opus 5.5
fa212b8dc5
[Test] Re-enable Ovis2.5 and Ovis2.6-MoE vLLM tests on Transformers v5 ( #59763 )
...
Signed-off-by: AbhayShuklaIIT <abhay.shukla@live.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 11:59:40 +00:00
Harry Mellor and Claude Opus 5.5
5d20979fc9
[Misc] Remove code paths for Transformers < 5.16.1 ( #59762 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 11:17:24 +00:00
Simon Danielsson and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
98b29aa99a
[ROCm][Perf][GLM-5.3-Flash] Add AITER topk backend for decodes ( #58167 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-02 11:47:41 +01:00
5688a4dd4a
[Bugfix][GLM-5.3] Size the image encoder cache from the exact token ceiling ( #59565 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Co-authored-by: Kimi Code <kimi-code@moonshot.cn >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-02 10:43:29 +00:00
Juan Pérez de Algaba
c15672cfb4
[Security] Accept zero-sum DeepSeek-OCR pixel tensors ( #59417 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
2026-10-02 09:17:12 +00:00
kliuae
0154979656
[ROCm][Perf] Kimi-K3 enable prefill checkpoints on ROCm ( #58344 )
...
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com >
2026-10-02 06:58:04 +00:00
Harry Mellor and Claude Opus 5
e3cae8d2ac
[Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor ( #57387 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-02 02:25:18 +00:00
aniskumar-nv and Cyrus Leung
83cadd65d9
[Bugfix] Tie lm_head.weight for Nemotron Parse when checkpoint omits it ( #53020 )
...
Signed-off-by: Anish Kumar <aniskumar@nvidia.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-01 09:26:33 -07:00
Shuolei Wang and Thien
c9578f10ab
[Qwen3.8-Flash-Next] Fuse main QK-norm/RoPE/gate and KV-cache write into the QSA pre-indexer launch ( #57097 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Thien <thien@inferact.ai >
Co-authored-by: Thien <thien@inferact.ai >
2026-10-01 14:21:06 +08:00
766b4ae54c
[Kimi-K3] Add FlashInfer speculative KDA backend ( #54255 )
...
Signed-off-by: Duncan Moss <dmoss@nvidia.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-01 02:28:50 +00:00
Harry Mellor and Claude Opus 5.5
fc2c801ad9
Disable mypy arg-type and assignment checks in tests ( #59428 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 15:18:50 +00:00
Yida Weng and Grok
17e9295dd5
[BugFix][Multimodal] Pick worst-case DeepSeek-V4 VL dummy image size ( #59271 ) ( #59373 )
...
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com >
Co-authored-by: Grok <grok@x.ai >
2026-09-30 15:09:11 +00:00
5225f1bacb
[MyPy][3/N] Fix MyPy errors in test groups (part 3) ( #55939 )
...
Signed-off-by: Taimys <ffy2281693949@gmail.com >
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com >
Co-authored-by: Taimys <ffy2281693949@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 11:06:07 +00:00
Mathew Odden and opencode+glm-5.3+vllm
91dab0eb7f
[ROCm][Perf] Allocate the pinned PLE prefetch buffer lazily ( #58797 )
...
Signed-off-by: Mathew Odden <modden@redhat.com >
Co-authored-by: opencode+glm-5.3+vllm <opencode+glm-5.3+vllm@example.com >
2026-09-30 17:43:16 +08:00
yhcheong (vllmellm)
16d4ac4ba3
[ROCm][MoE] Support MiMo-V2.6 MXFP4 on gfx942 ( #58262 )
...
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com >
2026-09-30 17:23:00 +08:00
Simon Danielsson
3627a6a124
[ROCm][Perf][GLM-5.3-Flash] Stride-aware decode KDA ( #57979 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-09-30 08:16:29 +00:00
cjackal
0a30bc3f9a
[Model] Extend device-side mm normalization to GLM4V/GLM5Next ( #55389 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-09-30 15:44:46 +08:00
5faf81a429
[CPU][Whisper] Support W4A16 quantized Whisper on the CPU WNA16 kernel ( #58268 )
...
Signed-off-by: Harshal Adhav <harshal.adhav@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-30 13:52:53 +08:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
834ad456d3
[Refactor] Remove dead tests code ( #58916 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-30 04:39:03 +00:00
Chauncey
3e2a7e74a5
[Feat][Model] Enable KDA prefill checkpoints for GLM-5.3-Flash ( #56960 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-09-29 09:14:30 +00:00
Jakub Zakrzewski and OpenAI Codex
70dc122f81
[MM] Enable device normalization for Llama Nemotron VL Embed/Rerank ( #57928 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-29 08:33:52 +00:00
Yifan Qiao and Claude Fable 5.1
4b2e1cfa7b
[Model] Decoder-side SWA bounded replay for DeepSeek-V4.1 ( #58132 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-29 08:25:46 +00:00
d86cf91240
[MM][Mistral3] Image preprocessing optimization ( #57531 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-29 06:53:38 +00:00
65ed0a2642
[Bugfix][GLM-5.3-Flash] Take video placeholder timestamps from the pixel path's frame sampler ( #55647 )
...
Signed-off-by: NNNtrance <232268405+NNNtrance@users.noreply.github.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: NNNtrance <232268405+NNNtrance@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-29 06:45:29 +00:00
Daniel M. García-Ocaña Hernández
295a0b8d64
[Bugfix][Multimodal] Fix flat/scoped mm_processor_kwargs merge and resolution ( #56372 )
...
Signed-off-by: Daniel M. García-Ocaña Hernández <danielgarciaocana@gmail.com >
2026-09-29 03:58:39 +00:00
cjackal and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
491f44adfa
[Model] Extend device-side mm normalization to Qwen3VL/Qwen3.5/Qwen4Next ( #51289 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-29 11:44:54 +08:00
Ashraf Bhuiyan and Wentao Ye
2619b6b72b
[Mypy] Fix mypy typing for Voxtral and vision models ( #58251 )
...
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-28 15:52:50 -04:00
xaguilar-amd and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a7ff44355c
[ROCm][Perf] Kimi-K3 Enable sharded latent MoE up-projection under EP ( #54956 )
...
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 10:31:29 -04:00
Jiangyun Zhu and OpenAI Codex
20b52e9f5b
[Perf][Qwen3.8] Reduce PLE metadata construction overhead ( #58114 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-28 20:46:22 +08:00
Thien Tran
d6cce94fd4
[Perf][Qwen4Exp] Fuse HC down projection and SiLU on NVIDIA ( #58957 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-28 18:51:38 +08:00
a2be4d3cc7
[Bugfix][Qwen4Exp] Release the profiling KV cache held by QSA key views ( #58961 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 08:11:46 +00:00
Kevin H. Luu and Claude Opus 5.5
0da126676e
[Bugfix] Fix the two multimodal root tests that fail on main (OpenPangu-VL embed merge, MiMo sink test fixture) ( #58900 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-28 04:52:51 +00:00
dd3bc9c72f
[Perf][MRV2] Reuse Mamba/GDN metadata across KV cache groups ( #58762 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Xin Yang <xyangx@amazon.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-09-28 04:36:58 +00:00
Thien Tran and Codex
55de40a2fc
[Minimax-M3] Add Encoder CUDA graph support ( #58673 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Codex <codex@openai.com >
2026-09-28 04:17:36 +00:00
Wentao Ye
bf93e1f92c
[Refactor] Remove dead tests utils ( #58803 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-26 09:16:53 -04:00
a4eb3f25d6
[Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer ( #57263 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-26 17:01:17 +08:00
Aarushi Jain and Andreas Karatzas
3b4566c5cf
[CI] Only isolate the registry tests that need a fresh process ( #58764 )
...
Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-26 14:03:45 +08:00
e3dd5b5a75
[ROCm][CI] Test AMD DeepSeek V4 MoE routing against a PyTorch reference ( #58740 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 18:05:54 +00:00
1417022c3d
[ROCm][Kimi-K3] Optimize low-concurrency speculative KDA ( #58045 )
...
Signed-off-by: jiacao-amd <jiahui.cao@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 12:33:30 -05:00
73a78e6f1f
[SpecDecode] Add LiLiCorr drafter ( #57934 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Signed-off-by: Andrii Skliar <andreyws96@gmail.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
2026-09-25 10:08:28 -04:00
Yongye Zhu and Claude Opus 5.5
378504a544
[MoE] Defer the TRTLLM-Gen top-k finalize on the modular path ( #58635 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-25 15:38:29 +08:00
8a269919f4
[MM] Add Triton kernel for mm_input_normal. ( #56798 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <2037008807@qq.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-25 04:31:20 +00:00
267eee54bf
[Qwen4Exp][ROCm] PLE n-gram table CPU offload ( #57497 )
...
Signed-off-by: Mathew Odden <modden@redhat.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: opencode+deepseek-v4-flash+vllm <opencode+deepseek-v4-flash+vllm@example.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-24 23:28:20 -05:00
4013f3ad6c
[ROCm] Fix CI runtime and tests for MI355 DPX ( #58244 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Mahesh Kunreddi <mahesh.kunreddi@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-24 23:15:11 -05:00
48d8880d09
[Bugfix][Qwen4Exp] Keep pinned PLE prefetch ids out of the CUDA graph pool ( #58489 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 10:51:50 +08:00
90a9515006
[Cleanup] Remove online quantization support in fp8.py in favor of online shorthands ( #53585 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-24 12:48:14 -07:00
5b3f280a6d
[Bugfix] Count unsplit Idefics3 image patches ( #48760 )
...
Signed-off-by: nightcityblade <nightcityblade@gmail.com >
Co-authored-by: nightcityblade <nightcityblade@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-24 11:36:26 -07:00
Julien Debache