sychen52 and Cyrus Leung
1b2c591cd0
[Bugfix] speedup nvfp4 kv for FMHA ( #55031 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-08 17:25:42 +00:00
Raya Elena Solano and mgoin
bcca76e7db
[Test] Dequantize NVFP4 KV cache scales in the layout the kernel writes ( #55908 )
...
Signed-off-by: Raya Elena Solano <raya.solano@mbinf.de >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-08 17:13:39 +00:00
8ebc5b0a18
[Bugfix] Fix NVFP4 fused SiLU+mul scale allocation and global scale direction ( #55643 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: jiahao <jxia77@terpmail.umd.edu >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-09-08 08:26:07 -07:00
Wentao Ye
6b15bea080
[Refactor] Remove utils dead code ( #53941 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-08 11:22:09 -04:00
Wentao Ye
07950d4734
[Kimi Bug] Fix kimi k3 startup cuda graph issue with recoverSSM ( #55774 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-08 10:29:53 -04:00
Roberto L. Castro and Codex
6ddbab03de
[4/N][warmup][DSv4] Migrate common attention kernels ( #50176 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-08 10:25:48 -04:00
Sherif Waly and Nicolò Lucchesi
f998862d46
[Bugfix] Fix Kimi K3 loading with interleaved weight streams ( #53379 )
...
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
2026-09-08 16:08:51 +02:00
263c4ff95f
[Bugfix][Spec Decode] Cache the Mamba state at the block-grid position of EAGLE resume ( #53945 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Signed-off-by: Adam Shaver <ashaver@nvidia.com >
Signed-off-by: akshaver <168006157+akshaver@users.noreply.github.com >
Co-authored-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: roikoren755 <26850796+roikoren755@users.noreply.github.com >
2026-09-08 09:59:58 -04:00
414057a3d3
[Bugfix] Apply attention sinks in the Transformers backend ( #52156 )
...
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-08 13:54:07 +00:00
Andy Lo and OpenAI Codex
5ebce2391e
[Bugfix][KV Offload] Respect prefix-cache bypass in SimpleCPUOffload ( #54998 )
...
Signed-off-by: Andy Lo <andy@mistral.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-08 13:14:19 +00:00
Shanshan Shen
25047604fe
[ROCm][Perf] Remove AITER paged-MQA outputs guard for DeepSeek-V4 ( #55808 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-08 06:58:26 -05:00
Kevin H. Luu
34b9899c8f
[CI] Increase ColQwen3 pooling test memory budget on H200 MIG ( #55878 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-09-08 11:39:01 +00:00
Stefano Castagnetta
13cf9e05c1
[Perf][Quant][NVFP4] Prefer W4A4 linear kernels over weight-only ones on SM120/121 ( #55170 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
2026-09-08 15:06:45 +04:00
jimmy-adams
e41a17e606
[ROCm][Quantization] Support AMD Quark per-block FP8 for fused MoE layers ( #52263 )
...
Signed-off-by: huji <huidong.ji@amd.com >
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com >
2026-09-08 10:30:26 +00:00
Kevin H. Luu and Codex
5db652225f
[Bugfix] Validate extension integers before engine serialization ( #55606 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-08 16:30:06 +08:00
Kevin H. Luu and Codex
96eccb8f49
[CI] Restore clean GPU state before OAI Triton MoE tests ( #55453 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-08 01:15:21 -07:00
ahmed xijiaat
81dabe2e5d
[Bugfix][KV Offload] Validate SWA coverage at unaligned cache-hit boundaries ( #55712 )
...
Signed-off-by: xijiade.aihemaiti <3146335281@qq.com >
2026-09-08 10:15:48 +03:00
Adababy and shaolila
5133e1d285
[Frontend] Migrate input-validation errors to VLLMValidationError (harmony/mistral/chat_utils/params) ( #55735 )
...
Signed-off-by: shaolila <shaolila@buaa.edu.cn >
Co-authored-by: shaolila <shaolila@buaa.edu.cn >
2026-09-08 06:45:40 +00:00
aoshen02
755541443f
[Bugfix][ROCm] Mask paged attention V cache padding ( #53856 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
2026-09-08 14:39:23 +08:00
weihan
9ea8f3ffc3
[Bugfix] Honor STEP token pooling in DispatchPooler.for_seq_cls ( #55307 )
...
Signed-off-by: weihan <130119942+krsish@users.noreply.github.com >
2026-09-08 04:13:09 +00:00
7fa2c63796
[Bugfix] Autotune FlashInfer deferred MoE decode kernels before CUDA graph capture ( #55377 )
...
Signed-off-by: ming <ming@inferact.ai >
Co-authored-by: Kimi Code <kimi-code@moonshot.cn >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-08 04:02:23 +00:00
He Liu and Kimi
eb6b619ab2
[Bugfix][KV Offload] Register mixed page sizes in one cache group ( #54756 )
...
Signed-off-by: heliubj18 <chengfeng@ppio.com >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-09-07 20:50:15 -07:00
Jee Jee Li
607a6e48c5
[CI/Build] Fix unused fake implementation testing ( #55797 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-09-08 03:36:18 +00:00
Wei Zhao
a69402aaa8
[Perf] Integrate FlashInfer KDA kernels ( #55364 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-09-07 20:29:30 -07:00
Wentao Ye
869f78732b
[Kimi Bug] Fix kimi k3 AssertionError assert 0 <= checkpoint_idx < len(blocks) ( #55747 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-07 22:34:48 -04:00
Andreas Karatzas
6b5a12c0f8
[ROCm][CI] Enable HY-V4 model initialization on ROCm ( #54405 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-09-07 20:55:44 -05:00
Artur Fierka and Claude
9dbdf8e9af
[Bugfix][Quantization][MoE] Normalise an unset group_size on the compressed-tensors WNA16 MoE path ( #53163 )
...
Signed-off-by: Artur Fierka <artur.fierka@intel.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-08 09:42:46 +08:00
Taneem Ibrahim
fdfa0a1659
[Pooling] Honor request_id from request bodies ( #55665 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-07 19:55:20 -04:00
Kevin H. Luu and Codex
58f09211dd
[CI] Synchronize shared offload unlink test before observing pathname ( #55604 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-07 12:52:36 -07:00
Sahil Patel
51da0ca66c
[CI/Build] Unskip ColQwen3 multimodal pooling tests on Transformers v5 ( #55588 )
...
Signed-off-by: Sahil <sip4818@gmail.com >
2026-09-07 15:15:39 +00:00
70584f69b1
[Model] Add Cohere Compass model ( #54774 )
...
Signed-off-by: am-cohere <312513976+am-cohere@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 07:36:49 -07:00
Abhi and Antigravity
7cc89a7dda
[Tests] Update SarvamMLA transformers v5 compatibility reason to hf ( #55728 )
...
Signed-off-by: Abhi <108084481+Aj2280@users.noreply.github.com >
Co-authored-by: Antigravity <antigravity@google.com >
2026-09-07 14:27:07 +00:00
f2d45f26bd
[Bugfix][Gemma] Conditionally create KV projections/norms on KV-shared layers ( #54917 )
...
Signed-off-by: Josephasafg <ajgard7@gmail.com >
Co-authored-by: minyichen <96753147+cyc00518@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-07 15:04:00 +01:00
42801b3a6b
[3/N][warmup][DSv4] Migrate FA4 MLA and shared CuTeDSL kernels ( #53565 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 09:36:35 -04:00
1713b9866a
[2/N][warmup][DSv4] Migrate sequence and DCP kernels ( #53564 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 09:35:54 -04:00
Adababy and shaolila
167858ea17
[Frontend] Migrate Responses harmony input validation to VLLMValidationError ( #55701 )
...
Signed-off-by: shaolila <shaolila@buaa.edu.cn >
Co-authored-by: shaolila <shaolila@buaa.edu.cn >
2026-09-07 13:30:11 +00:00
b339d75a41
[Bugfix][Spec Decode] Resolve n_predict from text_config for Qwen3.5 multimodal MTP ( #55369 )
...
Signed-off-by: SOUMYAJIT GHOSH <23051387@kiit.ac.in >
Signed-off-by: Soumyajit Ghosh <jobsoumyajit6124@gmail.com >
Co-authored-by: SOUMYAJIT GHOSH <23051387@kiit.ac.in >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 13:28:26 +00:00
Francisco Javier Arceo and OpenAI Codex
6a2a2bb02b
[Frontend] Add stateless /v1/responses/render endpoint ( #50195 )
...
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-07 12:50:42 +00:00
Artur Fierka
5e6f6a8ed4
[Bugfix][Quantization][XPU] Fix moe_wna16 linear weight loading ( #52651 )
...
Signed-off-by: Artur Fierka <artur.fierka@intel.com >
2026-09-07 20:20:17 +08:00
Kam Basra and Claude
4a806d08ee
[Bugfix] OffloadingConnector: stop zeroing offload hits under MTP/EAGLE spec decode ( #52771 )
...
Signed-off-by: Kam Basra <kameldipbasra@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-07 15:16:05 +03:00
Thien Tran
94e26dd3dd
[Qwen3.8-Flash-Next] Support FP8 indexer cache for QSA ( #54890 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-07 12:15:45 +00:00
Adababy and shaolila
7dbe386833
[Frontend] Migrate Responses API validation errors to VLLMValidationError ( #50257 )
...
Signed-off-by: shaolila <shaolila@buaa.edu.cn >
Co-authored-by: shaolila <shaolila@buaa.edu.cn >
2026-09-07 17:33:51 +08:00
Andreas Karatzas and OpenAI Codex
e476556189
[ROCm][CI] Add attention-sink support to ROCm AITER sparse MLA ( #54404 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-07 09:26:47 +00:00
Chaojun Zhang
392db567b2
[CI] [Test] skip test_wna16_cuda_high_bit_skips_humming on non-CUDA platforms ( #55660 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
2026-09-07 16:57:15 +08:00
5893426b88
[Bugfix] DSv4 MXFP4 selector: stop narrowing explicit aliases to their BF16 variant ( #53586 )
...
Signed-off-by: Zihua Wu <zihuaw@nvidia.com >
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-09-07 07:19:13 +00:00
Zhewen Li and zhewenl
34b1e9f7a6
[Bugfix][MooncakeStore] Fix finish-time save crash on hybrid models ( #54643 )
...
Signed-off-by: zhewenl <zhewenl@users.noreply.github.com >
Co-authored-by: zhewenl <zhewenl@users.noreply.github.com >
2026-09-07 07:18:30 +00:00
Andreas Karatzas
195bc9c4a1
[CI][ROCm] Temporarily skip unsupported HY-V4 initialization ( #55653 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-09-07 15:17:36 +08:00
6fbb00b188
[EPD] Add ECMooncakeConnector for encoder cache over Mooncake TransferEngine ( #41567 )
...
Signed-off-by: Teng Ma <sima.mt@alibaba-inc.com >
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com >
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Tianyu Guo <guoty@inferact.ai >
Co-authored-by: Zhou ziheng <jiaranran2@gmail.com >
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-07 06:21:43 +00:00
Andreas Karatzas
f7f060d253
[Bugfix][Audio] Restore soundfile-first automatic decoding ( #55642 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-09-07 06:21:02 +00:00
Thien Tran
d9105ea800
[Qwen3.8-Flash-Next] Remove torch.compile for NVIDIA implementation ( #55272 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-06 21:57:56 -07:00
