Thomas Ortner and Harry Mellor
e509d32b5a
[Transformers backend] Enable QKV-Fuser for Gemma4 ( #55690 )
...
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com >
Signed-off-by: Thomas Ortner <bohnstinglthomas@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-09 14:46:31 +01:00
Yicong Wang and OpenAI Codex
d2906091bf
[Bugfix] Fix Mooncake heterogeneous TP with replicated GQA heads ( #52516 )
...
Signed-off-by: wangyicong <wangyicong@bytedance.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-09 14:18:32 +08:00
Jing Wang
95cf420ac1
[CI][Test][Spec Decode] Fix CI failure of Qwen3 Omni DSpark loader mock ( #56011 )
...
Signed-off-by: Jing Wang <jingwang96@qq.com >
2026-09-08 23:06:15 -07:00
bf7eee9d7b
[Bugfix][EC Connector] Fix ECExampleConnector load device under TP>1 ( #40416 )
...
Signed-off-by: Hang Gong <kidogong@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-09 11:44:30 +08:00
Qiming Zhang
27b63fb8f8
[CI][XPU] skip test_sampling_mask_tensors_match_finite_support on XPU ( #55852 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
2026-09-09 11:16:03 +08:00
Akine-Ko
41541636fd
[Refactor][EC Connector] Add backend extension points to ECCPUWorker ( #54033 )
...
Signed-off-by: Akine-Ko <2923601304@qq.com >
2026-09-08 20:13:47 -07:00
Tianyu Guo and OpenAI Codex
f1aeff6007
[Bugfix][EC Connector] Fail the request, not the engine, when a remote encoding cannot arrive ( #55290 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-08 20:13:33 -07:00
djramic and Andreas Karatzas
c6ba5e89d6
[Bugfix][Elastic EP] Route only to surviving engines during scale-down ( #55772 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-08 21:28:28 -05:00
djramic
21a117ad87
[CI][IR] Speed up vLLM IR test group ( #55965 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
2026-09-08 21:28:06 -05:00
0b066293f3
[Core][KV Connector] Resolve connector block tables for every scheduled request ( #54853 )
...
Signed-off-by: Zhewen Li <zhewenli@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-09 02:15:53 +00:00
Matt
bc8587f829
[ROCm][Perf][Bugfix] Multi-stream perf improvements; rocprofiler fixes ( #55099 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
2026-09-08 17:42:34 -07:00
Flora Feng
d98c8c030b
[Bugfix] Parse DSML tool calls when the model omits the tool_calls wrapper ( #55954 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-08 17:38:04 -07:00
Michael Lazos and OpenAI Codex
28a73ccfba
[Kernel] Add NVFP4 support to the torch linear backend ( #53319 )
...
Signed-off-by: Michael Lazos <mlazos@meta.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-08 20:30:05 -04:00
Nick Hill
4aadb0f14d
[Core] Enhance cpu<->gpu sync checking to include paged async copies ( #53491 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-09-08 17:26:21 -07:00
Qidong Su and OpenAI Codex
9e137d8d23
[Kernel] Enable optimized FlashInfer add-RMSNorm NVFP4 fusion ( #51925 )
...
Signed-off-by: Qidong Su <qidongs@nvidia.com >
Signed-off-by: Qidong Su <soodoshll@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-08 17:20:32 -07:00
Taneem Ibrahim
5af4cc33ec
[Bugfix] Fix OpenPangu multimodal embedding merge ( #55941 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-08 23:53:59 +00:00
9c2d21046b
[Bugfix][Spec Decode] Honour the draft's moe_backend on Model Runner V2 ( #54788 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-08 16:50:07 -07:00
c0d8d5413e
[Transformers] Generalize merged-column linear fusion ( #55301 )
...
Signed-off-by: BANANASJIM <bananasjim1@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-08 23:49:37 +00:00
60ad959b6f
Fix block FP8 MTP in ModelOpt mixed checkpoints ( #55513 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-08 22:47:21 +00:00
Flora Feng and Wentao Ye
29f46ac475
[Perf] Eliminate full-history reasoning scans for structured outputs ( #55223 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-08 14:07:16 -07:00
Andreas Karatzas and OpenAI Codex
db3814a4f2
[Attention] Require explicit DCP support from attention implementations ( #55780 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-08 14:04:27 -07:00
Andreas Karatzas
cb22234687
[CI] Reuse ColQwen3 models across pooling tests ( #55889 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-09-08 14:03:54 -07:00
d29c88f162
[Core] Let SWA layers take the primary block size to avoid inflating the KV block LCM ( #53007 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-08 16:44:43 -04:00
Andreas Karatzas
18615ad1ee
[Bugfix] Handle null RoPE parameters for NoPE layers ( #55949 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-09-08 20:20:05 +00:00
e0aaef85f3
[2/N][KV Connector][NIXL] Support per-region transfer geometry ( #53780 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-08 16:01:12 -04:00
Roderick Wu and mgoin
2c9d68f80b
[Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering ( #54809 )
...
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com >
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-08 12:45:12 -07:00
sychen52 and Cyrus Leung
1b2c591cd0
[Bugfix] speedup nvfp4 kv for FMHA ( #55031 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-08 17:25:42 +00:00
Raya Elena Solano and mgoin
bcca76e7db
[Test] Dequantize NVFP4 KV cache scales in the layout the kernel writes ( #55908 )
...
Signed-off-by: Raya Elena Solano <raya.solano@mbinf.de >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-08 17:13:39 +00:00
8ebc5b0a18
[Bugfix] Fix NVFP4 fused SiLU+mul scale allocation and global scale direction ( #55643 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: jiahao <jxia77@terpmail.umd.edu >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-09-08 08:26:07 -07:00
Wentao Ye
6b15bea080
[Refactor] Remove utils dead code ( #53941 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-08 11:22:09 -04:00
Wentao Ye
07950d4734
[Kimi Bug] Fix kimi k3 startup cuda graph issue with recoverSSM ( #55774 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-08 10:29:53 -04:00
Roberto L. Castro and Codex
6ddbab03de
[4/N][warmup][DSv4] Migrate common attention kernels ( #50176 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-08 10:25:48 -04:00
Sherif Waly and Nicolò Lucchesi
f998862d46
[Bugfix] Fix Kimi K3 loading with interleaved weight streams ( #53379 )
...
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
2026-09-08 16:08:51 +02:00
263c4ff95f
[Bugfix][Spec Decode] Cache the Mamba state at the block-grid position of EAGLE resume ( #53945 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Signed-off-by: Adam Shaver <ashaver@nvidia.com >
Signed-off-by: akshaver <168006157+akshaver@users.noreply.github.com >
Co-authored-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: roikoren755 <26850796+roikoren755@users.noreply.github.com >
2026-09-08 09:59:58 -04:00
414057a3d3
[Bugfix] Apply attention sinks in the Transformers backend ( #52156 )
...
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-08 13:54:07 +00:00
Andy Lo and OpenAI Codex
5ebce2391e
[Bugfix][KV Offload] Respect prefix-cache bypass in SimpleCPUOffload ( #54998 )
...
Signed-off-by: Andy Lo <andy@mistral.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-08 13:14:19 +00:00
Shanshan Shen
25047604fe
[ROCm][Perf] Remove AITER paged-MQA outputs guard for DeepSeek-V4 ( #55808 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-08 06:58:26 -05:00
Kevin H. Luu
34b9899c8f
[CI] Increase ColQwen3 pooling test memory budget on H200 MIG ( #55878 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-09-08 11:39:01 +00:00
Stefano Castagnetta
13cf9e05c1
[Perf][Quant][NVFP4] Prefer W4A4 linear kernels over weight-only ones on SM120/121 ( #55170 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
2026-09-08 15:06:45 +04:00
jimmy-adams
e41a17e606
[ROCm][Quantization] Support AMD Quark per-block FP8 for fused MoE layers ( #52263 )
...
Signed-off-by: huji <huidong.ji@amd.com >
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com >
2026-09-08 10:30:26 +00:00
Kevin H. Luu and Codex
5db652225f
[Bugfix] Validate extension integers before engine serialization ( #55606 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-08 16:30:06 +08:00
Kevin H. Luu and Codex
96eccb8f49
[CI] Restore clean GPU state before OAI Triton MoE tests ( #55453 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-08 01:15:21 -07:00
ahmed xijiaat
81dabe2e5d
[Bugfix][KV Offload] Validate SWA coverage at unaligned cache-hit boundaries ( #55712 )
...
Signed-off-by: xijiade.aihemaiti <3146335281@qq.com >
2026-09-08 10:15:48 +03:00
Adababy and shaolila
5133e1d285
[Frontend] Migrate input-validation errors to VLLMValidationError (harmony/mistral/chat_utils/params) ( #55735 )
...
Signed-off-by: shaolila <shaolila@buaa.edu.cn >
Co-authored-by: shaolila <shaolila@buaa.edu.cn >
2026-09-08 06:45:40 +00:00
aoshen02
755541443f
[Bugfix][ROCm] Mask paged attention V cache padding ( #53856 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
2026-09-08 14:39:23 +08:00
weihan
9ea8f3ffc3
[Bugfix] Honor STEP token pooling in DispatchPooler.for_seq_cls ( #55307 )
...
Signed-off-by: weihan <130119942+krsish@users.noreply.github.com >
2026-09-08 04:13:09 +00:00
7fa2c63796
[Bugfix] Autotune FlashInfer deferred MoE decode kernels before CUDA graph capture ( #55377 )
...
Signed-off-by: ming <ming@inferact.ai >
Co-authored-by: Kimi Code <kimi-code@moonshot.cn >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-08 04:02:23 +00:00
He Liu and Kimi
eb6b619ab2
[Bugfix][KV Offload] Register mixed page sizes in one cache group ( #54756 )
...
Signed-off-by: heliubj18 <chengfeng@ppio.com >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-09-07 20:50:15 -07:00
Jee Jee Li
607a6e48c5
[CI/Build] Fix unused fake implementation testing ( #55797 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-09-08 03:36:18 +00:00
Wei Zhao
a69402aaa8
[Perf] Integrate FlashInfer KDA kernels ( #55364 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-09-07 20:29:30 -07:00