Commit Graph
20981 Commits
Author SHA1 Message Date
Taoyu ZhuandShanshan Shen ce6c241ddc [ROCm][Perf][M3] Fused allreduce+GemmaRMSNorm fast path (#54787)
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
2026-09-08 07:21:49 +00:00
ahmed xijiaat 81dabe2e5d [Bugfix][KV Offload] Validate SWA coverage at unaligned cache-hit boundaries (#55712)
Signed-off-by: xijiade.aihemaiti <3146335281@qq.com>
2026-09-08 10:15:48 +03:00
Adababyandshaolila 5133e1d285 [Frontend] Migrate input-validation errors to VLLMValidationError (harmony/mistral/chat_utils/params) (#55735)
Signed-off-by: shaolila <shaolila@buaa.edu.cn>
Co-authored-by: shaolila <shaolila@buaa.edu.cn>
2026-09-08 06:45:40 +00:00
aoshen02 755541443f [Bugfix][ROCm] Mask paged attention V cache padding (#53856)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
2026-09-08 14:39:23 +08:00
Subhash Polisetti 782f36cd0c [Bugfix][InternVL] Stop the video parser consuming image_embeds (#55779)
Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
2026-09-08 05:19:41 +00:00
6c73b08dec [Bugfix] reject string file on translation like transcription (#55618)
Signed-off-by: oskii <oscarchambersweb@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-09-08 05:02:53 +00:00
weihan 9ea8f3ffc3 [Bugfix] Honor STEP token pooling in DispatchPooler.for_seq_cls (#55307)
Signed-off-by: weihan <130119942+krsish@users.noreply.github.com>
2026-09-08 04:13:09 +00:00
mingg26Kimi Codemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
7fa2c63796 [Bugfix] Autotune FlashInfer deferred MoE decode kernels before CUDA graph capture (#55377)
Signed-off-by: ming <ming@inferact.ai>
Co-authored-by: Kimi Code <kimi-code@moonshot.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-08 04:02:23 +00:00
He LiuandKimi eb6b619ab2 [Bugfix][KV Offload] Register mixed page sizes in one cache group (#54756)
Signed-off-by: heliubj18 <chengfeng@ppio.com>
Co-authored-by: Kimi <noreply@moonshot.cn>
2026-09-07 20:50:15 -07:00
Jee Jee LiandYongye Zhu 9c297e3b8b [Kernel] PDL enablement for fusedQKNormRopeKernel (#55755)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
2026-09-07 20:49:03 -07:00
Jee Jee Li 607a6e48c5 [CI/Build] Fix unused fake implementation testing (#55797)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-09-08 03:36:18 +00:00
Wei Zhao a69402aaa8 [Perf] Integrate FlashInfer KDA kernels (#55364)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
2026-09-07 20:29:30 -07:00
leo 12579c7f01 [Kernel][Perf] Tune H20 block-FP8 MoE low-batch configs (+21%) (#54668)
Signed-off-by: liuyao0322 <yaoliu548926@gmail.com>
2026-09-08 03:20:21 +00:00
Wentao Ye 869f78732b [Kimi Bug] Fix kimi k3 AssertionError assert 0 <= checkpoint_idx < len(blocks) (#55747)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-09-07 22:34:48 -04:00
Andreas Karatzas 6b5a12c0f8 [ROCm][CI] Enable HY-V4 model initialization on ROCm (#54405)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
2026-09-07 20:55:44 -05:00
Artur FierkaandClaude 9dbdf8e9af [Bugfix][Quantization][MoE] Normalise an unset group_size on the compressed-tensors WNA16 MoE path (#53163)
Signed-off-by: Artur Fierka <artur.fierka@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-09-08 09:42:46 +08:00
Woosuk KwonandOpenAI Codex 06334fae2d [Build] Remove obsolete TPU Dockerfile (#55376)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-07 18:07:27 -07:00
54da70c1eb [Feature] Support EAGLE3 for Sarvam (#53052)
Signed-off-by: mohit-sarvam <mohit@sarvam.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-08 00:08:35 +00:00
Taneem Ibrahim fdfa0a1659 [Pooling] Honor request_id from request bodies (#55665)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-09-07 19:55:20 -04:00
Taneem Ibrahim 252ed87621 [Bugfix] Restore Responses validation error boundary (#55761)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-09-07 22:12:18 +00:00
Kevin H. LuuandCodex 537af2c3a4 [CI] Recover empty multi-node Docker networks and finish partial cleanup (#55454)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-07 12:53:13 -07:00
Kevin H. LuuandCodex 58f09211dd [CI] Synchronize shared offload unlink test before observing pathname (#55604)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-07 12:52:36 -07:00
Sahil Patel 51da0ca66c [CI/Build] Unskip ColQwen3 multimodal pooling tests on Transformers v5 (#55588)
Signed-off-by: Sahil <sip4818@gmail.com>
2026-09-07 15:15:39 +00:00
Zeyu Yang ad2f18e8af [CI] reduce npu CI use time and add timeout (#55731)
Signed-off-by: yangzeyu <yangzeyu7@huawei.com>
2026-09-07 22:51:09 +08:00
am-cohereCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
70584f69b1 [Model] Add Cohere Compass model (#54774)
Signed-off-by: am-cohere <312513976+am-cohere@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 07:36:49 -07:00
AbhiandAntigravity 7cc89a7dda [Tests] Update SarvamMLA transformers v5 compatibility reason to hf (#55728)
Signed-off-by: Abhi <108084481+Aj2280@users.noreply.github.com>
Co-authored-by: Antigravity <antigravity@google.com>
2026-09-07 14:27:07 +00:00
Nicolò Lucchesi cd64c2dea9 [Docs] Clarify admission control limits apply server-wide, not per DP rank (#55124)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
2026-09-07 14:08:48 +00:00
f2d45f26bd [Bugfix][Gemma] Conditionally create KV projections/norms on KV-shared layers (#54917)
Signed-off-by: Josephasafg <ajgard7@gmail.com>
Co-authored-by: minyichen <96753147+cyc00518@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
2026-09-07 15:04:00 +01:00
Roberto L. CastroCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
42801b3a6b [3/N][warmup][DSv4] Migrate FA4 MLA and shared CuTeDSL kernels (#53565)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 09:36:35 -04:00
Roberto L. CastroCodexOpenAI CodexLucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
1713b9866a [2/N][warmup][DSv4] Migrate sequence and DCP kernels (#53564)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 09:35:54 -04:00
Taneem Ibrahim ecd600d91e [Pooling] Honor max_embed_len for chunked embeddings (#55551)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-09-07 09:31:40 -04:00
Adababyandshaolila 167858ea17 [Frontend] Migrate Responses harmony input validation to VLLMValidationError (#55701)
Signed-off-by: shaolila <shaolila@buaa.edu.cn>
Co-authored-by: shaolila <shaolila@buaa.edu.cn>
2026-09-07 13:30:11 +00:00
Soumyajit GhoshSOUMYAJIT GHOSHmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
b339d75a41 [Bugfix][Spec Decode] Resolve n_predict from text_config for Qwen3.5 multimodal MTP (#55369)
Signed-off-by: SOUMYAJIT GHOSH <23051387@kiit.ac.in>
Signed-off-by: Soumyajit Ghosh <jobsoumyajit6124@gmail.com>
Co-authored-by: SOUMYAJIT GHOSH <23051387@kiit.ac.in>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 13:28:26 +00:00
Francisco Javier ArceoandOpenAI Codex 6a2a2bb02b [Frontend] Add stateless /v1/responses/render endpoint (#50195)
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-07 12:50:42 +00:00
Artur Fierka 5e6f6a8ed4 [Bugfix][Quantization][XPU] Fix moe_wna16 linear weight loading (#52651)
Signed-off-by: Artur Fierka <artur.fierka@intel.com>
2026-09-07 20:20:17 +08:00
Kam BasraandClaude 4a806d08ee [Bugfix] OffloadingConnector: stop zeroing offload hits under MTP/EAGLE spec decode (#52771)
Signed-off-by: Kam Basra <kameldipbasra@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-09-07 15:16:05 +03:00
Thien Tran 94e26dd3dd [Qwen3.8-Flash-Next] Support FP8 indexer cache for QSA (#54890)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-09-07 12:15:45 +00:00
Sean Westfall 58ad1f3b89 [Docs] Add OLMo 2 to batch-invariance tested models (#55691)
Signed-off-by: Sean Westfall <sean.westfall@gmail.com>
2026-09-07 09:56:49 +00:00
Chaojun Zhang 8648446515 [XPU][LoRA] Support LoRA for DeepSeek V4 on XPU (#53689)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
2026-09-07 17:44:12 +08:00
Marceli FylcekandClaude Opus 5 9b85112e13 [XPU] Route grouped_topk to the fused _moe_C kernel on XPU (#53580)
Signed-off-by: Marceli Fylcek <marceli.fylcek@intel.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 17:43:38 +08:00
Adababyandshaolila 7dbe386833 [Frontend] Migrate Responses API validation errors to VLLMValidationError (#50257)
Signed-off-by: shaolila <shaolila@buaa.edu.cn>
Co-authored-by: shaolila <shaolila@buaa.edu.cn>
2026-09-07 17:33:51 +08:00
Andreas KaratzasandOpenAI Codex e476556189 [ROCm][CI] Add attention-sink support to ROCm AITER sparse MLA (#54404)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-07 09:26:47 +00:00
Chaojun Zhang 392db567b2 [CI] [Test] skip test_wna16_cuda_high_bit_skips_humming on non-CUDA platforms (#55660)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
2026-09-07 16:57:15 +08:00
AlexHuang 49eb2accf0 [KVConnector] Guard lmcache_mp_connector state transition with num_external_tokens (#47505)
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Signed-off-by: Alex-ai-future <Alex-ai-future@users.noreply.github.com>
2026-09-07 08:19:07 +00:00
Gabriel Wumergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Jee Jee LiYongye Zhu
5893426b88 [Bugfix] DSv4 MXFP4 selector: stop narrowing explicit aliases to their BF16 variant (#53586)
Signed-off-by: Zihua Wu <zihuaw@nvidia.com>
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
2026-09-07 07:19:13 +00:00
Zhewen Liandzhewenl 34b1e9f7a6 [Bugfix][MooncakeStore] Fix finish-time save crash on hybrid models (#54643)
Signed-off-by: zhewenl <zhewenl@users.noreply.github.com>
Co-authored-by: zhewenl <zhewenl@users.noreply.github.com>
2026-09-07 07:18:30 +00:00
Andreas Karatzas 195bc9c4a1 [CI][ROCm] Temporarily skip unsupported HY-V4 initialization (#55653)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
2026-09-07 15:17:36 +08:00
6fbb00b188 [EPD] Add ECMooncakeConnector for encoder cache over Mooncake TransferEngine (#41567)
Signed-off-by: Teng Ma <sima.mt@alibaba-inc.com>
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-07 06:21:43 +00:00
Andreas Karatzas f7f060d253 [Bugfix][Audio] Restore soundfile-first automatic decoding (#55642)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
2026-09-07 06:21:02 +00:00
zofiaandKunshang Ji ed29dfae6e [XPU] Use fused_input_norm kernel in FusedInputNorm (#52945)
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-09-07 13:44:31 +08:00