Commit Graph
22547 Commits
Author SHA1 Message Date
b721a4c709 [Feature] Add fixed-token prefill scoring (#54335)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-28 07:50:09 -07:00
xaguilar-amdandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> a7ff44355c [ROCm][Perf] Kimi-K3 Enable sharded latent MoE up-projection under EP (#54956)
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 10:31:29 -04:00
Clinton ThomasLucas Bourtoulemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
cd63bf9996 [Bugfix] Use the correct repository revision for secondary artifact loaders (#57461)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 14:25:40 +00:00
Misha GoinandOpenAI Codex d7f5722d7b [Bugfix][Logging] Preserve application log record factories (#58747)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-28 13:20:16 +00:00
Isotr0py 522101f6ed [Multimodal] Avoid extra d2d for encoder cudagraph with fused input norm (#56711)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-28 21:04:01 +08:00
Shanshan Shen 7ba3df63cb [ROCm][Refactor] Move DeepSeek-V4/V4.1 multi-stream overlap gate to ROCm platform (#58983)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2026-09-28 05:46:28 -07:00
Jiangyun ZhuandOpenAI Codex 20b52e9f5b [Perf][Qwen3.8] Reduce PLE metadata construction overhead (#58114)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-28 20:46:22 +08:00
BaoYunkai f6877f563c [Bugfix][Spec Decode] Implement get_top_tokens() on the ROCm DeepSeek V4 MTP drafter (#57568)
Signed-off-by: BaoYunkai <ybao@amd.com>
2026-09-28 20:16:38 +08:00
Yannick Schnider 8cc9aa5ad3 [Bugfix][Model] Gemma4: register aliased embedding scalars as buffers (#54213)
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
2026-09-28 10:56:31 +00:00
Thien Tran d6cce94fd4 [Perf][Qwen4Exp] Fuse HC down projection and SiLU on NVIDIA (#58957)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-09-28 18:51:38 +08:00
Kevin H. LuuandClaude Opus 5.5 252279314b Revert "[CI] Shard (H200 MIG 35GB / MI355 DPX) Entrypoints Integration (Pooling) into named jobs (#58652)" (#59011)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 03:51:17 -07:00
Louie Tsai ccfd1cea75 [CPU] Include vLLM Recipes tooling in release image to deploy models using vLLM Recipes (#58796)
Signed-off-by: louie-tsai <louie.tsai@intel.com>
2026-09-28 16:46:53 +08:00
Gabriel WuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a2be4d3cc7 [Bugfix][Qwen4Exp] Release the profiling KV cache held by QSA key views (#58961)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 08:11:46 +00:00
Thang NguyenandClaude Opus 5.5 66d2d4b2e7 [CI] Split Dynamic Shapes out of (H200 MIG 35GB) PyTorch Compilation + (MI300) mirror (#58451)
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 00:32:11 -07:00
413ae4b167 [CI] Shard (H200 MIG 35GB / MI355 DPX) Entrypoints Integration (Pooling) into named jobs (#58652)
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: khluu <khluu000@gmail.com>
2026-09-28 00:21:08 -07:00
Eugenio "Jay" ZuccarelliClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
004e37ece2 [Misc] Name each backend and its kernel block sizes in block-size errors (#58557)
Signed-off-by: Eugenio "Jay" Zuccarelli <11176606+jayzuccarelli@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 14:33:18 +08:00
Thien TranandKimi 4694145045 [Skills] Update kernel-microbenchmark to include ROCm (#58646)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi <noreply@moonshot.cn>
2026-09-28 14:18:38 +08:00
Rehan KhanandLi, Jiang <jiang1.li@intel.com> fd2f3fcfa5 [CPU] Add video inferencing via torchcodec on s390x (#58693)
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-09-28 14:16:59 +08:00
zhao, zhenhuiandLi, Jiang <jiang1.li@intel.com> 1668e5976d [CPU] Use accelerator memory API in DiffusionGemma (#58964)
Signed-off-by: zhejiangxiaomai <zhenhui.zhao@intel.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-09-28 13:31:42 +08:00
Turner JabbourandClaude Sonnet 5 2407f405b5 [CI] Allowlist-shrink batch 1: wire 17 root-level tests + drop 5 stale watermarking entries into misc.yaml (#58055)
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-27 22:23:46 -07:00
MOVZX 76af434852 Merge remote-tracking branch 'upstream/main' into patches 2026-09-28 12:06:52 +07:00
af7f9488c2 [Benchmark] Add Responses API backend to vllm bench serve (#54628)
Signed-off-by: QHarshil <harshil_c@hotmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
2026-09-28 05:04:11 +00:00
Kevin H. LuuandClaude Opus 5.5 0da126676e [Bugfix] Fix the two multimodal root tests that fail on main (OpenPangu-VL embed merge, MiMo sink test fixture) (#58900)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 04:52:51 +00:00
dd3bc9c72f [Perf][MRV2] Reuse Mamba/GDN metadata across KV cache groups (#58762)
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
2026-09-28 04:36:58 +00:00
Thien TranandCodex 55de40a2fc [Minimax-M3] Add Encoder CUDA graph support (#58673)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <codex@openai.com>
2026-09-28 04:17:36 +00:00
Flora Feng 6f65fde419 [Bugfix][Frontend] Apply Harmony adjust_request in batched chat completions (#58958)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
2026-09-28 04:12:04 +00:00
jz-yoloandjz-yolo 4a190dc518 [Model] Enable LoRA support for RobertaForSequenceClassification (#58884)
Signed-off-by: jz-yolo <jz-yolo@users.noreply.github.com>
Signed-off-by: Jane Zhu <jane.zhu@slack-corp.com>
Co-authored-by: jz-yolo <jz-yolo@users.noreply.github.com>
2026-09-28 03:50:12 +00:00
Andreas Karatzas 77fbd9e225 [Test][ROCm] Stabilize the mixed OLMoE LoRA test (#58945)
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
2026-09-28 11:47:41 +08:00
zhao, zhenhuiandjiang1.li 5eaa50f151 [CPU] Build CPU wheels on Ubuntu 22.04 with AMX-FP8 support (#58515)
Signed-off-by: zhejiangxiaomai <zhenhui.zhao@intel.com>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
2026-09-28 11:46:02 +08:00
Tianyu Guo 53d2e16a97 [Bugfix][EPD] Skip sampling for encoder-only async steps (#58490)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
2026-09-27 20:35:09 -07:00
Flora Feng 027b6f3a22 [Bugfix][Frontend] Use a fresh parser per choice in non-streaming chat completions (#58939)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
2026-09-28 02:01:35 +00:00
Flora Feng 4d07e90653 [Bugfix][Frontend] Sample batched chat completions from the adjusted requests (#58929)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
2026-09-28 01:46:55 +00:00
Ashraf Bhuiyan b9e2903b89 [Mypy] Fix mypy typing for Ultravox and Unlimited-OCR models (#58239)
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com>
2026-09-27 21:19:04 -04:00
Thien Tran 39d49539de [KimiViT][Perf] Fuse per-layer QK RoPE into one in-place kernel (#58651) 2026-09-27 18:18:06 -07:00
Flora Feng 3184226984 [Bugfix][Frontend] Count Responses reasoning tokens per tool round (#58927)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
2026-09-28 00:34:38 +00:00
31afa4f1bb [ROCm] Bump AITER to v0.1.23 (#58867)
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-27 19:04:58 -05:00
Anton Peganovmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>OpenAI Codex
73859fec58 [Frontend] Switch Python Harmony dependency to oss-harmony (#55128)
Signed-off-by: Anton Peganov <apeganov@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-28 00:43:56 +04:00
Chao Lei 386ac2573e [Bugfix][KV Connector] Retry Mooncake bootstrap registration on timeout (reopens #55763) (#58919) 2026-09-27 19:47:36 +00:00
Fangzhou Ai 187c81b32d [ROCm][Bugfix] Fall back to default GEMM for CPU tensors on ROCm builds (#58923)
Signed-off-by: fai <fangzhouai@gmail.com>
2026-09-27 14:47:07 -05:00
Jiangyun Zhu 231fdb83cc [Perf][MoE] Use fused MiniMax2 routing with non-unit routed scaling (#58880)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
2026-09-27 19:37:33 +00:00
Xun SunandIsotr0py e7900156e1 [Fast Start] Add /health endpoint for the weight cache daemon (#58552)
Signed-off-by: Xun Sun <UNIDY2002@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-09-27 17:48:54 +00:00
2b9b55c7f1 [Bugfix] Support repsonse_format + tool_choice=auto (#56086)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: pablopupo <145598901+pablopupo@users.noreply.github.com>
Co-authored-by: hubunt <150658615+hubunt@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-27 21:17:34 +04:00
Frank WangMisha Goinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Vadim Gimpelson
0376f81530 [Perf][Pooling] Avoid blocking seq_lens GPU-to-CPU copy for pooling in FlashInfer metadata builder (#57214)
Signed-off-by: frankwang28 <frank.wbb@hotmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
2026-09-27 16:06:38 +00:00
Shanshan Shen 44af287ebe [ROCm][Perf] Enable layer-aware CSA2 multi-stream overlap for DeepSeek-V4.1-Flash (#57407)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2026-09-27 23:48:01 +08:00
924707f1bf [CI] Split (H200 MIG 35GB) Spec Decode Speculators + MTP into 4 named jobs (#57237)
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Thang Nguyen <thang.nguyen@inferact.ai>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 05:01:31 -07:00
Nicolò Lucchesi c8d7a7dd13 [Perf][Attention] Remove D2H sync from FlashInfer SM90 sparse MLA plan under async scheduling (#58684)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
2026-09-27 13:06:23 +02:00
lesj0610 521235d6df Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
2026-09-27 18:25:21 +09:00
5567cc1b25 [Mamba] RecoverSSM align: keep a boundary-ending state in its block
A speculative window whose accepted tokens end exactly on a recurrent block
boundary selected the following block-table column for the committed state
(commit plan) and recorded it as the running column (postprocess). That
column can still be unallocated, so the next step reads an uninitialized
page. Select the block that holds the last committed token, matching the
(n - 1) // block convention used elsewhere in align mode. Applies to
Kimi-K3 KDA and to the Qwen GDN/PLE RecoverSSM paths that share the plan.

Fix and test from local-inference-lab/vllm a02f585d (Martin Vit, root cause
reported by astra); the GDN/PLE align tests now expect the same convention.

Signed-off-by: Jürgen Schmied <juergenschmied70@gmail.com>
Co-authored-by: Martin Vit <martin@voipmonitor.org>
Co-authored-by: Claude <noreply@anthropic.com>
2026-09-27 11:12:40 +02:00
Rakul-Chauhanmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Cursor
24c9772d19 [Attention][CPU] Use zentorch SDPA for CPU MLA prefill (#54967)
Signed-off-by: Rakul Chauhan <rakul.chauhan@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-27 08:30:55 +00:00
Jürgen Schmied ed2a9665ea Merge branch 'main' into gdn-recoverssm 2026-09-27 08:01:32 +02:00