Commit Graph
22038 Commits
Author SHA1 Message Date
31afa4f1bb [ROCm] Bump AITER to v0.1.23 (#58867)
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-27 19:04:58 -05:00
Anton Peganovmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>OpenAI Codex
73859fec58 [Frontend] Switch Python Harmony dependency to oss-harmony (#55128)
Signed-off-by: Anton Peganov <apeganov@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-28 00:43:56 +04:00
Chao Lei 386ac2573e [Bugfix][KV Connector] Retry Mooncake bootstrap registration on timeout (reopens #55763) (#58919) 2026-09-27 19:47:36 +00:00
Fangzhou Ai 187c81b32d [ROCm][Bugfix] Fall back to default GEMM for CPU tensors on ROCm builds (#58923)
Signed-off-by: fai <fangzhouai@gmail.com>
2026-09-27 14:47:07 -05:00
Jiangyun Zhu 231fdb83cc [Perf][MoE] Use fused MiniMax2 routing with non-unit routed scaling (#58880)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
2026-09-27 19:37:33 +00:00
Xun SunandIsotr0py e7900156e1 [Fast Start] Add /health endpoint for the weight cache daemon (#58552)
Signed-off-by: Xun Sun <UNIDY2002@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-09-27 17:48:54 +00:00
2b9b55c7f1 [Bugfix] Support repsonse_format + tool_choice=auto (#56086)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: pablopupo <145598901+pablopupo@users.noreply.github.com>
Co-authored-by: hubunt <150658615+hubunt@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-27 21:17:34 +04:00
Frank WangMisha Goinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Vadim Gimpelson
0376f81530 [Perf][Pooling] Avoid blocking seq_lens GPU-to-CPU copy for pooling in FlashInfer metadata builder (#57214)
Signed-off-by: frankwang28 <frank.wbb@hotmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
2026-09-27 16:06:38 +00:00
Shanshan Shen 44af287ebe [ROCm][Perf] Enable layer-aware CSA2 multi-stream overlap for DeepSeek-V4.1-Flash (#57407)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2026-09-27 23:48:01 +08:00
924707f1bf [CI] Split (H200 MIG 35GB) Spec Decode Speculators + MTP into 4 named jobs (#57237)
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Thang Nguyen <thang.nguyen@inferact.ai>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 05:01:31 -07:00
Nicolò Lucchesi c8d7a7dd13 [Perf][Attention] Remove D2H sync from FlashInfer SM90 sparse MLA plan under async scheduling (#58684)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
2026-09-27 13:06:23 +02:00
Rakul-Chauhanmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Cursor
24c9772d19 [Attention][CPU] Use zentorch SDPA for CPU MLA prefill (#54967)
Signed-off-by: Rakul Chauhan <rakul.chauhan@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-27 08:30:55 +00:00
Juntian LiuandClaude Opus 5.5 a9eafde59c [Perf][DSv4.1] Fuse small-batch WO-A with inverse RoPE and MXFP8 quant on SM100/SM103 (#58634)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 21:16:12 -07:00
VarshithandClaude fba47397f3 [Bugfix] V1: clear stale allowed_token_ids mask in InputBatch.condense (#43931)
Signed-off-by: Varshith <kvarshithgowda@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-09-27 03:44:54 +00:00
YannikHintereggerandxijiade.aihemaiti fff03267da [Bugfix][NIXL] Release a dead peer's NIXL state without waiting for TTL (#50047)
Signed-off-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com>
Co-authored-by: xijiade.aihemaiti <3146335281@qq.com>
2026-09-26 20:30:09 -07:00
Hongxia Yang 3137ff0773 [ROCm][Kimi-K3] Make VLLM_ROCM_USE_AITER_MOE_SITUV2 select a4w4/a8w4/a16w4 (#58201)
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
2026-09-27 02:26:49 +00:00
Lio Einaudiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 0c87a197b8 [Bugfix] Disable sequence parallelism / async TP under batch invariance and add a TP regression test (#56377)
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-26 22:19:08 -04:00
Thien Tran 7a877ae34d [Qwen3.8-Flash-Next] Avoid memory fragmentation in QSA indexer logits workspace (#57105)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-09-27 10:11:52 +08:00
eb0f2ca37f [mooncake] support CUSTOM_MEM_POOL in vllm (#49300)
Signed-off-by: bruce.xu <bruce.xb@alibaba-inc.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: bruce.xu <bruce.xb@alibaba-inc.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-27 00:26:05 +00:00
mevince cd1daf58a9 [Metrics][KV Offload] Add Prometheus metrics for SimpleCPUOffloadConnector (#57251)
Signed-off-by: Vincent <vincexxchan@gmail.com>
2026-09-26 17:17:11 -07:00
4be061c5af [Kernel] Bump FlashKDA to keep the recurrent state in fp32 (#58846)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
2026-09-26 22:20:44 +00:00
927c87b347 [PCP][DCP] Support DCP target model with non-DCP Dspark (#56723)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-26 18:12:35 -04:00
Yongye ZhuandClaude Opus 5.5 77871126f9 [Kernel][DSV4.1] Fuse MoE finalize into the TP all-reduce + mHC boundary (#58586)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 03:08:36 +08:00
Cyrus Leung 379e9a1ea8 [Security] Harden message sanitization (#58832)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2026-09-26 16:28:32 +00:00
Kevin H. Luu 7d8c5fe9a9 [Bugfix][DSV4.1] Avoid host sync in ViT CUDA graph replay metadata (#58499) 2026-09-26 11:37:36 -04:00
Andreas Karatzas 0239bd2549 [CI] Stabilize batch submission in full CUDA graph tests (#58810) 2026-09-26 11:35:56 -04:00
Wentao Ye 3576691c42 [GLM5.3 Bug] Fix sparse indexer attn topk backend selection (#58594)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-09-26 11:35:28 -04:00
Misha Goin 79ac28c5bd [CI] Reduce CUDA graph mode test overhead (#58749) 2026-09-26 11:22:33 -04:00
Ashraf Bhuiyan 5113cf9d16 [Mypy] Fix mypy typing for Qwen and Qianfan models (#58046)
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com>
2026-09-26 14:31:51 +00:00
Robert ShawandRobert Shaw dfab504333 [Bugfix][Frontend] Detect Anthropic inline-system merge against the resolved chat template (#58754)
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
2026-09-26 14:00:56 +00:00
Robert ShawandRobert Shaw d64f277c54 [Bugfix] Fix Anthropic Thinking Disabled with P/D (#58786)
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
2026-09-26 13:50:57 +00:00
Willian Zandmgoin ad6817b68d [Perf][MoE] Index expert mapping lookups in RoutedExperts.load_weights (#58720)
Signed-off-by: Willian <willian@willian.email>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
2026-09-26 06:48:24 -07:00
Wentao Ye bf93e1f92c [Refactor] Remove dead tests utils (#58803)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-09-26 09:16:53 -04:00
Itay AlroyandTyler Michael Smith 2bb7605ae1 [Elastic EP] Fix EPLB load statistics during scaling (#58473)
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
2026-09-26 12:05:37 +00:00
Juan Pérez de AlgabaandCursor 31f2e70cd3 [Security] Gate per-request multimodal processor kwargs (#58830)
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-26 11:10:35 +00:00
2f41e00235 [CI] Split (B200) Miscellaneous Kernels into mHC, FLA Ops and Misc named jobs (#58609)
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-09-26 04:10:14 -07:00
a4eb3f25d6 [Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer (#57263)
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
2026-09-26 17:01:17 +08:00
Aarushi JainandAndreas Karatzas 3b4566c5cf [CI] Only isolate the registry tests that need a fresh process (#58764)
Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-26 14:03:45 +08:00
ddd6fbca14 [Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasoning effort mappings (#58316)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: zhec <chengyunfei@ruc.edu.cn>
2026-09-26 02:18:58 +00:00
xiao-llmandCursor 5840d95284 [Bugfix][ROCm] AMD-Quark mixed-precision DeepSeek-V4.1 support (#57071)
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-25 20:51:18 -05:00
Ke Wenmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>aoshen02
b6d2b4fc25 [RL] Add sharding-aware NCCL M2N weight transfer (#51520)
Signed-off-by: Ke Wen <kwen@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
2026-09-26 01:38:53 +00:00
Shuolei Wang 4bb804cc5b [Perf][DSv4.1] Shard the Engram wkv projection across TP ranks (#58678)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
2026-09-26 01:15:57 +00:00
Chandra MouryaandVS Chandra Mourya 8a2364605c [Core] Bound draft-token RPC waits by the execute-model timeout (#58779)
Signed-off-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com>
Co-authored-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com>
2026-09-26 00:40:22 +00:00
Baljinder HothiandFlora Feng fdcce47e9b [Bugfix][Frontend] Fix Inkling tool name leaking into content after reasoning (#58792)
Signed-off-by: Baljinder Hothi <baljinder.hothi@cohere.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
2026-09-26 00:38:52 +00:00
da1aaec31f [CI] Split (H200 MIG/MI300) Basic Correctness into named jobs (#57054)
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-25 17:05:33 -07:00
Elvir CrnčevićandFlora Feng 9bf44c4013 [Bugfix][Reasoning] Count Kimi K3 reasoning tokens (#58372)
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
2026-09-25 23:29:56 +00:00
Rui ZhuandClaude 974cb65155 [Bugfix] V1: fix allowed_token_ids_mask aliasing in InputBatch.swap_states (#48419)
Signed-off-by: Rui Zhu <rui.zhu.rz399@yale.edu>
Co-authored-by: Claude <noreply@anthropic.com>
2026-09-25 23:23:11 +00:00
Ricardo-M-LandClaude Opus 4.6 b040321b33 fix: perf: use startswith(x, i) instead of string slicing to avoid O(N^2) (#52580)
Signed-off-by: Ricardo-M-L <ricardoporsche001@icloud.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-09-25 23:22:13 +00:00
Taneem IbrahimandWentao Ye 6ec2f84e1a [Pooling] Preserve BERT-family heads for raw logits (#57664)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
2026-09-25 19:10:14 -04:00
Linkun 614b55f0b6 [AuxOutput] Only require Model Runner V2 on GPU platform (#58205)
Signed-off-by: Linkun Chen <github@lkchen.net>
2026-09-25 15:55:21 -07:00