Commit Graph
14883 Commits
Author SHA1 Message Date
Cheng RuiandCodex 2a4e3cc3db [Bugfix][KV Offload] Ensure tracker progress for oversized offers (#54759)
Signed-off-by: Cheng Rui <286040359@qq.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-02 14:09:29 +03:00
Nicolò Lucchesiandcjackal f81eb41934 [Bugfix] adjust_dcp_kv_cache_interleave_size for NixlConnector only (#54803)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
2026-09-02 12:31:14 +02:00
Isotr0py 1c26e57d3c [Bugfix] Raise for unavailable piecewise CUDA graphs (#54782)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-02 08:52:24 +00:00
jackLeiandOpenAI Codex 584e8f0dda [Model] Fix GLM-OCR MTP weight loading (#49869)
Signed-off-by: jackLei0901 <42642542+jackLei0901@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-02 08:38:36 +00:00
Andreas KaratzasandOpenAI Codex e52407ef4c [ROCm][CI] Add MTP and other spec-decode acceptance coverage (#53399)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-02 03:27:05 -05:00
Fabian JoswigandYifan Qiao 8052102c21 [Bugfix] Fix cross-batch buffer race corrupting DiskBackend loads (#51667)
Signed-off-by: Fabian Joswig <fjosw@users.noreply.github.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
2026-09-02 08:18:41 +00:00
Chauncey b2558f8da3 [Bugfix] Fix launch render hanging on shutdown (#54913)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-09-02 08:15:35 +00:00
Itay EtelisItay EtelisOr Ozerimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
df09c76735 [KV Connector][Offloading] Look through UniformTypeKVCacheSpecs in the canonical portability gate (#51690)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-02 10:43:26 +03:00
Yusuke OdaandChauncey 798b557e06 [Frontend] Add site-packages support for reasoning/tool parser plugins (#45241)
Signed-off-by: odashi <yusuke.oda@predicate.jp>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
2026-09-02 07:41:13 +00:00
Francisco Javier Arceo c00091e026 [Frontend] Gate scale-out endpoints behind opt-in flag (#54579)
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
2026-09-02 14:31:14 +08:00
Flora Feng 300f688321 [Bugfix] Implicitly close DeepSeek DSML parameters (#54838)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
2026-09-02 14:13:36 +08:00
Thien TranandCodex f870b92976 [Qwen3.8-Flash-Next] Fuse Qwen4Exp PLE kernels (#54517)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <codex@openai.com>
2026-09-02 12:32:16 +08:00
Gabriel WuandKimi Code 396c5a5632 [Bugfix] Lazy-import FlashInfer PCIe IPC all-reduce in kernel_warmup (#54869)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
2026-09-02 04:31:03 +00:00
Harry Huang 1e300895ac [Qwen4] validate FP8 PLE weight scale after loading (#54722)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
2026-09-02 04:30:58 +00:00
ee3c00bbf4 [Performance] Register Triton W4A16 GEMM as a custom op (#51453)
Signed-off-by: giuseppegrossi <ggrossi@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-01 22:55:34 -05:00
Isotr0py 1d8d7a3965 [Bugfix] Fix RoPE construction for deepseek-v4 sparse SWA layers (#54815)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-01 20:48:50 -07:00
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> a566ea7e8f [Perf] Avoid more h2d copies from non-pinned tensors (#54660)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-01 20:37:09 -07:00
Yongye ZhuandOpenAI Codex 3ba9907a1d [Kimi-K3] Overlap low-M TP8 KDA projections (#54697)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-02 03:35:28 +00:00
12b9573c98 [Bugfix][KV Offloading] Fix eager SimpleCPUOffload cache registration and final flush (#53532)
Signed-off-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com>
Co-authored-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
2026-09-02 03:23:09 +00:00
Divakar VermaandAndreas Karatzas dbf1a044ea [Bugfix] Fix ColQwen3.5 pooler projector initialization (#54847)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-02 10:35:43 +08:00
Thien Tran 003e34341a [Qwen3.8-Flash-Next] Separate prefill and decode paths for QSA indexer (#54513)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-09-02 09:08:10 +08:00
mingg26andOpenAI Codex 259a209bfa [Kimi-K3][Perf] Prefetch ll_bf16 router weights for M=1 (#53524)
Signed-off-by: ming <ming@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-01 21:40:19 +00:00
Wentao Ye fc72fc39ac [Kimi Bug] Fix cannot access local variable 'active_non_spec_mask_cpu' (#54781)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-09-01 21:08:50 +00:00
Micah Williamson ce6a283c09 [Bugfix] Restore weight_dtype in QuarkW8A8Fp8MoEMethod to fix GPT-OSS FP8 MoE weight loading (#54824)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
2026-09-01 16:08:11 -05:00
Jeff (Junze) MaandOpenAI Codex 18c53727ce [Bugfix][KV Connector] Fix DecodeBench DCP block selection (#54679)
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-01 20:31:30 +00:00
7a977c0699 [ModelOpt] Redesign the LinearMethod classes using the generic QuantKey-driven method (#49381)
Signed-off-by: Juhi Mittal <juhim@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
2026-09-01 16:28:56 -04:00
Zhewen Liandzhewenl 5cc32fbdfa [Bugfix][KV Connector] Fix Mooncake physical-block transfer length (#54272)
Signed-off-by: zhewenl <zhewenl@users.noreply.github.com>
Co-authored-by: zhewenl <zhewenl@users.noreply.github.com>
2026-09-01 13:23:46 -07:00
7cb9a88f50 [Bugfix][Mooncake] Offload producer partial tails on request finish (#52832)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-01 13:23:35 -07:00
Alberto MassiddaandAlberto Massidda 55178f2d09 [Bugfix][Profiler] Fix API server crash on double /stop_profile (#51678)
Signed-off-by: Alberto Massidda <amassidda@nvidia.com>
Co-authored-by: Alberto Massidda <amassidda@nvidia.com>
2026-09-01 13:05:04 -07:00
stefankoncarevic 73723b707f [ROCm][MoE] Fix gfx950 block scale swizzle for AITER Triton MXFP4 W4A16 (#54773)
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
2026-09-01 19:48:45 +00:00
Julien Debache ab54f5bd83 [Chore] Remove redundant _pack_topk_ids_weights_kernel in TrtLLM NvFP4 MoE (#46872)
Signed-off-by: jdebache <jdebache@nvidia.com>
2026-09-01 15:30:11 -04:00
Luyi XiaoandOpenAI Codex aa71f9bcc4 [Bugfix] Log platform plugin detection failures (#52285)
Signed-off-by: Luyi Xiao <xiao.luy@northeastern.edu>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-01 19:11:52 +00:00
Wentao Ye 6c58595c2d [Feature] Avoid flashinfer autotune each time when vllm source change (#54794)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-09-01 15:03:30 -04:00
waizuichougou d98bb2a879 [Bugfix][Frontend] Honor skip_decoder_start_token in async encoder-decoder rendering (#54799)
Signed-off-by: waizuichougou <2082431897@qq.com>
2026-09-01 19:00:28 +00:00
Pratham ShirbhateandCyrus Leung 0ad5652a52 [Bugfix][Frontend] Restore the chat template content format mismatch warning (#54622)
Signed-off-by: Pratham Shirbhate <prathamshirbhate1909@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-09-01 18:17:54 +00:00
fxmarty-amdandAndreas Karatzas d1c15e589d [CI] Speed up quantization test group (#53291)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-01 13:15:24 -05:00
e7cf4730d6 [Bugfix] Drop incomplete tool-call markup in non-streaming to match streaming (#47562)
Signed-off-by: Suvrakamal Das <jyb2084.usa@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
2026-09-01 18:11:36 +00:00
周峥andGuanyi Chen c866ba9d11 [KV Connector] Support heterogeneous TP sharing in Mooncake Store Connector (#53129)
Signed-off-by: Guanyi Chen <939416532@qq.com>
Signed-off-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: Zheng Zhou <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: zane <zhouzh93@mail2.sysu.edu.cn>
Co-authored-by: Guanyi Chen <939416532@qq.com>
2026-09-01 10:18:06 -07:00
2fe5cef35e [Fix] Fix FSE compatibility detection for Quark-produced models (#54573)
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: xuebwang-amd <xuebwang@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-01 12:12:20 -05:00
Roberto L. CastroCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
514c7314a0 [Perf][Kernel] Initialize NVFP4 padding in quant kernel (#53568)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-01 09:49:07 -07:00
Eli Nabro Yeongmin OhandCodex cdefd9d499 [Bugfix] Support Sentence Transformers 5.4+ serialized configs (#54533)
Signed-off-by: maireneu <maireneu@gmail.com>
Co-authored-by: Codex <codex@openai.com>
2026-09-01 15:53:38 +00:00
Harry MellorandClaude Opus 5 f1e5fdd7f2 [Transformers backend] Replace vocab embeddings in recursive_replace (#54760)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-01 15:35:24 +00:00
Misha GoinandOpenAI Codex 82b7d49a6e [MoE] Generalize masked activation for padded layouts (#51217)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-01 11:34:13 -04:00
Taneem Ibrahim 76f3249fbd [Mypy] Fix typing for M models (#54262)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-09-01 11:26:00 -04:00
Taneem Ibrahim d9eb4e344f [Bugfix] Reject tokenless chat and audio streams (#54708)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-09-01 11:24:19 -04:00
Chauncey 339e16cbb6 [Bugfix] Support MCP SDK 2.x tool input schemas (#53870)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-09-01 11:02:39 -04:00
Luke AlonsoandLucas Wilkinson 0d4ad47981 [Kernel] Add B12X causal paged attention backend (#52017)
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
2026-09-01 10:42:02 -04:00
Andrii SkliarandAndrii Skliar adebc41b7e [Mamba] Add FlashInfer ReplaySSM backend (#52506)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
2026-09-01 10:26:43 -04:00
Ilya Markov 25efcfa788 [Attention] Enable adaptive verification for FLASHINFER_MLA_SPARSE_DSV4 (#52724)
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
2026-09-01 13:43:28 +00:00
Jan Kwieciński 1f1f628859 [Feat][MM Hashing] include media_io_kwargs in multi-modal hashes (#54241)
Signed-off-by: Jan Kwiecinski <jkwiecinski@nvidia.com>
2026-09-01 13:17:01 +00:00