Commit Graph
21722 Commits
Author SHA1 Message Date
Micah Williamson d6696e2385 [ROCm] Begin Deprecation Window for CUDA_VISIBLE_DEVICES on ROCm (#46636)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
2026-06-24 20:40:28 +00:00
Chauncey 84c2f9f0fb [Frontend] Fix Kimi K2 tool call IDs for required tool choice (#46344)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-06-24 19:59:40 +00:00
shivamprMatthew Bonannimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
49f2104c53 [Feature] Support DCP with FP8 KV cache in MLA decode path (#44044)
Signed-off-by: shivampr <shivampr.dev@gmail.com>
Signed-off-by: Shivam <shivamprasad91@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-24 19:28:17 +00:00
Ashwin PhadkeandHarry Mellor d511b5bae9 Chore: Fix minor doc sentence, grammar, quote errors (#40469)
Signed-off-by: Ashwin Phadke <23502062+ashwin-phadke@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-06-24 18:58:24 +00:00
3c43237233 [Bugfix][Model Runner V2][Spec Decode] Fix int32 offset overflow in sampler kernels (#46560)
Signed-off-by: xiaojun.wei <jessiewei747@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
2026-06-24 11:00:57 -07:00
HDCharlesandClaude Opus 4.6 56ca5997ea Humming support for 2/3/5/6/7-bit pack-quantized weight-only inference (#46389)
Signed-off-by: HDCharles <charlesdavidhernandez@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-24 13:53:54 -04:00
Aarushi Jain cf57311187 Run DeepSeek-V2-Lite prefetch-offload eval eager on ROCm (#46386)
Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com>
2026-06-24 12:25:57 -05:00
Lucas Wilkinson e7df232288 [KV Offload] Gate packed HMA KV cache on cross-layer config (#46252)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
2026-06-24 11:55:30 -04:00
Micah WilliamsonandMatthew Wong b3a688cb9e [ROCm] Fix OOB During Model Warmup With ROCM_ATTN and MRV2 (#46548)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
2026-06-24 10:53:21 -05:00
Wentao Ye 1cd3e0e945 [Bug] Fix IndentationError: expected an indented block after 'with' statement (#46627)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-06-24 23:14:17 +08:00
Yiwei Hu f889325c51 [KV Offload] Use background thread for mmap / cpu_tensors pinning (#45850)
Signed-off-by: Sorryhorizon <arikara6666@gmail.com>
2026-06-24 18:13:27 +03:00
Ronen SchafferandOr Ozeri bb61177e49 [KV Offloading] Replace bool|None lookup return with LookupResult enum (#46363)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
2026-06-24 18:06:08 +03:00
Walter Beller-Moralesandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 7f99e80c3b [Perf][ThinkingBudget] reduce search space for thinking tokens (#46425)
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-24 23:02:25 +08:00
Liangliang Maandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2801b11156 [Test] Pin block_size in auto-fit max_model_len test (#45914)
Signed-off-by: Ma, Liangliang <liangliang.ma@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-24 22:56:21 +08:00
Wentao YeandTJian 007b5a52ed [Log] Update to log once (#46511)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
2026-06-24 14:45:16 +00:00
Cyrus Leung 24d5186138 [Bugfix] Re-enable FP8 MoE on NVIDIA Thor (#46339)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2026-06-24 07:35:46 -07:00
Nemani Harsha Vardhan 7dc036058b [Doc] Document Qwen3.6 (dense + MoE) ViT CUDA graph support (#44720)
Signed-off-by: harsha20032020 <nhvardhan2020@gmail.com>
2026-06-24 14:35:08 +00:00
djramicandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 61ee183d28 [ROCm] Fix AITER FP8 quantization schema tests (#46414)
Signed-off-by: Djordje Ramic <djoramic@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-24 22:29:19 +08:00
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 84c62e1cbd [Model Runner V2][MM] Support EVS (#46535)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-24 10:18:56 -04:00
Fadi Arafeh 061043eaca [CPU][Perf] Accelerate unquantized MoE for AArch64 (#46353)
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
2026-06-24 14:14:35 +00:00
93ec645878 [Bugfix] Fix illegal memory access from a forward during a partial wake_up (#44483)
Signed-off-by: Meihan-chen <zr010426ztt@outlook.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
2026-06-24 22:12:23 +08:00
Kunshang Ji 563c628968 [XPU] bump up vllm_xpu_kernels to v0.1.10.1 (#46607)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
2026-06-24 10:05:31 -04:00
LynnandClaude Opus 4.8 0bc479e6eb [Perf][LoRA] Replace O(n) list.index() with a dict in convert_mapping (#46542)
Signed-off-by: Lynn <lynnhe02@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 21:41:46 +08:00
Tae Jeong 62890e204c Fix duplicated logging when loading a corrupt or partial video (#46467)
Signed-off-by: hhhhhhhhhhhhhhhhho <man2719@naver.com>
2026-06-24 06:14:13 -07:00
Nicolò Lucchesi a2cb08b3d5 [Misc][PD] Disable bidirectional xfer mode for NixlPushConnector (#46473)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
2026-06-24 21:14:05 +08:00
Rui YinandOr Ozeri cf9fd6457e Fix KV offload request-finished lifecycle contract (#46284)
Signed-off-by: test test <2260891073@qq.com>
Signed-off-by: Rui Yin <2260891073@qq.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
2026-06-24 15:42:40 +03:00
Kunshang Ji d4448b511d [XPU][Docker] switch to ubuntu 24.04 as base image (#45973)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
2026-06-24 20:39:20 +08:00
Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> f1a6703edd [Bugfix][Config] Keep pydantic validation for fields with a TYPE_CHECKING Literal alias (#46220)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-24 12:25:50 +00:00
Roy Wang 160c80a34c [Rust Frontend] Raise frontend JSON body limit (#46582)
Signed-off-by: esmeetu <jasonailu87@gmail.com>
2026-06-24 12:15:31 +00:00
Martin Hickey f237e16b41 [KV Offload] Replace OffloadingHandler with OffloadingWorker (#45053)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
2026-06-24 14:44:24 +03:00
JartXandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 70749fdcca [Feature] Triton INT4 per-token-head KV cache quantization (#40835)
Signed-off-by: JartX <sagformas@epdcenter.es>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-24 10:21:25 +00:00
Wei ZhaoandYifan Qiao d20dbf921b [Mooncake] Only check and store new KV cache range (#46412)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
2026-06-24 03:10:50 -07:00
Xinyu ChenandKunshang Ji ede54b926e set AttentionCGSupport.UNIFORM_BATCH for fa2 on xpu (#46555)
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-06-24 18:05:02 +08:00
LynnandClaude Opus 4.8 52fbe12283 [Perf][Multimodal] Avoid building a full timestamps list in video frame sampling (#46543)
Signed-off-by: Lynn <lynnhe02@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 09:38:27 +00:00
Dakai An dc0d318177 [Attention] Add FLASH_ATTN_MLA_SPARSE backend for Hopper sparse MLA (#46189)
Signed-off-by: Dakai An <dakaian108@gmail.com>
2026-06-24 09:33:10 +00:00
soaringk d7c1821b5a [Model][MiniMax-M3] Add pipeline parallelism support (#45810)
Signed-off-by: soaringk <k3vin.zhang@gmail.com>
2026-06-24 08:23:03 +00:00
Tiezhen WANGandClaude 4cd1a84c88 [Model] Remove BaiChuanForCausalLM and BaichuanForCausalLM (#46362)
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-06-24 16:13:57 +08:00
Mohammad Miadh Angkad 191826ec61 [CI/Build] Fix topk histogram build on SM75 (#46550)
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
2026-06-24 00:51:11 -07:00
Andreas Karatzas 549c7074cd [ROCm][CI] Skip the MoE Marlin tile-padding helper assertion (#46580)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-06-24 07:31:33 +00:00
hurukawaOpenAI CodexRoger Wangmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
489abadfb8 feat: support to OpenMOSS-Team (#44124)
Signed-off-by: nagisa-kun <1434936049@qq.com>
Signed-off-by: nagisa19 <1434936049@qq.com>
Signed-off-by: nagisa <1434936049@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-24 00:08:13 -07:00
Woosuk Kwon 96de8bb389 [MoE] Free unused MXFP4 scales in OAI Triton Backend (#46549)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-06-24 00:06:41 -07:00
Jee Jee Li 9d6fdc2901 [Kernel] GLM5 Router GEMM (#46385)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-06-23 22:54:50 -07:00
Benjamin Chislett 4c5bc41ba6 [Bugfix][Spec Decode] Fix probabilistic sampling for parallel drafting (#45956)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
2026-06-24 05:36:23 +00:00
Michał Ganczarenko ac1fa74616 [Bugfix] Fix NemotronLayerNorm1P hardcoded cuda device type (#46495)
Signed-off-by: <Michal Ganczarenko> <michal.ganczarenko@intel.com>
2026-06-24 13:21:02 +08:00
Sting Lin 556bc4e3a0 Upgrade tpu-inference to v0.23.0 (#46568) 2026-06-23 21:15:14 -07:00
Wei Zhao 05a0caba91 [Mooncake] Optimize lookup pool key string construction (#46188)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
2026-06-24 11:51:53 +08:00
Nick Hill 7ee4d22009 [Spec Decode] Reject placeholder (-1) draft tokens in rejection sampler (#46533)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-06-24 03:32:32 +00:00
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> ce9f64020b [Rust Frontend] Pass effective reasoning_parser_kwargs for structured output (#46360)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-06-24 03:13:44 +00:00
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 4ed8eaafb0 [Rust Frontend] Integrate xgrammar-structural-tag for strict and required tool calling (#46057)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-06-24 10:46:49 +08:00
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 6af0559ddb [Core][DP] Throttle prefills based on local prefill work (#46532)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-24 02:27:12 +00:00