MOVZX
49d708e158
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-18 18:09:11 +00:00
MOVZX and Hinata
6cf86cd95f
[Bugfix] Load checkpoint KV-cache scales in the Qwen3.5 family
...
Checkpoints that ship calibrated k_scale/v_scale tensors (e.g.
unsloth/Qwen3.8-27B-NVFP4, which carries one pair per full-attention
layer) name them ...self_attn.{k,v}_scale, but the runtime parameters
live on the inner attention layer (...self_attn.attn.{k,v}_scale).
Qwen3_5Model.load_weights hands the iterable straight to
AutoWeightsLoader, which performs no scale-name remapping, so the
scales are silently dropped and --kv-cache-dtype fp8 runs at scale 1.0
with only a warning_once that fires even on success (the MTP draft
layer legitimately has no scales).
Route scale-suffixed names through maybe_remap_kv_scale_name first,
mirroring gemma4/gpt_oss. No behavior change for checkpoints without
scales.
Measured on RTX 5090 / Qwen3.8-27B NVFP4 / fp8 KV / MTP k=3 (GSM8K
greedy, n=300): 92.3% at scale 1.0 vs 93.7% with scales loaded
(bf16-KV control 94.5%).
Fixes #54623
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-18 17:18:47 +00:00
alp and mottopanikeiku
ff01a6b14f
[Bugfix][Frontend] Stop beam search on generation config EOS tokens ( #62072 )
...
pre-commit / pre-run-check (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
2026-10-18 16:49:42 +00:00
aoshen02 and Claude Opus 5.5
fe8f2140ee
[Doc] Document /inference/v1/generate request/response fields and DP rank header ( #62305 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 15:26:36 +00:00
e80b6c642f
[ROCm][DSv4] Overlap the unfused shared expert with AITER MegaMoE ( #61819 )
...
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-18 22:39:38 +08:00
aoshen02 and Claude Opus 5.5
371745ebd6
[Doc][RL] Document routed experts return for MoE routing replay (R3) ( #62302 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 21:11:45 +08:00
072c0886d5
[Build] Keep plain CUDA arch when its suffixed variant is also requested ( #56549 )
...
Signed-off-by: Andre Lustosa <alustosa@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-18 20:13:17 +08:00
6b24947628
[CI/Build] Build DeepEP for SM103 (B300/GB300) in the Docker image ( #61829 )
...
Signed-off-by: Muhammad Ibrahim <muhammad.ibrahim@multiversecomputing.com >
Co-authored-by: Muhammad Ibrahim <muhammad.ibrahim@multiversecomputing.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-18 20:12:34 +08:00
Harry Huang and Thien Tran
494663940b
[Qwen4Exp] Clean up HC and PLE implementations for Qwen3.8-Flash-Next ( #61454 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-18 19:38:21 +08:00
Rémi Delhaye
3c13653738
[Frontend] Score with the chat template declared by Sentence Transformers CrossEncoders ( #60877 )
...
Signed-off-by: rdlhio <contact@rdlh.io >
2026-10-18 11:30:12 +00:00
Kevin H. Luu and Claude Opus 5.5
953fa442fa
[CI] Per-test timeout for Language Models (Extended Pooling) ( #62279 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 04:00:16 -07:00
Isotr0py
3f1b04e045
[Bugfix] Fix Opencv video backend frames count missing for WebM video inputs ( #61759 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-10-18 10:37:39 +00:00
aoshen02 and Claude Opus 5.5
2365177ddd
[Doc][RL] Document the weight-transfer HTTP protocol for trainers without vLLM ( #62267 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 09:26:56 +00:00
MOVZX
c5ab42bc0f
Merge branch 'main' into patches
2026-10-18 16:04:36 +07:00
Louie Tsai
1b2a666391
[Doc][Recipe] Fix for redhat fp8 xeon support issue ( #62245 )
2026-10-18 16:46:56 +08:00
83c9a547bf
[CI][Release] Fix release wheel builds: stop excluding tracked recipes/ from Docker context ( #62221 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-18 00:48:00 -07:00
b510857d37
[Bugfix][Build] Build fused GDN MTP decode for SM80-SM90 with CUDA < 13 ( #60896 )
...
Signed-off-by: kimseunghyun <incongnito12@gmail.com >
Signed-off-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com >
2026-10-18 15:14:06 +08:00
7c9042c05a
[Bugfix][MRV2] Fix speculative warmup deadlock with pipeline parallelism ( #62129 )
...
Signed-off-by: chenjr97 <chenjr97@mail2.sysu.edu.cn >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 23:51:32 -07:00
da9134bc89
[ROCm][CI] Give the MoE parallel-launch watchdog room for AITER JIT builds ( #62161 )
...
Signed-off-by: Andreas Karatzas <AndreasKaratzas@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <AndreasKaratzas@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-17 21:49:27 -05:00
b2456c622b
[ROCm][CI] Make ROCm 10.0 the default image; keep ROCm 7.2 as -rocm72 ( #58761 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-17 21:43:59 -05:00
03ee2e134d
[CI][ROCm] Fix order-dependent test_rocm_aiter_sampler_defers_import (CPU tensors hit Triton) ( #62213 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-18 01:10:08 +00:00
djramic and Andreas Karatzas
9c778e9ff8
[Bugfix][EPLB] Wait for pending expert buffer copies before NIXL READs ( #62081 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-17 18:18:02 -05:00
7879f3687b
[NIXL] Drop unreachable producer-side notif counting in push mode ( #62210 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 21:24:35 +00:00
Kevin H. Luu and Claude Opus 5.5
255f14599e
[Bugfix][Rust Frontend] Fix macOS build: read inherited listener family from its address ( #62177 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 11:56:28 -07:00
03e8300d2c
[Bugfix] Enable inductor batch-invariant mode under VLLM_BATCH_INVARIANT ( #61118 )
...
Signed-off-by: Yifan Feng <evanfengff@gmail.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-17 14:53:06 -04:00
Afonso Ribeiro and Flora Feng
5c1c903501
[Perf][Parser] Stream Qwen3 tool argument values incrementally (12 KB call: 2.1 s to 11 ms) ( #62027 )
...
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-17 18:49:06 +00:00
Tyrone
bafb263664
[Bugfix][Benchmarks] Name the real cause when the /tokenize alignment probe fails ( #59888 )
2026-10-17 14:47:52 -04:00
Nick Hill and Claude Opus 5.5
0c5dd18e7a
[MRV2] Collect KV transfer results before the drafter runs ( #62041 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-17 10:27:57 -07:00
763e3f7c91
[Bugfix][Spec Decode] Disable dynamic speculative decoding for fixed-K proposers instead of crashing ( #46399 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-17 12:41:04 -04:00
Alec and Claude Opus 5.5
e744eacf68
[Rust Frontend] Validate bad_words limits during request lowering ( #61933 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 09:10:40 -07:00
c0b6f74bee
[Benchmark] Add per-case kernel profiles to vllm bench forward ( #61692 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 08:27:11 -07:00
siyu and Xun Sun
c61a1e73c6
[Feature] Skip MLA weight absorption when weights are pre-processed ( #57320 )
...
Co-authored-by: Xun Sun <UNIDY2002@outlook.com >
2026-10-17 23:10:04 +08:00
aoshen02 and Claude Opus 5.5
237d99b4af
Revert "[Docs] Add train-inference consistency agent skill" ( #61551 ) ( #62186 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 13:54:30 +00:00
aoshen02 and Claude Opus 5.5
2d8efee9a7
[Docs] Add train-inference consistency agent skill ( #61551 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 20:49:50 +08:00
Roy Wang and Claude Opus 5.5
21e09ea959
[Doc] Sync DeepSeek-V4.1-Flash recipe with vllm-project/recipes#1136, #1137 ( #62179 )
...
Signed-off-by: esmeetu <jasonailu87@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 12:27:52 +00:00
837e87e1c5
[Feature][LoRA][MoE] Expert LoRA for FlashInfer TRTLLM MXFP4 experts ( #61979 )
...
Signed-off-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 20:17:56 +08:00
liuzhenwei and Kunshang Ji
4e964b2623
[XPU] Fix Event init failure w/ external ( #61743 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-17 19:10:39 +08:00
Vadim Gimpelson and Claude
ec8b20fbf0
[Bugfix] Fix Gemma 4 encoder CUDA graph capture OOM at startup ( #61365 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-17 11:04:49 +00:00
2b85d9cb65
[Perf][Bugfix] Prefer MNNVL for single-node FlashInfer allreduce again ( #61989 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-17 02:55:28 -07:00
026f6ab268
[Model] DeepSeek-V4/V4.1 VL: stream language-model weights instead of sorting the checkpoint ( #58857 )
...
Signed-off-by: git bisector <gitbisector@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-17 09:37:56 +00:00
Roy Wang and Claude Opus 5.5
b515648641
[Doc] Host model recipes in recipes/ ( #61302 )
...
Signed-off-by: esmeetu <jasonailu87@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 09:14:36 +00:00
f98ae6a9fd
[Bugfix][Core] Keep resumable sessions that stop while preempted ( #58778 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Yifan Zong <yzong@redhat.com >
2026-10-17 09:08:14 +00:00
2067297b2f
[ROCm][Perf] Use AITER tuned configs to select a4w4 gemm kernel instead of env var ( #61278 )
...
Signed-off-by: Andy Friedrich <afriedri@amd.com >
Signed-off-by: afriedri <afriedri@amd.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-17 04:00:28 -05:00
Aarushi Jain and Andreas Karatzas
3bf25c5097
[CI][Bugfix] Ignore AMD CI webhook no-op builds in /amd-ci run and retry ( #62087 )
...
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-17 03:54:31 -05:00
Jee Jee Li
e56b1603af
[Kernel]Add norm fused kernel for Step-5 Preview and Muse Spark Open ( #62132 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-17 01:53:00 -07:00
Ting SUN
eb8de8d1a1
[Bugfix][OpenAI] Respect allowed_token_ids in beam search ( #47630 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-10-17 08:50:40 +00:00
dskwe
b05ede5616
[Bugfix][Frontend] Preserve token IDs in completion derender responses ( #62114 )
...
Signed-off-by: Yuan Li <dskwelmcy@163.com >
2026-10-17 08:27:48 +00:00
61bb590877
[Bugfix][Spec Decode] Clear stale shared block-table rows ( #61174 )
...
Signed-off-by: Cheng Rui <286040359@qq.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com >
2026-10-17 01:25:02 -07:00
c3de0b37e2
[Core][Spec Decode] Replace EAGLE group heuristics with explicit draft-layer tags ( #61455 )
...
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 08:22:33 +00:00
MOVZX
e0fe5df731
Merge branch 'main' into patches
2026-10-17 08:06:56 +00:00
08a65e287e
[Bugfix][Kernel] hadacore: don't load past the input on a partial last warp ( #61944 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 00:52:17 -07:00
znzryb and Woosuk Kwon
ad2688ab30
[Bugfix] Validate draft-token capacity in forward benchmark ( #61809 )
...
Signed-off-by: znzryb <znzryb34@gmail.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
2026-10-17 00:51:36 -07:00
Jovan and OpenAI Codex
5be4266f5c
[Bugfix][Frontend] Preserve Cohere and Anthropic streaming errors ( #61947 )
...
Signed-off-by: Kevin <jia__kuan@163.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-17 06:10:16 +00:00
Wei Zhao
90d6d871d6
[Attention] Add CuTe DSL backend selection for MLA context prefill ( #62069 )
...
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-10-17 01:21:55 -04:00
Cy Cheng and Cy
d74f4180cd
[Bugfix][Frontend] Detokenize echoed prompts after truncation ( #61876 )
...
Signed-off-by: Cy <15118312+cycancode@users.noreply.github.com >
Co-authored-by: Cy <15118312+cycancode@users.noreply.github.com >
2026-10-17 05:20:38 +00:00
Jee Jee Li
39983a5e7a
[Kernel] Add FP32 router GEMM support for Step-5 Preview and Muse Spark Open ( #62126 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-16 22:17:55 -07:00
41b8c15809
[Feature][MoRIIO] Support hybrid WRITE transfers with DSpark ( #60544 )
...
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com >
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-17 04:57:58 +00:00
Micah Williamson and Cursor
a0e7137c4b
[CI][ROCm][Bugfix] DeepSeek V4.1: don't pass expert intermediate padding to AITER ( #61960 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-17 04:25:39 +00:00
7b6310c109
[Perf][Sampler] Speed up the small-batch split top-p pipeline ( #61996 )
...
Signed-off-by: Bowen Wang <bowen@inferact.ai >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 21:22:11 -07:00
Fadi Arafeh and vllm-agent
556ac15eb9
[Bugfix] Remove extra args from CPU's compute slot mapping kernel ( #61992 )
...
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 12:11:33 +08:00
Gabriel Wu and Claude Opus 5.5
dcd07967ca
[CPU] Keep the hybrid block size across the engine core's config rerun ( #61805 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 12:06:03 +08:00
ce49c47cbb
[Bugfix][Sampler] Carry the boundary tie count across blocks in the split top-p mask kernel ( #62046 )
...
Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com >
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com >
Co-authored-by: Bowen Wang <bowen@inferact.ai >
2026-10-17 02:26:57 +00:00
Sherif Waly and vllm-agent
ce8853d27a
[Bugfix][KV Offload] Allow explicit SimpleCPUOffload block counts ( #61674 )
...
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 02:11:12 +00:00
71c20f73f2
[FlashInfer] Migrate autotune warmup to flashinfer.autotune_v2 managed cache ( #55671 )
...
Signed-off-by: Yang Xu <yanxu@nvidia.com >
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Signed-off-by: YangXu <yanxu@nvidia.com >
Co-authored-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 02:08:28 +00:00
Shuolei Wang and aoshen02
b3a7e2ff6e
[RL][Frontend] Return weight-version spans on OpenAI completions and chat ( #61347 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-17 02:08:07 +00:00
6b6a6c5c98
[RL] Support per-parameter source layouts for NCCL M2N ( #60934 )
...
Signed-off-by: pkousha <pkousha@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-16 21:41:17 -04:00
Harry Mellor and Claude Opus 5.5
f8112704d5
[Chore] Use upstream Transformers InklingProcessor ( #61883 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 01:04:54 +00:00
f019e3e105
[ROCm] attn res + sigmoid_mul + conv fusions ( #52968 )
...
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com >
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-17 00:54:54 +00:00
Vadim Gimpelson and Claude
783380af39
[Bugfix] Fix NaN output of the Mamba2 chunk scan kernel on sm90 ( #61718 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-16 17:24:51 -07:00
Wei Zhao and Codex
be25977532
[CI][Tests] Account for FP8 KV-cache rounding in MLA reference ( #62067 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-16 17:10:08 -07:00
523d74e2ac
[Bugfix][LoRA] Fix garbage/NaN lora_expand output on SM90 with fp32 i… ( #61906 )
...
Signed-off-by: BeichenHuang <polarishuang0.0@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-16 19:42:30 -04:00
a94c315711
[DP] Engines own wave wake-ups; drop the front-end FIRST_REQ ( #59559 )
...
Signed-off-by: fh3 <a@b>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 19:41:54 -04:00
b544587408
[Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning ( #53558 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-16 23:16:47 +00:00
alp and mottopanikeiku
2268d83bd9
[Bugfix][Multimodal] Honor RGB PNG transparency when loading images ( #61986 )
...
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
2026-10-16 23:09:12 +00:00
28e92eeff2
[ROCm][Perf][Kimi-K3] Use AITER Gluon chunk_kda for KDA prefill ( #59552 )
...
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com >
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 22:58:56 +00:00
Afonso Ribeiro and Flora Feng
b088eba944
[Perf][Parser] Stream MiniMax M2 tool argument values incrementally (16 KB call: 5.9 s to 26 ms) ( #62035 )
...
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-16 22:46:02 +00:00
596e530675
[Kernel] Add router GEMM kernels for (5120, 256) and BF16 output to ll_bf16_gemm ( #62042 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Lain <fusiyuan2000@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 15:39:46 -07:00
153a17166e
[Frontend] Add the Beam reasoning and tool parsers ( #62052 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com >
Co-authored-by: neilkg <33635204+neilkg@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 14:08:29 -07:00
Dan Blanaru and Claude Opus 5.5
9731f30a03
[Bugfix] Keep VLLM_RAY_BUNDLE_INDICES out of the compile cache key ( #61211 )
...
Signed-off-by: DanBlanaru <48605845+DanBlanaru@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 20:59:48 +00:00
68ebea789a
[Perf][MoE] Skip padded work in SM100 FP4 DeepGEMM experts ( #59044 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-16 16:20:52 -04:00
cc2e4fecf2
[ROCm] [fusion] support ROCm aiter fusion of per tensor quant + rmsnorm ( #52831 )
...
Signed-off-by: Andy Friedrich <afriedri@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 15:19:07 -05:00
9eb7fb9157
[Build] Remove unused tools/install_deepgemm.sh ( #61265 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 15:57:49 -04:00
cjackal
663e535640
[Perf][Spec Decode] Skip bonus-logit reduction for short drafts in rejection sampling ( #61451 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-10-16 12:42:48 -07:00
Oxana Korzh and Claude Opus 5.5
5bf3d9f665
[ROCm][Kimi-K3] Support DSpark adaptive verification on MI355X ( #59824 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-16 13:53:49 -05:00
djramic
307c5844ad
[ROCm][CI] Enable batch invariance for the ngram scheduling accuracy test ( #62048 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
2026-10-16 18:36:14 +00:00
Guan-Ming Chiu and vllm-agent
3dc6def360
[Attention][Spec Decode] Enable varlen Mamba2 decode for adaptive verification ( #53036 )
...
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 11:23:59 -07:00
djramic
e7a5bdb454
[ROCm][CI] Move Spec Decode Speculators + MTP to MI355 ( #62053 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
2026-10-16 18:22:38 +00:00
alberto
ae8f41238c
[Bugfix][Parser] Honor a prompt-closed reasoning block in non-streaming parse and reasoning token counts ( #61857 )
...
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com >
2026-10-16 12:54:42 -04:00
cffc0cfa31
[CI] Fix B200 FusedMoE Layer Kernels hang on NVSHMEM IB setup ( #61695 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 12:00:12 -04:00
Andreas Karatzas and Claude
706791d516
[CI][AMD] Move 17 test groups from MI300 to MI355 and test large DeepSeek/Nemotron evals on AITER ( #61955 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-16 10:58:59 -05:00
Sawyer Bowerman and Codex
76081b9ee4
[Bugfix][Model] Honor quantization config in Inkling MTP ( #61921 )
...
Signed-off-by: Sawyer Bowerman <sbowerma@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-16 15:49:26 +00:00
bnellnm
1a5b187066
Add @bnellnm to CODEOWNERS ( #62025 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-10-16 11:16:03 -04:00
Wentao Ye
8e25bbca19
[Refactor] Remove dead code multiple places ( #61843 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-16 15:01:07 +00:00
8a6c0449b3
[Bugfix][ROCm][GLM-5.2] Stop forcing skip_final_all_reduce on AMD DSA model path ( #58589 )
...
Signed-off-by: avininjamay8 <Avinash.Paul@amd.com >
Signed-off-by: avininjamay8 <avpaul@amd.com >
Co-authored-by: avininjamay8 <avpaul@amd.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 21:38:55 +08:00
Ayushman Singh
de7eeaa87b
[Perf] Share Triton JIT warmup compilation pool across registered kernels ( #61422 )
2026-10-16 08:59:42 -04:00
MOVZX
be95edb7f6
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-16 17:35:42 +07:00
Martin Hickey
64175d3827
[Bugfix][Frontend] Keep the parser's tool call IDs in batch derender ( #61852 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-16 10:27:17 +00:00
alp and mottopanikeiku
4ab99ff136
[Bugfix][Frontend] Honor the constructor chat template in offline scoring ( #61990 )
...
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
2026-10-16 10:20:46 +00:00
024c5188bf
[Frontend] Resolve logprobs on the parsed streaming derender path ( #60437 )
...
Signed-off-by: Shimi Bandiel <shimib@google.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 18:04:15 +08:00
Hexiang Wang
9a16d43ef0
[Bugfix][Mooncake] Fix MultiConnector stats and proxy SSE responses ( #61218 )
...
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com >
2026-10-16 17:54:48 +08:00
MOVZX and Hinata
8dc54f35dd
[Offloader] Stream every multimodal tower parameter to host RAM
...
Qwen3_VisionTransformer derived its compute device from
patch_embed.proj.weight and fed pos_embed.weight straight into the
position-embedding interpolation kernel, so the last two anchor weights
had to stay in VRAM for a tower to be streamable at all.
Record the device the tower was built on before the offloader moves
anything, and fall back to it when no weight is on the accelerator.
Move the interpolation table to that device per encode, which costs one
8.4 MiB host-to-device copy for a 456 MiB tower. Towers now leave no
parameters in VRAM between encodes; only small buffers such as the fused
input norm and rotary inv_freq remain.
Also let a tower restrict streaming through tower_offload_children, and
warn when a tower resolves its device property to another device type,
so a tower that reads an anchor weight reports the cause instead of a
Triton pointer error.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-16 16:07:53 +07:00
zofia
07f3857fb9
[XPU][Linear][MXFP4] enable torch as mxfp4 linear backend on xpu ( #51965 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Signed-off-by: zofia <110436990+zufangzhu@users.noreply.github.com >
2026-10-16 17:07:05 +08:00
MOVZX and Hinata
18c06ed406
[Offloader] Stream multimodal towers from host RAM with bulk-copy
...
Add --offload-towers: place multimodal tower weights in pinned host
memory and bulk-copy each unit to the GPU only during its forward,
keeping towers out of VRAM between encodes. Unlike UVA zero-copy,
whole-tensor H2D transfers avoid re-reading operand tiles over PCIe
inside the GEMM inner loop. Tower streaming is independent of layer
group scheduling, so group_size may be 0 for tower-only offload.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-16 14:32:14 +07:00
fa75d3456d
[Feature][PCP] Add DeepSeek-V4.1-Flash support ( #59857 )
...
Signed-off-by: wangyicong <wangyicong@bytedance.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 23:17:51 -07:00
Chaojun Zhang and vllm-agent
074c83a09a
[CI][Intel] Skip ROCm-only test_topk_gating_enabled_by_cdna on XPU ( #61932 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 13:44:32 +08:00
Wei Zhao
e365792997
[Dependency] Upgrade TokenSpeed MLA to 2.0.3 ( #61689 )
...
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-15 22:28:59 -07:00
dskwe
49cb2bbf71
[Bugfix] Honor code_revision in config-based tokenizer loading ( #61611 )
...
Signed-off-by: Yuan Li <dskwelmcy@163.com >
2026-10-16 05:21:12 +00:00
4ac1f8d2ac
[Bugfix][Structured Output] Mask grammar-invalid implicit reasoning ends ( #61574 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-16 12:53:41 +08:00
maithilijoshi20 and Cyrus Leung
92dc3dd393
[Bugfix] Honor OPT token embedding width for prompt embeddings ( #61736 )
...
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-16 04:41:50 +00:00
Andreas Karatzas and Claude
b466fbacd2
[ROCm][Perf] Any-shape low-latency MoE router GEMM on MI355 ( #61895 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-15 23:22:31 -05:00
642c18d64c
[Perf] Kimi K3 Specialized Decode: CuTeDSL Optimized GB300 (1.5-2x) ( #61867 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-15 22:50:41 -04:00
3ddc6702a8
[XPU][Docker] Bump nixl to 1.5.0 ( #60356 )
...
Signed-off-by: Sandeep Maddipatla <sandeep.maddipatla@intel.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 10:35:29 +08:00
Shuolei Wang
05ad0a8ace
[Perf][DSv4.1] Split fused_wqa_wkv by token across TP ranks on SM90 ( #58872 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
2026-10-16 02:31:34 +00:00
dd9e07a25f
[Feature] Add External LB Mode Support for Elastic EP Scaling ( #43202 )
...
Signed-off-by: lqh-hh <liuqihang6@huawei.com >
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: lqh-hh <liuqihang6@huawei.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: 2017403603 <705328646@qq.com >
Co-authored-by: Julien Mancuso <jmancuso@nvidia.com >
Co-authored-by: Itay Alroy <75032521+itayalroy@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-15 21:29:02 -04:00
Monishver and Kevin H. Luu
fb3fec970e
[Bugfix][Kernel] silu_and_mul_quant: run the vectorized loop ( #61716 )
...
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 18:16:53 -07:00
Andreas Karatzas and Claude
29345f42a6
[ROCm][Qwen4Exp] Support HC and MoE sequence parallelism on ROCm ( #61471 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-15 20:11:17 -05:00
Kevin H. Luu and Claude Opus 5.5
a956a8126e
[Bugfix][MoE] Keep the gate's hash_indices_table out of RoutedExperts parameters ( #61930 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 18:00:17 -07:00
Jee Jee Li
bee95149e2
Update DeepGEMM commit ( #61744 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-15 17:54:40 -07:00
Tessa Pham and Benjamin Chislett
8f93c64866
[Bugfix] remove duplicate hash-routing capability method ( #61924 )
...
Signed-off-by: Tessa Pham <tepham@redhat.com >
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com >
2026-10-15 17:40:23 -07:00
a8be1500ba
[Bugfix][MoE] Restore TP-sharded routed-expert capture for DP=1 ( #61388 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Signed-off-by: Ao Shen <281812371+aoshen02@users.noreply.github.com >
Signed-off-by: Ao Shen <aoshen@inferact.ai >
Co-authored-by: Ao Shen <281812371+aoshen02@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Ao Shen <aoshen@inferact.ai >
2026-10-16 00:21:24 +00:00
Jack Hu and Douglas Lehr
ef38e8d006
[Model][Hardware][AMD][Kernel] Part 2/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_ATTN ( #42755 )
...
Signed-off-by: Jack Hu <Jack.Hu@amd.com >
Signed-off-by: Jack Hu <jingxujaster@hotmail.com >
Signed-off-by: Jack Hu <jack.hu@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-15 17:10:14 -07:00
Anerudhan Gopal and OpenAI Codex
af28c171f5
[Dependency] Upgrade FlashInfer to 0.7.2 and restore KDA auto backend ( #61913 )
...
Signed-off-by: Anerudhan Gopal <agopal@nvidia.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-15 17:04:24 -07:00
Juntian Liu and Claude Opus 5.5
19b0efce30
[Perf][Core] Skip the null head of sliding-window block tables ( #61724 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 16:14:51 -07:00
Woosuk Kwon and Claude Opus 5.5
4a89316efa
[Core] Drop the Ray driver-node GPU pin left over from the legacy executor ( #61713 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 16:00:18 -07:00
3f1e57225a
[ROCm][Perf][Kimi-K3] Use AITER Gluon KDA kernels for decode and spec decode ( #57392 )
...
Signed-off-by: omuhamma <omar.muhammad@amd.com >
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
2026-10-15 17:54:24 -05:00
de7d70676d
[Perf] Low-latency CPU all-reduce for DP metadata syncs ( #61767 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: hjjq <50634613+hjjq@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-15 15:29:13 -07:00
Stefano Castagnetta and Kevin H. Luu
784d1fff0c
[MRV2][Spec Decode] Quantized draft lm_head for drafters that share the target head ( #59973 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 22:06:40 +00:00
d92a4f25f2
[watermarking] Add DFlash support to dual-key gumbel-max watermarking ( #59554 )
...
Signed-off-by: Shern Shiou Tan <shernshiou@gmail.com >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Raphael Rialland <raphael.rialland@mistral.ai >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-15 22:06:06 +00:00
e711723352
[ROCm] gfx1250 optimizations and model enablement ( #58800 )
...
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com >
Signed-off-by: Juan Villamizar <jpvillam@amd.com >
Signed-off-by: jpvillam <jpvillam@amd.com >
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com >
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com >
Co-authored-by: jpvillam <jpvillam@amd.com >
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com >
Co-authored-by: Daniel <danichan@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com >
2026-10-15 14:51:57 -07:00
Thang Nguyen and Claude Opus 5.5
9b99ccb56b
[CI] Shard the AMD mirrors of the DP, RayExecutorV2 and batch invariance steps automatically ( #61861 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 16:48:19 -05:00
DimensionSTP and Kevin H. Luu
9edcc7fa7e
[Bugfix][Spec Decode] Avoid reusing random draws across block verification rounds ( #61161 )
...
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 14:46:38 -07:00
Mikko Tukiainen
0c25849292
[ROCm][Perf][Bugfix] Probe the preshuffled FP4 GEMM config by logical K ( #59406 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
2026-10-15 21:11:33 +00:00
f8460ef236
[Kernel][DSv4.1] Bump FlashMLA to the 2026.09.30 release and its interleaved V4.1 KV cache ( #61371 )
...
Signed-off-by: juntianl <juntianl@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-15 13:58:25 -07:00
33212db2c2
[KV Connector][NIXL] Reject NVFP4 KV cache with heterogeneous TP head split ( #61865 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 13:57:37 -07:00
Thang Nguyen and Claude Opus 5.5
c414bf0295
[CI] Shard four AMD mirrors automatically ( #61817 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 13:57:35 -07:00
Wentao Ye
54d9b1cb48
[GLM5.3 Perf] Inplace update draft decode metadata, 1.7% E2E Throughput Improvement ( #61873 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-15 20:20:12 +00:00
SeongJun Lee and Wei Zhao
ee8c47961b
[CI] Use an ungated model as the attention test config default ( #61732 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-15 20:19:09 +00:00
21f4b6dc84
[KV Offload] Split the CPU offload region across shm files ( #59546 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 20:17:40 +00:00
Thang Nguyen and Claude Opus 5.5
16532876e7
[CI] Shard four more AMD mirrors automatically ( #61860 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 15:13:51 -05:00
d12ea8e309
[Bugfix][MRV2] Keep input_ids on later PP stages for raw-token models ( #61016 )
...
Signed-off-by: booth-algo <kevinlauofficial01@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 12:54:21 -07:00
d486219fc2
[Bugfix][Core] Uncount stop-trimmed spec tokens from num_computed_tokens ( #58668 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-15 12:50:06 -07:00
49f944be3e
[Perf][HiSparse] Reuse pinned host pool after graph profiling ( #61668 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 12:33:03 -07:00
Rohan Potdar and Claude Opus 5.5
105605b8e9
[ROCm] Support DeepSeek-V4 hash routing in AITER Triton W4A16 MoE ( #60282 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 14:10:12 -05:00
Thien Tran
82b9ccaa79
[Perf][MoE] Skip the TRTLLM fp8-block activation-scale transpose copy ( #61773 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-15 12:09:07 -07:00
4d58fa254a
[MoE] Unify DeepGEMM MegaMoE experts behind one feature-matrix layer; add NVFP4 ( #61538 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 19:02:01 +00:00
Misha Goin and Claude Opus 5.5
4d9a8d6a41
[Bugfix] Make Humming DeviceInfo lookups traceable under torch.compile ( #61858 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 11:12:21 -07:00
Mikko Tukiainen and Cursor
47fd76e4ce
[ROCm][Perf] Route softmax MoE top-k, including fused shared-expert sigmoid, through AITER topk_gating ( #56226 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-15 17:53:18 +00:00
4f49b55c1f
[MoE] Shape-based padding for FlashInfer TRT-LLM MXFP8 experts (gated and non-gated) ( #61293 )
...
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-15 13:44:11 -04:00
7cd7982dfd
[Spec Decode] Don't reserve DFlash/DSpark draft slots in the scheduler token budget ( #59468 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-16 01:23:58 +08:00
bzsuni and Flora Feng
338dab077c
[Bugfix] Fix Apertus streaming scientific notation ( #61812 )
...
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-15 16:49:57 +00:00
957d512e96
[Bugfix][Frontend] Preserve streaming chunk boundaries in RequestOutputCollector ( #58941 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Co-authored-by: Benedikt Falk <bfalk@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-15 16:38:28 +00:00
Jovan
a9e90b9d1d
[Bugfix][Frontend] Reject Cohere REQUIRED tool choice without tools ( #61825 )
...
Signed-off-by: Kevin <jia__kuan@163.com >
2026-10-15 16:37:08 +00:00
d6f0b7a416
[CI] Skip exited descendants when finding the worker in the FT e2e test ( #61648 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-15 16:08:14 +00:00
ed3050add4
[Core] Apply prefill_schedule_interval outside data parallelism ( #54627 )
...
Signed-off-by: lifulu <fululi12@amd.com >
Signed-off-by: lifulu <lifulu@users.noreply.github.com >
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
Co-authored-by: lifulu <lifulu@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-15 11:03:13 -05:00
c825e1d235
[Perf] Fuse the mamba align block-table gather into one Triton kernel, 7 kernels -> 1, 6x speedup on all shapes for this kernel ( #58737 )
...
Signed-off-by: lifulu <fululi12@amd.com >
Signed-off-by: lifulu <lifulu@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: lifulu <lifulu@users.noreply.github.com >
2026-10-15 11:02:53 -05:00
MOVZX and Hinata
5d6754ff6b
fix: keep the refusal-direction projection dynamo-safe
...
With VLLM_ABLITERATION set, boot crashed: dynamo traces
abliteration.register from Qwen3NextGatedDeltaNet.__init__, and
buf.copy_(r) fails on the meta device during the compile-time model
construction pass. Disable tracing for register and guard the copy so
an unmaterializable buffer turns projection off with a warning.
Log activation at registration instead of inside project_: project_
runs in the compiled region, and a logger call there aborts dynamo.
The width guard uses the same escape hatch.
Also project the embedding that Qwen3NextMultiTokenPredictor builds
for itself, which the first pass missed, and add tests for the
2560-value Flash-Next direction against a 5120 model and for a
half-width activation.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-15 22:56:06 +07:00
CharlesWu and Yichao Zhu
a97d19d792
[UMBP][1/10] Add shared connector, runtime contracts, and request lifecycle ( #61789 )
...
Signed-off-by: CharlesWu <Charles.Wu@amd.com >
Co-authored-by: Yichao Zhu <Yichao.Zhu@amd.com >
2026-10-15 15:12:39 +00:00
3dcfaeb4e8
[Frontend] Apply each endpoint's top_logprobs cut in derender ( #59518 )
...
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-15 15:04:04 +00:00
MOVZX and Hinata
0076adc78b
[Qwen3.5] Project the refusal direction out of the residual stream at runtime
...
orcarouter/Qwen3.8-27B-Uncensored removes refusal by editing weights. The
Swift release we serve is AutoRound W8A16, so re-editing weights means
re-quantizing. Project one unit direction r out of every residual-stream
writer output instead: x <- x - r (r . x), which equals the weight edit
W' = W - r(r^T W) on every writer feeding that stream (Arditi et al. 2024).
Sites: attention output for both the full-attention and GDN branches, the
MLP output, and token embeddings, in Qwen3NextDecoderLayer.forward and
Qwen3NextModel.embed_input_ids, which Qwen3_5DecoderLayer and Qwen3_5Model
inherit but whose base constructors they skip, so both register there too.
The MTP drafter reuses Qwen3_5DecoderLayer and calls its own
embed_input_ids, so the draft path is covered at the same width, and
_maybe_share_embeddings at PP=1 hands the draft the target embedding
module. Sequence parallelism shards dim 0 only, so the last dim is
hidden_size at every site.
VLLM_ABLITERATION names a JSON file with key refusal_direction, a
safetensors tensor, or a bare name resolved under
/opt/qwen38/abliteration then /data/abliteration. Empty or 0 is off.
register() always creates the buffer _abliteration_r, zeros when off, so
one compiled graph serves ON and OFF and toggling the switch
(declared in envs.py, hence in compile_factors) invalidates the compile
cache. project_() is an in-place rank-1 update with no full-size
temporary, and skips tensors whose width is not hidden_size.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-15 21:40:16 +07:00
Matthias Gehre
011fa68ca9
[ROCm][Perf] Pad fp16/bf16 GEMM weight row stride for L2 cache ( #55090 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
2026-10-15 14:26:32 +00:00
MOVZX
76df77c04e
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-15 21:16:39 +07:00
Ronald and aoshen02
d12d5460a5
[Bugfix][MoE] Keep routing rows during DeepSeek V4.1 decoder replay ( #61643 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-15 14:02:06 +00:00
24f632ca7e
[Bugfix] Engage indexer TP row sharding on total rows, not per-rank rows ( #61647 )
...
Signed-off-by: Zheng Cai <caizheng1993@gmail.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-15 13:54:14 +00:00
mosafariuk
69425846f8
[Bugfix][CPU] Report the real cause when the worker outgrows its memory reservation ( #59472 )
...
Signed-off-by: Mo <mosafariuk@gmail.com >
2026-10-15 21:52:05 +08:00
b7b20b1b8f
[Bugfix][Frontend] Leave the final stop token out of chat derender text ( #61757 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-15 21:51:34 +08:00
littlecircle0730 and Shanshan Shen
71536bd5f8
[MM][CG] Support ViT full CUDA graph for DeepSeek-VL2 ( #46005 )
...
Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com >
Signed-off-by: littlecircle0730 <43994952+littlecircle0730@users.noreply.github.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-15 21:38:52 +08:00
0951fc183c
[ROCm][DSv4.1] Disambiguate ROCm attention class names ( #58689 )
...
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 21:38:39 +08:00
1463501553
[Bugfix] Terminate streaming sessions on ABORT/ERROR and prevent zombie sessions ( #58942 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Benedikt Falk <bfalk@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-15 13:32:19 +00:00
Xunzhuo
766c837bd5
[Model] Support every Decision 3.0 size (d3-flash, d3-mini, d3-nano, d3-lite, d3-edge) ( #61778 )
...
Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com >
2026-10-15 13:29:24 +00:00
Laith Sakka and Nick Hill
9cede34a67
[Bugfix] Create cuBLAS handles before memory profiling to avoid OOM ( #61044 )
...
Signed-off-by: Laith Sakka <lsakka@meta.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-15 08:59:22 -04:00
Martin Hickey
252be28c18
[Frontend] Replace return_mm_kwargs with an mm_payload enum on render requests ( #61427 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-15 12:41:46 +00:00
Martin Hickey
c6b772cf6b
[Bugfix][Frontend] Emit kv/ec transfer params at the top level of render output ( #61241 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-15 12:34:15 +00:00
Micah Williamson
e8e1e96196
[ROCm][DSv4] AITER MegaMoE Fixed-Slot Dispatch For DeepSeek-V4 ( #61262 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
2026-10-15 19:10:59 +08:00
8b27d27402
[Bugfix][KV Offload] Always attempt NIXL agent removal during peer teardown ( #60279 )
...
Signed-off-by: nej1gotnochill <divyanshlalotra2@gmail.com >
Co-authored-by: Codebuff <noreply@codebuff.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 14:07:23 +03:00
Talor Abramovich
81e42c9987
[Feature][Frontend] Control the log level of request JSON body using env var ( #60630 )
...
Signed-off-by: talora <talora@nvidia.com >
2026-10-15 11:07:19 +00:00
jimmy-adams
365aaa77bd
[Bugfix][DSv4] Remap Quark MTP layer_quant_config onto draft prefixes ( #56054 )
...
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com >
2026-10-15 18:51:57 +08:00
e10c10a644
[Core] Add PROMOTION_LATENCY histogram metric for tiering offload ( #53910 )
...
Signed-off-by: Wen Zhou <wenzhou@redhat.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-15 13:47:41 +03:00
Pei Li and Kevin H. Luu
6fe7401fac
[Perf] Read VLLM_COMPUTE_NANS_IN_LOGITS once per IterationStats ( #61710 )
...
Signed-off-by: Pei Li <peili@nvidia.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 18:35:54 +08:00
Timofey and Codex
716c5225ae
[Model] Add AliceAI support ( #57963 )
...
Signed-off-by: timka <timka.byzov@yandex.ru >
Co-authored-by: Codex <noreply@openai.com >
2026-10-15 10:15:03 +00:00
7020600b1d
[KV Connector][NIXL] Register packed MLA KV rows as a single region ( #61074 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-10-15 09:18:15 +00:00
Flora Feng and Chauncey
80dd705c1d
[Frontend] Make finish_reason and Responses item status truthful ( #59324 )
...
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-15 10:12:02 +01:00
Denis Khachyan and Claude Code
427915bed2
[Build][CPU] Install Run:ai Model Streamer in vllm-openai-cpu image ( #61382 )
...
Signed-off-by: Denis Khachyan <khachyanda@gmail.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-10-15 09:11:51 +00:00
Kevin H. Luu and Claude Opus 5.5
7f730b4cea
[CI] Run 4xH100 steps nightly only ( #61782 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 01:32:43 -07:00
wang.yuqi
0f0bc4dbb1
[Test] Split the chat_utils tests. ( #61765 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-15 08:26:22 +00:00
b965ee564d
[Chore] Use upstream Transformers DeepseekV4Config ( #61392 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 01:08:08 -07:00
Misha Goin and Claude Opus 5.5
1c4c319792
[Frontend] Advertise --enable-x/--disable-x for boolean CLI flags ( #61479 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 01:08:02 -07:00
48f17b77bb
[Bugfix] Reject mismatched float16/bfloat16 KV cache dtype ( #60885 )
...
Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com >
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 08:06:59 +00:00
fc8e08a2a6
[Bugfix][KV Offload] Do not let mark_miss() flip a pending or in-flight async lookup probe ( #59096 )
...
Signed-off-by: Sohom Chakraborty <16609933+sohom-cs@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 10:57:46 +03:00
Bugen Zhao and Claude Opus 5.5
8fa5ae844a
[Frontend] Let an unset --tool-strict-level fall back to the tool parser's default ( #61328 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 15:48:38 +08:00
91ed2859c7
[CPU][Zen] Update zentorch to 2.14.0.1 & enable float16 on Venice and later ( #61411 )
...
Signed-off-by: Ganesh R <ganesh.r@amd.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-15 00:23:50 -07:00
85c94c1706
[Tests][XPU] Skip modelopt_fp4 online MoE re-quantization test on XPU ( #61723 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-15 00:23:45 -07:00
Kevin H. Luu and Claude Opus 5.5
8f111b22ee
[CI] Run 4xB200 steps nightly only ( #61541 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 00:08:48 -07:00
Kevin H. Luu and Claude Opus 5.5
32feecd91f
[CI] Pin DeepSeek-V4.1-Flash-NVFP4 eval to its FP8-Engram revision ( #61763 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 00:05:18 -07:00
Chaojun Zhang and Kevin H. Luu
ab593678b5
[CI][XPU] Ignore test_reload.py on Intel CI ( #61740 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 15:03:34 +08:00
a9fe30e9f6
[Bugfix][Multimodal] Recover PyNvVideoCodec decoder slots after failures ( #59597 )
...
Signed-off-by: Daniel Afrimi <dafrimi@nvidia.com >
Signed-off-by: danielafrimi <45691845+danielafrimi@users.noreply.github.com >
Signed-off-by: Daniel Afrimi <dafrimi@oci-hsg-cs-001-login-02.cm.cluster >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Daniel Afrimi <dafrimi@oci-hsg-cs-001-login-02.cm.cluster >
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com >
2026-10-15 09:17:19 +03:00
liranschour and Kevin H. Luu
c5f8adc0b1
[KV Offload] Expose OffloadingManager.lock()/unlock() and service the tiering control plane off the scheduler thread ( #58168 )
...
Signed-off-by: Liran Schour <lirans@il.ibm.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 08:50:56 +03:00
Harshil and Cyrus Leung
c54dca96bd
[Bugfix][Benchmark] Fix probe latency and duration in bench serve ( #60511 )
...
Signed-off-by: QHarshil <harshil_c@hotmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-15 05:50:30 +00:00
5341b5ba7d
[KV Offload] Pin only each worker's own offload slots ( #59544 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-15 08:49:42 +03:00
6605802904
[Perf][KV Connector][Mooncake] Send the KV layout once per peer ( #59825 )
...
Signed-off-by: staryxchen <staryxchen@tencent.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-10-15 05:36:31 +00:00
a400804ce4
[CI] Relax MTP sharded sampling cross-boot noise bound ( #57026 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 22:31:45 -07:00
3f1bcfd82b
[Bugfix][KV Offload] Bound unanswered P2P lookup waits ( #56836 )
...
Signed-off-by: jacklin78911-collab <jacklin78911@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 08:23:34 +03:00
81e1c3f183
[XPU][MLA] Integrate XPU MLA support into the existing FLASH_ATTN_MLA backend ( #58297 )
...
Signed-off-by: baodii <di.bao@intel.com >
Signed-off-by: Zhouxiaoya25 <xiaoya.zhou@intel.com >
Signed-off-by: ZhouXiaoya25 <xiaoya.zhou@intel.com >
Co-authored-by: baodii <di.bao@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-15 12:47:37 +08:00
Vivaan Sharma and Kevin H. Luu
96b5b0ee4b
[Model] Enable torch.compile and CUDA graphs for EmbeddingGemma2 ( #61572 )
...
Signed-off-by: Vivaan Sharma <99265792+Vivaansharma2003@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 12:32:39 +08:00
Haiyu Zuo and Kevin H. Luu
ff5a91a456
[Feature][Model Loader] Apply model-owned MTP shard selection to RunAI ModelStreamer ( #60759 )
...
Signed-off-by: 浩韵 <zuohaiyu.zhy@alibaba-inc.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 12:23:11 +08:00
Chaitanya Sri Krishna Lolla and Cursor
833665cded
[ROCm][DI] Enable MoRI FP4 dispatch for mxfp4 w4a4 moe ( #59596 )
...
Signed-off-by: lcskrishna <lollachaitanya@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-14 23:20:14 -05:00
Tuukka Sarvi and Kevin H. Luu
10d2c6083d
[ROCm][DSv4.1] Enable DSpark adaptive verification ( #58151 )
...
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 12:05:43 +08:00
Shanshan Shen
f46021a7fa
[ROCm][Perf] Enable DPA+ETP prefill compute/comm dual-batch-overlap for DeepSeek-V4 ( #57773 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-10-15 11:16:48 +08:00
Feanor and Taneem Ibrahim
5d5c3d7c43
[Bugfix] Preserve padding token types for cross encoders ( #58143 )
...
Signed-off-by: zhaoye <772971548@qq.com >
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-10-15 03:09:24 +00:00
429a3da665
[Bugfix][ROCm][Disagg] Allow MoRIIO HMA per-layer block_len ( #58585 )
...
Signed-off-by: avininjamay8 <Avinash.Paul@amd.com >
Signed-off-by: avininjamay8 <avpaul@amd.com >
Co-authored-by: avininjamay8 <avpaul@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-15 09:35:31 +08:00
Kai and Kunshang Ji
de2b869ee9
[Bugfix][Kernel] Register int8 scaled_mm as a custom op ( #57331 )
...
Signed-off-by: Kai <limkc.reply@gmail.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-15 09:27:49 +08:00
Aarushi Jain
2abd479aee
[ROCm][CI] Add AMD job for DeepSeek-V4.1 Mono Decode Numerics ( #61720 )
...
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com >
2026-10-15 01:23:25 +00:00
Woosuk Kwon and Claude Opus 5.5
e57bb4ace7
[Core] Remove the legacy Ray compiled-graph executor ( #61712 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 18:13:27 -07:00
bfdbfa1632
[Bugfix][PCP] Align MoE routing padding with gathered tokens ( #60946 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: wangyicong <wangyicong@bytedance.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-14 21:04:00 -04:00
Woosuk Kwon and Claude Opus 5.5
8530aed2b7
[Spec Decode] Remove Medusa and MLP speculator support ( #61701 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 17:29:23 -07:00
josh-schell-amd and Claude Opus 5.5
273a04964e
[ROCm] Bump AITER to v0.1.25 and relax the DSv4.1 mono MoE tolerance ( #61697 )
...
Signed-off-by: josh schell <josh.schell@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-14 23:50:24 +00:00
Aarushi Jain and Andreas Karatzas
7840bede98
[ROCm][CI] Add AMD jobs for test groups that only exist in tes… ( #59678 )
...
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-14 23:00:44 +00:00
Kevin H. Luu
4b61f300c0
[CI] Fix /ci run reporting a failed merge on fork PRs ( #61684 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-10-14 15:26:15 -07:00
Micah Williamson
a4d553a46a
Revert "[Build] Update deep_gemm git tag for locality_domain update" ( #61703 )
2026-10-14 14:57:44 -07:00
Rohan Potdar and Claude Opus 5.5
9a648b4649
[ROCm][Kimi-K3] Don't read W_UK_T before weight loading in the MLA wrapper ( #61654 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 21:44:55 +00:00
Ben Browning and Flora Feng
4c5ea86f53
[Bugfix][Parser] Stream Gemma4 string tool arguments incrementally ( #61494 )
...
Signed-off-by: Ben Browning <56071+bbrowning@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-14 21:25:41 +00:00
984c242ede
[Quantization] Support compressed-tensors MXFP4 MoE with FP8 activations via Mxfp4MoEMethod ( #59815 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 13:33:35 -07:00
fe98d90a9d
[Dependency] Update Humming and remove NVFP4 scale workaround ( #61556 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 13:32:36 -07:00
Woosuk Kwon and Claude Opus 5.5
2e8a5930f6
[Benchmark] Remove the temporary bench forward dummy loader ( #61676 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 13:30:18 -07:00
Juntian Liu
7c3f7e5500
[Perf][DSv4.1] Length-aware candidate block selection in the decode indexer ( #61039 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
2026-10-14 20:15:58 +00:00
rasmith and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2622ee21aa
[ROCm][BugFix] Don't use AITER rms_norm as an IR OP on gfx90a when AITER always on. ( #59290 )
...
Signed-off-by: Randall Smith <Randall.Smith@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 12:48:37 -07:00
alp and mottopanikeiku
e73fa00981
[Bugfix][Frontend] Apply the JSON schema in Cohere Chat v2 response_format ( #60586 )
...
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
2026-10-14 19:42:20 +00:00
Chris Leonard and Shengqi Chen
ded7ae35e5
[Build] Update deep_gemm git tag for locality_domain update ( #61491 )
...
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Signed-off-by: Shengqi Chen <harry-chen@outlook.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-14 15:41:16 -04:00
Juntian Liu and Claude Opus 5.5
388a610dfb
[Perf][DFlash] Capture context combine and anchor in the draft CUDA graph ( #59511 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-14 15:40:52 -04:00
Wei Zhao and Jee Jee Li
4975fda882
[Dependency] Upgrade FlashInfer to 0.7.1 ( #60948 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-10-14 12:31:08 -07:00
Ayushman Singh
9b5c6707a3
[Bugfix][MRV2] Capture DCP FULL cudagraphs with KV context ( #61496 )
...
Signed-off-by: Ayushman Singh <40520701+ayush1399@users.noreply.github.com >
2026-10-14 19:20:48 +00:00
Stefano Castagnetta and Misha Goin
222e00fc44
[CI/Build] Fix test_deepseek_v41_mxfp8_scale_loading after #57797 ( #61678 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-14 12:17:59 -07:00
Andreas Karatzas and Claude
e96c822660
[Bugfix][ROCm][MiniMax-M3] Reject unbuilt index-head counts in the AITER indexer ( #60990 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-14 14:13:48 -05:00
Sandeep Agrawal and Kevin H. Luu
21e207efa1
[Bugfix][MRV2][Spec Decode] Honor EAGLE3 single-state draft configuration ( #61563 )
...
Signed-off-by: Sandeep Agrawal <sandeepa@nvidia.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 12:09:06 -07:00
amd-xavierwang and JartX
06265beb25
[ROCm] Enable fp8 query for Triton attention ( #60296 )
...
Signed-off-by: xawang <xawang@amd.com >
Co-authored-by: JartX <sagformas@epdcenter.es >
2026-10-14 15:03:20 -04:00
857ddac4e5
[Perf][MRV2] Skip empty draft-token resets on request admission ( #61640 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-14 12:01:01 -07:00
c1407a7e86
[Spec Decode][MRV2] Warm up top-k/top-p and greedy rejection kernels for speculative decoding ( #59260 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 14:59:21 -04:00
simon-veitner-redhat and Claude
ae94e9c9e8
[Bugfix] Don't crash DeepGEMM MegaMoE warmup when get_bf16_mega_gate_config is missing ( #61657 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-14 18:58:35 +00:00
Simon Danielsson
3f729dafcc
[Perf][ROCm][GLM-5.3-Flash] AITER prefill indexer topk ( #60753 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-10-14 13:48:56 -05:00
lucas-santos-amd and Douglas Lehr
e534024e49
[ROCm][Bugfix][MLA] Fix FP8 MLA prefill unpad for interleave-padded heads ( #60907 )
...
Signed-off-by: Lucas Santos <Lucas.Santos@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-14 14:34:28 -04:00
d1d8e6c096
[Perf][Nemotron-Parse] Do image rescale/normalize on the device ( #60500 )
...
Signed-off-by: DiegoCao <DiegoCao@users.noreply.github.com >
Co-authored-by: DiegoCao <DiegoCao@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 18:33:22 +00:00
Kevin H. Luu and OpenAI Codex
0b6d07cbe2
[CI/Build] Use FileStore for DeepEP test rendezvous ( #56595 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-14 14:29:12 -04:00
Robert Shaw and Claude
d3f6b2a7b3
[Frontend] Clean Client Facing 500 Report ( #58043 )
...
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-14 14:27:58 -04:00
Ganesh R and Cyrus Leung
47232f3c7f
[Bugfix][CI] Fix renderers tests broken by tool name removal from warn logs ( #61652 )
...
Signed-off-by: Ganesh R <ganesh.r@amd.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-14 18:14:58 +00:00
Rohan Potdar and Claude Opus 5.5
52a8cc5316
[ROCm][CI] Add dtype to the KDA metadata test's fake model config ( #61650 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 18:04:14 +00:00
Woosuk Kwon and Claude Opus 5.5
9d3dfa2886
[Benchmark] Add vllm bench forward: exact-shape forward timing ( #61339 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 10:52:26 -07:00
Taimys and OpenAI Codex
2c74fd1873
[Model Runner V2] Enable batch-sharded sampling for GPT-OSS ( #61257 )
...
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-14 10:11:11 -07:00
Xunzhuo
89cab55086
[Model] Support Decision 3.0 decision models (vllm-sr/d3) ( #61565 )
...
Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com >
2026-10-14 17:10:55 +00:00
zexplorerhj and Codex
3895a8babe
[Bugfix] Fix TP loading of small block-FP8 MLA gates in BailingMoeV3 ( #61642 )
...
Signed-off-by: zexplorerhj <zhjoneson@163.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-14 10:09:23 -07:00
SeongJun Lee and Wei Zhao
e2e3e2b834
[CI] Use an ungated model in the DGX Spark XQA test ( #61416 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-14 17:04:07 +00:00
Harry Mellor and Claude Opus 5.5
3e851a7803
[Bugfix][Attention] Make the query contiguous for TRTLLM ragged MLA prefill ( #61634 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 16:38:52 +00:00
Zheng Gong and Cursor
355553ac22
[ROCm][Perf] Drop two dead per-step ops from the AITER sparse MLA metadata build ( #58354 )
...
Signed-off-by: ZhengGong-amd <zgong@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-14 11:15:51 -05:00
13dbf96093
[Model Loader] Apply EP weight filtering in InstantTensor ( #61167 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-14 09:10:10 -07:00
Nick Hill and Ilia Yastrebov
eaf25f675a
[NIXL] Prepare KV transfer descriptors as NIXL 1.5 strided lists ( #61032 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Ilia Yastrebov <iyastrebov@nvidia.com >
2026-10-14 18:07:24 +02:00
Nan Zhang and Robert Shaw
9e2b579aab
[Model] Support EAGLE3/DFlash aux hidden states for GraniteMoe and GraniteMoeShared ( #61110 )
...
Signed-off-by: ZN1010 <neilzhang1010@126.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-14 12:01:14 -04:00
e2aabc8b15
[Bugfix][KV Offload][P2P] Keep store pins and AbortAck until NIXL transfers stop ( #59790 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 18:31:31 +03:00
Juan Pérez de Algaba and OpenAI Codex
5d88c034f6
[Security] Reject local verifier references in Speculators metadata ( #61587 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-14 15:30:20 +00:00
bf6c80c7e8
[Misc] Remove tool name from warn logs ( #59294 )
...
Signed-off-by: Benjamin Merkel <benjamin.merkel@tngtech.com >
Co-authored-by: Benjamin Merkel <benjamin.merkel@tngtech.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-14 08:03:11 -07:00
Itay Etelis and Codex
efad400092
[HiSparse] Prefault the shared host pool in parallel ( #60869 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-14 10:28:26 -04:00
02653fa22c
[Bugfix][KV Offload] Fix inaccurate proportional store dropping ( #59140 )
...
Signed-off-by: Thuongvu Ho <thuongvuho@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-14 17:05:21 +03:00
Ashish Rao and Vadim Gimpelson
b4b7eca0ef
[Bugfix][Quantization] Clamp FP8 scales before UE8M0 rounding ( #58032 )
...
Signed-off-by: Ashish Rao <asrao@nvidia.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
2026-10-14 13:56:42 +00:00
7115b70c01
[Bugfix][KV Offload][P2P] Drop a timed-out store job's blocks from the round's parked supply ( #59329 )
...
Signed-off-by: Sohom Chakraborty <16609933+sohom-cs@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-14 13:22:28 +00:00
Luca Motz and Claude Opus 5.5
2d09fdcdc2
[Quantization] Auto-disable DeepGEMM for Kolibri 1 on SM12x ( #61274 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 12:59:50 +00:00
3723a1ea96
[Bugfix][Transformers backend] Fix CUDA graph capture for dynamic-rope models ( #58064 )
...
Signed-off-by: YongfengX <erkang040804@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 05:56:54 -07:00
hcl and Harry Mellor
2631afd3fe
[Bugfix][Platform] Fall back to torch when device-level NVML memory is unsupported (GB10) ( #57378 )
...
Signed-off-by: hclsys <chenglunhu@gmail.com >
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-14 12:45:29 +00:00
Kapil Arya and Or Ozeri
a68dbbad05
[Bugfix][KV Offload] Retain offload mappings after unsafe KVCR startup failure ( #57500 )
...
Signed-off-by: Kapil Arya <kapila@nvidia.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-14 15:43:04 +03:00
775fefa804
[Metrics][KV Offload] Add vllm:kv_offload_config_info - generic info metrics for offloading the offloading connector ( #56867 )
...
Signed-off-by: Amir Friedman <Amir.Friedman1@ibm.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-14 12:34:37 +00:00
ssingh231219
3a5a6b11a3
[Perf][Model] Batch PaddleOCR-VL vision encoder and projector ( #61458 )
...
Signed-off-by: ssingh231219 <simran.singh@artificialy.com >
2026-10-14 19:59:44 +08:00
d39c2fb05a
[Bugfix][Gemma4] Quantize audio towers independently ( #59935 )
...
Signed-off-by: sudhanshu shukla <sudhanshu112233shukla@users.noreply.github.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: sudhanshu shukla <sudhanshu112233shukla@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-14 11:27:44 +00:00
Lazurite
23fbc2ec2f
[Model] Support precomputed image embeddings for LightOnOCR ( #61585 )
...
Signed-off-by: Rapisurazurite <43494437+Rapisurazurite@users.noreply.github.com >
2026-10-14 10:54:09 +00:00
xuebwang
81503550f3
[ROCm] Disable AITER FP8/FP4 BMM for MLA by default ( #53123 )
...
Signed-off-by: xuebwang-amd <xuebwang@amd.com >
2026-10-14 10:46:02 +00:00
linshi wang and Or Ozeri
c67c69e657
[Bugfix][KV Offload] Track allocated block IDs when num_external_tokens is 0 ( #47046 )
...
Signed-off-by: linshi-w <linshiw.edu@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-14 13:34:10 +03:00
wang.yuqi
6ecb39eaa7
[Docs] Update structured decisions docs. ( #61588 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-14 10:28:22 +00:00
Ronald and aoshen02
19aba3e4d5
[Bugfix][RL] Load attention sinks through weight_loader for layerwise reload ( #57797 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-14 10:17:58 +00:00
f9b65c30c7
[ROCm][Perf] Make AITER FlyDSL GDN prefill default on gfx942/gfx950 ( #60645 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 10:11:57 +00:00
3f466c852c
[Bugfix][Parser] Recover genuine tool calls after unclosed Qwen3 <function= opener in prose ( #59928 )
...
Signed-off-by: Abhay Joshi <105213625+abhayjoshi201@users.noreply.github.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Gemini <noreply@google.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-14 13:01:25 +03:00
8a8e2baa02
[Bugfix][Rust Frontend] Inject VLLM_USE_RUST_FRONTEND=1 into vllm-rs managed engine envs ( #61586 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-14 17:09:08 +08:00
Sherif Waly
be4ce2e505
[Bugfix] Share PCP-aware local GPU indexing for NUMA binding ( #61237 )
2026-10-14 10:54:03 +02:00
Thien Tran and Kimi Code
937d6c19df
[Perf][GDN] Skip FLA chunk metadata when FlashInfer GDN prefill is active ( #61569 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-10-14 01:46:54 -07:00
Kevin H. Luu and Misha Goin
d36158f919
[CI] Merge the target branch into stale PRs on /ci run ( #61592 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-14 01:43:03 -07:00
Artem Perevedentsev
bb1438a93f
[CI] Fix Unlimited-OCR loading after upstream model_type rename ( #61590 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-14 01:24:24 -07:00
467b024f21
[Rust Frontend] Report vllm-proto schema version in ServerInfo ( #61533 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-14 16:21:48 +08:00
aa0c433a3e
[Frontend] Accept --reasoning-parser none as the unset default ( #61525 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-14 15:50:35 +08:00
c9bfa574ce
[Online quantization] Extend re-quantization API to MOE (showcase NVFP4 modelopt/CT/quark to any online MOE method) ( #60276 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com >
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com >
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com >
2026-10-14 15:49:37 +08:00
Chauncey and Cyrus Leung
e1726d9142
[Frontend] Add cache salt and tracing to openai decisions ( #61550 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-14 00:48:44 -07:00
zofia
4dc5add6ea
[XPU] [Linear] [MXFP8] enable torch as mxfp8 linear backend ( #51933 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Signed-off-by: zofia <110436990+zufangzhu@users.noreply.github.com >
2026-10-14 00:48:38 -07:00
Ronald
ec8a372566
[ROCm][Bugfix] Pin AITER back to v0.1.24.post1 for DSv4.1 mono decode numerics ( #61559 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
2026-10-14 00:19:38 -07:00
30e79c3a76
[Core] Update PyTorch to 2.14.0, torchvision to 0.29.0, triton to 3.8.0 (release channel) ( #52183 )
...
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com >
Signed-off-by: Andrey Talman <atalman@fb.com >
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 00:17:59 -07:00
ec317a4618
[CI] Keep vLLM log propagation on through the first LLM init in caplog_vllm ( #61535 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-14 15:10:35 +08:00
Jiangyun Zhu and Claude Opus 5.5
badfcdbf91
[Perf][GDN] Internal prefill checkpoints for the FlashInfer GDN prefill backend ( #60050 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 14:49:46 +08:00
a2fb482732
[KVConnector][P2P] Register peers off the scheduler thread ( #57139 )
...
Signed-off-by: Liran Schour <lirans@il.ibm.com >
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-14 08:21:57 +03:00
77154ca611
[Perf][Model][Qwen4Exp] QSA: decode four FP8 e4m3 bytes per register below sm_89 ( #60992 )
...
Signed-off-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-14 13:06:16 +08:00
Akshan Krithick
02d69c663a
[Bugfix][Benchmark] Preserve SSE output across transport chunk boundaries ( #51172 )
...
Signed-off-by: Akshan Krithick <akshankrithick305@gmail.com >
2026-10-13 21:58:58 -07:00
a78d050eff
[Bugfix] Bump tokenspeed-mla to 2.0.2 to fix nanobind conflict with Triton 3.9 ( #61125 )
...
Signed-off-by: Warren Deng <warrdeng@meta.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-13 21:28:07 -07:00
3ccc0a90b6
[KV Connector] Support sleep mode with MooncakeConnector over RDMA ( #59625 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: Dakai An <77474977+andakai@users.noreply.github.com >
2026-10-14 04:18:47 +00:00
Lanze Liu and Ning Xie
f4b8d43a68
[Bugfix] Keep force-killing the process tree when a child exits mid-walk ( #59984 )
...
Signed-off-by: Lanze Liu <lanzetech@gmail.com >
Co-authored-by: Ning Xie <andy.xning@gmail.com >
2026-10-13 20:33:18 -07:00
Shanshan Shen
2521446ac8
[ROCm][Perf] Enqueue the default chain first in CSA/HCA at small batch sizes for DeepSeek-V4 ( #61378 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-10-14 10:57:26 +08:00
2308b6f00b
[Perf][MoE] Cheaper triton_kernels 3.8 routing build in the OAI Triton MoE ( #58897 )
...
Signed-off-by: Zheng Gong <zgong@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-14 10:48:05 +08:00
xaguilar-amd
a1f51a0226
[ROCm][MLA] Route gfx942 DCP causal multi-token verify to Triton split-KV decode ( #58555 )
...
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com >
2026-10-14 10:17:15 +08:00
John Calderon and Artem Perevedentsev
449c05a0ca
[CI][Tests] Upgrade torch version for Rubin prerelease ( #61468 )
...
Signed-off-by: John Calderon <jcalderon@nvidia.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-14 10:10:00 +08:00
Kevin H. Luu and Codex
d5e62627a3
[CI] Shard chat APIs, model evals and model tests automatically ( #61475 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-13 18:16:27 -07:00
Stefano Castagnetta and Kartikey Vyas
4e0681ed56
[Quantization] Support packed NVFP4 Qwen4Exp PLE embeddings ( #56273 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Kartikey Vyas <kartikey@artificialanalysis.ai >
2026-10-14 08:24:55 +08:00
Almog Tavor and Itay Alroy
87e7dce3a7
[MRV2] Keep warmup out of the EPLB load stats ( #56716 )
...
Signed-off-by: almogtavor <almogtavor@gmail.com >
Co-authored-by: Itay Alroy <75032521+itayalroy@users.noreply.github.com >
2026-10-14 00:00:48 +00:00
Ryan Rock and Micah Williamson
f43d7e1a0f
[CI][AMD] Gate Aiter MHA FA on dtype ( #60679 )
...
Signed-off-by: Ryan Rock <ryan.rock@amd.com >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
2026-10-13 18:55:19 -05:00
josh-schell-amd and Micah Williamson
7a9d3db896
[ROCm] Bump ROCm base images to AITER v1.25.0 ( #61230 )
...
Signed-off-by: josh schell <josh.schell@amd.com >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
2026-10-13 18:55:09 -05:00
c923e7769b
[Feat] Add text-only OpenAI Decisions API MVP ( #60465 )
...
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-14 07:51:08 +08:00
Chauncey and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ad8d39cbab
[Bugfix] Fix DSpark graph-padding handling in the DeepSeek-V4.1 MoE fused top-k ( #59166 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 07:49:52 +08:00
Giancarlo Delfin and Woosuk Kwon
791f92135a
[Attention] Set ALWAYS cudagraph support for FA4 ( #60908 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
2026-10-13 16:22:52 -07:00
Kevin H. Luu and Codex
6a03954b45
[CI] Shard quantization, pooling, multimodal APIs, core kernels and PCP eval automatically ( #61477 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-13 16:15:30 -07:00
Kevin H. Luu and Codex
0da66a6e46
[CI] Shard batch invariance, Kimi prefix cache, EP eval and generate APIs automatically ( #61474 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-13 16:14:18 -07:00
8082b62f32
[Mooncake] Allow Mamba cache mode none for capacity-only workers ( #60712 )
...
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 23:13:58 +00:00
Tim002 and Thien Tran
96fc2da315
[Perf][Qwen4Exp] Add SM121 TP=2 merged QSA projection plans ( #61437 )
...
Signed-off-by: TimRen01 <HolaTim@hotmail.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-13 22:58:44 +00:00
Nick Hill and Claude Opus 5.5
78b32b3fdc
Revert "[Build] DeepGEMM migration to torch Stable ABI ( #59482 )" ( #61510 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-13 15:33:50 -07:00
Wenbin Chen and Misha Goin
6463d04601
Fix write-after-write hazard in xpu KV RoPE triton kernel ( #46685 )
...
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-13 22:17:14 +00:00
698e79f3bd
[Model] Kolibri 1 via the Transformers modeling backend ( #60827 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-13 22:17:52 +01:00
Anirudh Ravichandran
e90158d912
[Bugfix][MoE] Clamp BLOCK_SIZE_K to the FP8 quantization block in the batched Triton MoE ( #60733 )
2026-10-13 17:02:17 -04:00
Matthew Bonanni and Claude Opus 5.5
292da21f9d
[HiSparse] Read GPU-resident KV directly for mixed and BF16 prefills ( #60417 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 16:40:20 -04:00
Wentao Ye
c19c263ac8
[GLM 5.3 Perf] Handle two elements at the same time for per token group quant, 2.8% E2E ITL improvement ( #60927 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-13 13:06:41 -07:00
Andreas Karatzas and Claude
67516b6c42
[Bugfix][ROCm][Kimi-K3] Stage fused KDA decode weights after loading ( #61207 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-13 14:55:57 -05:00
djramic
93d0bd9d50
[ROCm][CI] Drop the -inf padding check from the AITER MQA logits test ( #61420 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
2026-10-13 15:49:30 -04:00
a31054f5ef
[NIXL] Bump CUDA 12 NIXL to 1.5.0 ( #60428 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 12:47:41 -07:00
68fc395554
[Bugfix] Keep GateLinear batch invariant under VLLM_BATCH_INVARIANT ( #59985 )
...
Signed-off-by: yifanFengg <evanfengff@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-13 15:43:54 -04:00
yifanFengg and Claude Fable 5.1
b33cb81237
[Bugfix] Keep compressed-tensors wNa16 linear layers batch invariant ( #60524 )
...
Signed-off-by: yifanFengg <evanfengff@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-13 15:43:40 -04:00
Wentao Ye
2f18d13cf0
[GLM5.3 Perf] Enable fp8 kv cache for GLM5.3 Flash by default, 9.8% E2E Throughput Improvement ( #61228 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-13 15:29:17 -04:00
483e0c24c7
[KV Offload] Split replicated-layout pre-faulting across workers ( #59545 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 19:18:56 +00:00
c068db0e1d
[Bugfix][Compile] Replace Transformers AOT caching with graph caching ( #58918 )
...
Signed-off-by: fangyao fan <2281693949@qq.com >
Signed-off-by: Taimys <ffy2281693949@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Taimys <ffy2281693949@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-13 19:17:29 +00:00
Anish Shanbhag and OpenAI Codex
3f550129e8
[Perf][Attention][MiniMax-M3] Select CUTLASS MSA by EAGLE3 decode shape ( #55838 )
...
Signed-off-by: Anish Shanbhag <ashanbhag@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-13 15:05:28 -04:00
GuanLuo and sfeng33
6d1daa9fe5
[Bugfix][Frontend] Expose completion response schemas in OpenAPI ( #60926 )
...
Signed-off-by: Guan Luo <gluo@nvidia.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-13 18:54:06 +00:00
c5bda658d2
[Model][MoE] Load MegaMoE experts directly into the DeepGEMM layout ( #60096 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Aaron Hao <hao-aaron@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Aaron Hao <hao-aaron@users.noreply.github.com >
2026-10-13 11:52:08 -07:00
Matthew Bonanni and Claude Opus 5.5
624a5ddafd
[Attention] Fetch MLA prefill workspace at use instead of init ( #60654 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 17:25:03 +00:00
e83222396f
[ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches ( #57199 )
...
Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-13 17:21:30 +00:00
Roberto L. Castro and Claude Opus 5.5
7fad7db330
[Perf][Feat] Add generic cuteDSL LL FP32 router (GEMM) ( #44343 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 12:38:46 -04:00
6849ca696a
[Quantization] Add ModelOpt IQ2_XS, IQ2_XXS, and Q8_0 support via b12x ( #59604 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Signed-off-by: Luke Alonso <lalonso@gmail.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Luke Alonso <lalonso@gmail.com >
2026-10-13 09:06:33 -07:00
Woosuk Kwon and Claude Opus 5.5
ac71b17517
[Model Loader] Keep norm layers neutral under dummy weight init ( #61323 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 08:54:24 -07:00
Taksh Kothari
12b005eefa
[Bugfix] Don't split a multimodal item that a prefill chunk starts exactly on ( #52412 )
...
Signed-off-by: Taksh <takshkothari09@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-13 15:32:38 +00:00
Elvir Crnčević
b9b401b2ef
[Kernel] Fuse DeepEP-v2 decode globalize+align into one launch ( #53911 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
2026-10-13 11:10:38 -04:00
wang.yuqi
669f85e686
[Misc] Break chat_utils into more maintainable modules. ( #60972 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-13 15:08:07 +00:00
Rishabh Agarwal
f395a6b1ad
[ROCM][Bugfix] Deferring to TRITON_ATTN backend for Prefix lm models instead of ROCM_ATTN and fixed minor bug in Gemma3 Multimodal code ( #46754 )
...
Signed-off-by: GodRishUniverse <rishabh.agarwal@amd.com >
Signed-off-by: Rishabh Agarwal <risagarw@amd.com >
Signed-off-by: GodRishUniverse <risagarw@amd.com >
2026-10-13 10:03:48 -05:00
Chris Leonard and Shengqi Chen
f5fefbffe5
[Build] DeepGEMM migration to torch Stable ABI ( #59482 )
...
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Signed-off-by: Shengqi Chen <harry-chen@outlook.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-13 08:01:44 -07:00
Jiangyun Zhu and Claude Opus 5.5
61d9c777bf
[CI] Run the Clef pooler test on GPU ( #61409 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 14:42:53 +00:00
de8291e24f
[Core] Add lifecycle hooks for worker extensions ( #59463 )
...
Signed-off-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-13 21:51:05 +08:00
Thien Tran
0fae39a014
[Perf] per_token_group_quant_8bit: multiply by reciprocal instead of dividing per element ( #61343 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-13 21:37:10 +08:00
31e90ac370
[Attention] Enable FA4 sparse MQA decode on Blackwell ( #55866 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-13 09:31:37 -04:00
Nathan Sala and Shaun Kotek - Nvidia
0838cd6593
[Bugfix] Load and apply Nemotron Super final vision LayerNorm ( #61401 )
...
Signed-off-by: Shaun Kotek <skotek@nvidia.com >
Co-authored-by: Shaun Kotek - Nvidia <skotek@nvidia.com >
2026-10-13 12:42:29 +00:00
Hexiang Wang
7342dc0317
[MoRIIO] Require pending remote prefill for READ cache hits ( #60638 )
...
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
2026-10-13 20:19:53 +08:00
253e02b95e
[Perf] Warm up DeepGEMM MegaMoE kernels before serving ( #60468 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
2026-10-13 20:17:48 +08:00
Aditya Ganesh Kumar
d9a543c500
[Bugfix][Frontend] Forward --enable-log-outputs to the Cohere Chat v2 ( #61089 )
...
Signed-off-by: Aditya Ganesh Kumar <adty910@gmail.com >
2026-10-13 12:01:59 +00:00
818f70c295
[Bugfix] Mask special added tokens in xgrammar JSON, regex and EBNF grammars ( #60749 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-13 14:38:23 +03:00
Vadim Gimpelson and Claude
cd7517fb17
[Bugfix] Fix ReplaySSM ring shape assertion when TP does not divide Mamba2 n_groups ( #60704 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-13 11:31:53 +00:00
Linze Shi
e6e84f350c
[Feature][Pooling] Add offline fixed-length late chunking ( #60164 )
...
Signed-off-by: Linze-Shi <linzeshi0@gmail.com >
2026-10-13 11:16:21 +00:00
noone-zhhh and Isotr0py
426f3bd36b
[Bugfix][Model][Rust Frontend] Qwen3_ASR: Handle ragged Qwen audio batches from Rust frontend ( #60979 )
...
Signed-off-by: zh98 <workallday.zhh@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-13 10:49:39 +00:00
samayp01 and Artem Perevedentsev
54db759eea
[Bugfix][Structured Output] Reject structural_tag and unhandled specs in backend validation ( #58862 )
...
Signed-off-by: Samay <samay.piduru@gmail.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-13 13:27:34 +03:00
Juan Pérez de Algaba and OpenAI Codex
7173c64e2e
[Security] Bound decoded and downloaded media per request ( #61226 )
...
Signed-off-by: juanpas97 <juan.perezalgaba@gmail.com >
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-13 10:17:14 +00:00
JackRao123 and Claude Opus 5.5
877f6df889
[Bugfix][DSv4] Don't bake the C128 compressor skip into CUDA graphs ( #59709 )
...
Signed-off-by: Jack Rao <jack.rao@baseten.co >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 10:15:11 +00:00
c7a305ec10
[Bugfix][Mamba] Store conv_kernel - 1 inputs in Mamba2 prefill checkpoints ( #60797 )
...
Signed-off-by: Sungsoo Ha <sungsooh@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com >
2026-10-13 10:09:17 +00:00
Kevin H. Luu and Claude Opus 5.5
206e45d470
[CI] Shard V1 Attention automatically instead of by fixed pytest shards ( #61200 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 02:24:06 -07:00
Thien Tran and OpenAI Codex
656d0d4506
[Perf] Tune Triton GDN decode tiles on SM100-family GPUs ( #61355 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-13 17:01:12 +08:00
Vadim Gimpelson and Claude
183625d4f9
[Bugfix] Fall back to auto for unquantized MoE layers when --moe-backend is marlin ( #60219 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-13 08:48:13 +00:00
Harry Mellor and Claude Opus 5
6f11c220b5
[Transformers backend] Fuse MoE routers that score after the top-k ( #57410 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-13 08:41:23 +00:00
stefankoncarevic
24aa1191f4
[ROCm][Bugfix] Run the AITER AR+RMSNorm fallback test on the platform… ( #61350 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-13 08:35:57 +00:00
Jiangyun Zhu
ff305934f1
[Tests][Mamba] Wait for accepted counts in fake MTP proposer ( #61342 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-13 16:15:47 +08:00
zhouyou
3ad096faf8
[Bugfix][EC Connector] Don't create hash directories on cache-miss lookups in ECExampleConnector ( #60826 )
...
Signed-off-by: zhouyou9505 <zhouyou9505@gmail.com >
2026-10-13 15:59:48 +08:00
2cc2e68d1c
[Core][Mamba] Remove leftovers of mamba_cache_mode="all" ( #60043 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 00:58:59 -07:00
bc3ddb198a
[Model][Qwen4Exp] FP8 QSA main KV cache on Ampere (sm_80, sm_86) ( #60602 )
...
Signed-off-by: kimseunghyun <hellonypcteam@gmail.com >
Signed-off-by: kimseunghyun <incongnito12@gmail.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-13 15:30:19 +08:00
Kevin H. Luu and Claude Opus 5.5
4e4eab73ed
[CI] Shard Model Executor and L4 Pipeline + Context Parallelism automatically ( #61202 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 23:57:32 -07:00
e2ad0ce5f5
[Bugfix][LoRA] Slice expert LoRA by the unpadded per-rank size under TP ( #61235 )
...
Signed-off-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-13 14:25:16 +08:00
Yongye Zhu and Claude Opus 5.5
a6009ea85b
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer ( #61149 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-12 23:24:43 -07:00
Nicolò Lucchesi and Claude Opus 5.5
86b1c2ef15
[Attention][DCP] Support a2a DCP comm backend for FlashMLA sparse ( #61285 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 13:51:22 +08:00
380638afbb
[Model] Add Clef-Flash support ( #61308 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-13 05:46:37 +00:00
0d2689e0df
[ROCm][Perf] Skip redundant sparse index remap on non-indexer layers ( #51309 )
...
Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
2026-10-13 05:25:56 +00:00
2ce8012fa0
[ROCm][Perf] Skip cleaning sparse prefill MQA logits ( #51314 )
...
Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 00:21:52 -05:00
Divakar Verma and Claude Sonnet 5
f7ba007b5e
[ROCm][MoE] Split AiterExperts by weight-quantization key ( #60322 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-10-13 00:13:33 -05:00
b337523c11
[XPU] [Test] Extend spawn-safe sitecustomize monkeypatch to XPU ( #57911 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
Co-authored-by: xiangdong <40376367+zxd1997066@users.noreply.github.com >
2026-10-13 13:12:10 +08:00
Mook and cjackal
5555a2d8f7
[Bugfix][Model] Fix EXAONE-4.5 image inputs: ViT received un-normalised pixels ( #61104 )
...
Signed-off-by: godmook <cmoh4135@naver.com >
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-10-13 13:02:12 +08:00
Yoni Gozlan and Bugen Zhao
69d00bc264
[Bugfix] Accept whole-number floats for integer tool arguments in the response parser ( #61297 )
...
Signed-off-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-13 12:55:20 +08:00
dskwe and Cyrus Leung
28d679e762
[Bugfix] Preserve independent tokenizer code_revision in snapshot resolution ( #61272 )
...
Signed-off-by: Yuan Li <dskwelmcy@163.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-13 04:23:45 +00:00
Yoni Gozlan and Bugen Zhao
657a2e9417
[Rust Frontend] Accept whole-number floats for integer tool arguments ( #61299 )
...
Signed-off-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-13 12:22:48 +08:00
RAHUL KUMAR and Claude Sonnet 5.5
c5422f4c46
[Test][Renderer] Cover the reasoning -> reasoning_content mirror in chat message parsing ( #56129 )
...
Signed-off-by: pctablet505 <55033230+pctablet505@users.noreply.github.com >
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com >
2026-10-13 03:58:32 +00:00
df6c7ad031
[Core][Refactor] Move KV cache planning logic to kv_cache_planning.py ( #61183 )
...
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 03:05:39 +00:00
frida-andersson
c077dc663d
[ROCm][Perf][DSv4.1] Enable decoder SWA bounded replay ( #59039 )
...
Signed-off-by: Frida Andersson <fanderss@amd.com >
2026-10-13 02:19:14 +00:00
xiangdong and Kunshang Ji
69d2b6f82d
[CI][XPU] Standardize test job labels for intel jobs ( #51755 )
...
Signed-off-by: zengxian <xiangdong.zeng@intel.com >
Signed-off-by: xiangdong <40376367+zxd1997066@users.noreply.github.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-13 10:18:25 +08:00
Fangzhou Ai and Claude Opus 5.5
cc2a6eea1a
[ROCm][Perf][DSv4.1] Mono decode on every backbone layer ( #61082 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 10:16:50 +08:00
Woosuk Kwon and Claude Opus 5.5
f47792af7f
[Model Loader] Make dummy weight init dtype-aware ( #61134 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 18:33:26 -07:00
c6566bb9f6
[ROCm][MLA] Keep DCP prefill context in FP8 through AllGather ( #59070 )
...
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
2026-10-13 09:20:38 +08:00
98f7211162
[Attention][Spec Decode] Multi-query Triton split-K for spec-decode verify, with empty-segment/padding hardening ( #56148 )
...
Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com >
Signed-off-by: Venky <23023424+venkywonka@users.noreply.github.com >
Co-authored-by: OpenAI GPT-6 Astra <noreply@openai.com >
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-13 04:37:56 +04:00
189bbfb9f4
Add MoNe experts pruning support ( #50415 )
...
Signed-off-by: Ilya Boytsov <ieboytsov@nebius.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Ilya Boytsov <ieboytsov@nebius.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 23:18:43 +00:00
1484266938
[Perf] Improve _topk_topp_kernel() for masked logits ( #61213 )
...
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-12 16:01:11 -07:00
1bc6e072b0
[Rust Frontend] Add per-request timing metrics to chat and completions ( #59314 )
...
Signed-off-by: wxin <wxin@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 15:39:10 -07:00
802feb6770
[Bugfix][KVConnector] Fix MoRI-IO WRITE-mode requests stranded in WAITING_FOR_REMOTE_KVS ( #57705 )
...
Signed-off-by: Akshay Viswanathan <4953343+akshayv@users.noreply.github.com >
Co-authored-by: Akshay Viswanathan <4953343+akshayv@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com >
2026-10-12 22:29:31 +00:00
Maroon Ayoub and Flora Feng
44f0ddfd7b
[Bugfix][Parser] Stop re-parsing the whole GLM tool argument per chunk ( #61253 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-12 21:28:35 +00:00
Almog Tavor and Robert Shaw
69d54eb078
[Docs] Fix broken llm-d.ai links on the llm-d integration page ( #61022 )
...
Signed-off-by: Almog Tavor <almogtavor@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-12 21:26:43 +00:00
Matthew Bonanni and Claude Opus 5.5
fe341a53dd
[HiSparse] Report host pool in KV cache capacity logs ( #60285 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 20:49:49 +00:00
deedc7be00
[Perf][Humming] Skip the redundant MoE input copy before the w13 GEMM ( #59340 )
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-12 20:24:12 +00:00
Andrey Talman
654b725ea9
[CI] Pace the CRCR nightly report under the relay rate limit ( #61269 )
...
Signed-off-by: Andrey Talman <atalman@fb.com >
2026-10-12 13:23:53 -07:00
Mikko Tukiainen
abad32a718
[ROCm][Perf] Fix Gemma AR+RMS prefill fallback ( #58547 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Signed-off-by: Mikko Tukiainen <mikko.tukiainen@amd.com >
2026-10-12 19:53:08 +00:00
Jiangyun Zhu
fabaecbfa5
[Misc] Expand @ZJY0516 CODEOWNERS to Mamba and KV cache ( #61181 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-12 19:32:25 +00:00
Martin Hickey
7039c7f17b
[Bugfix][Rust Frontend] Omit unset sampling params in render output ( #61220 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-12 19:31:03 +00:00
ztysth and OpenAI Codex
1bdd3fa2d5
[Bugfix][Core] Check multimodal chunk boundaries before encoder cache reuse ( #59721 )
...
Signed-off-by: ztysth <rnk.liulw@gmail.com >
Signed-off-by: ztysth <166926442+ztysth@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-12 19:25:35 +00:00
Martin Hickey
08582942d8
[Bugfix][Frontend] Honor request_id and X-Request-Id in chat and completions render ( #61236 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-12 19:17:12 +00:00
Wentao Ye
d0948f754d
[Bug] Honor spec decode draft model config ( #61240 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-12 15:13:23 -04:00
Maroon Ayoub and Claude
d6aca837c3
[Attention] Make the sparse MLA prefill workspace size configurable ( #61023 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 14:56:51 -04:00
SeongJun Lee and Ning Xie
56e3b081ae
[Bugfix][weight loader] Stop warning when a parameter only widens the weight ( #60020 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Ning Xie <andy.xning@gmail.com >
2026-10-12 14:33:31 -04:00
8055293353
[ROCm][MLA] Cut ~21x host dispatch from the MLA metadata build ( #58381 )
...
Signed-off-by: lifulu <fululi12@amd.com >
Signed-off-by: lifulu <lifulu@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: lifulu <lifulu@users.noreply.github.com >
2026-10-12 13:11:20 -05:00
c9e62f13c2
[CI][ROCm] Add AITER nightly overlay scripts ( #60883 )
...
Signed-off-by: josh schell <josh.schell@amd.com >
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
2026-10-12 13:04:37 -05:00
Woosuk Kwon and Claude Opus 5.5
e1a5086256
[Attention] Remove cascade attention ( #61126 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 10:41:07 -07:00
5b0b8f251e
[Bugfix][HiSparse] Fix TP group hang on steps without scheduled tokens ( #60941 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 17:38:12 +00:00
a7b7bc1fbe
[Metrics] Count finished requests by watermark status ( #58715 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-12 17:21:22 +00:00
Misha Goin and Claude Opus 5.5
45bb710eca
[CI] Add nightly Python 3.14 smoke test ( #60456 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 10:11:26 -07:00
Laith Sakka and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
12e6a585ae
[Fix][Compile] Declare group_shape in the static FP8 quant fusion … ( #61108 )
...
Signed-off-by: Laith Sakka <lsakka@meta.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 01:09:57 +08:00
Chester
c599b5971f
[Bugfix][Frontend] Return all logprobs when completions requests logprobs=-1 ( #55717 )
...
Signed-off-by: Chester <42577861+superdiaodiao@users.noreply.github.com >
2026-10-12 17:05:55 +00:00
haotiaa
6e6e86078a
[Bugfix][Platform] Don't resolve current_platform during import vllm ( #60766 )
...
Signed-off-by: Haotian An <haotiaa@amazon.com >
2026-10-12 17:05:34 +00:00
fa39582756
[Frontend] Serve LiquidAI/d1-3B on /v1/systemone ( #60676 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 13:00:47 -04:00
Misha Goin and Claude
d79b52b182
[Misc] Consolidate csrc shared headers and prune dead code ( #60721 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 09:46:22 -07:00
e0317b28ce
[Feature] Support batch invariance on ROCm ( #52231 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-12 10:53:26 -05:00
TJian
50b21a2667
[Perf] [Feat] [ROCm] Add densemha support to ROCm AITER Sparse MLA ( #49263 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-10-12 15:50:28 +00:00
Nick Riasanovsky
a686c8e1b7
[Triton] Migrate Triton kernels off tl.make_block_ptr for triton-main compat ( #60502 )
...
Signed-off-by: Nick Riasanovsky <njriasan@meta.com >
2026-10-12 11:38:43 -04:00
Shang Wang and Claude Opus 5.5
ffec221276
[Bugfix][DSv4.1] Match decoder-replay input_ids dtype for FlashInfer MegaMoE ( #61041 )
...
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 14:23:00 +00:00
Hoai Vo and Claude Opus 5.5
f3cb10b690
[Model][LoRA] Enable LoRA support for BERT and RoBERTa embedding models ( #61033 )
...
Signed-off-by: hoaivoke <vokehoai@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 14:09:53 +00:00
Shang Wang and Claude Opus 5.5
716961cf1f
[Bugfix] Drop enable_in_kernel_fc2_reduce from FlashInfer CuteDSL MegaMoE config ( #59900 )
...
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 09:05:09 -05:00
Ilya Markov
a281ad32f0
[EPLB] Add EPLB support to dspark draft models (DeepSeekV4) ( #57549 )
...
Signed-off-by: Markov Ilya <markovilya197@gmail.com >
2026-10-12 10:03:09 -04:00
Ilya Markov
b3ee560ae8
[Bugfix][MoE] Apply expert_map remapping for CUTLASS FP4 MOE expert parallel ( #59042 )
...
Signed-off-by: Markov Ilya <markovilya197@gmail.com >
2026-10-12 10:02:56 -04:00
bzsuni
f022975964
[Bugfix] Reject beam search in completion render ( #61212 )
...
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io >
2026-10-12 13:49:46 +00:00
4c36d21280
[Bugfix][LoRA] Apply PEFT rank_pattern / alpha_pattern scaling per module ( #59801 )
...
Signed-off-by: suraj <surajsharan85@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-12 21:49:30 +08:00
3facf822b7
[Performance] Load HF tokenizers and processors from a local snapshot ( #60825 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-12 13:45:54 +00:00
092c35dd15
[Model] Support Cloudflare/clef-omni on /v1/systemone ( #61064 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-12 21:27:34 +08:00
DimensionSTP and Artem Perevedentsev
314ab8cffd
[Bugfix][Structured Output] Drain compilation tasks before releasing the backend ( #61047 )
...
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-12 16:25:36 +03:00
Chauncey
ac28877f0e
[Bugfix] Fix out-of-bounds lookahead in adaptive block verification ( #60945 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-10-12 09:21:35 -04:00
stefankoncarevic
8c175f403a
[CI][ROCm] Compare greedy output only in the AITER attn+quant test ( #61185 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-12 08:20:35 -05:00
Rohan Potdar and Claude Opus 5.5
7900d2eabc
[Test] Accept tied tokens at the k-th value in the Triton top-k test ( #60726 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 08:18:44 -05:00
Harry Huang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
c75cb63cd2
[Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next ( #56322 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-12 20:56:44 +08:00
Tianyu Guo
258b97ff6b
[Bugfix][EC Connector] Size transfers by measured encoder output widths ( #57904 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-10-12 12:32:39 +00:00
Liana Koleva and Bugen Zhao
aa1d44df16
[Bugfix][Rust Frontend] Alias qwen3_xml to qwen3_coder and add qwen3_json for the Qwen JSON format ( #60136 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-12 20:20:53 +08:00
8d7b93fec2
[Frontend] Report stop_reason on /inference/v1/generate and pass it through derender ( #60251 )
...
Signed-off-by: Hyeonki Hong <hhk7734@gmail.com >
Signed-off-by: Shimi Bandiel <shimib@google.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Hyeonki Hong <hhk7734@gmail.com >
Co-authored-by: Mimir <mimir@nousresearch.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-12 19:37:53 +08:00
32b05addc1
[Bugfix][Rust Frontend] Restore DP wave delivery after XPUB reconnects ( #60249 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 19:12:43 +08:00
Reid and Bugen Zhao
f5cda48c90
[Bugfix][Rust Frontend] Export corrupted request metric ( #59937 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-12 19:12:39 +08:00
3b9e462774
[Bugfix][Rust Frontend] Correct DP routing after request removal ( #59631 )
...
Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 19:12:35 +08:00
21445806bd
[Rust Frontend] Support logprob_token_ids on OpenAI endpoints ( #59549 )
...
Signed-off-by: Wang Zupeng <zupenwang@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 19:12:30 +08:00
Thang Nguyen and Claude Opus 5.5
ee8f7ea449
[CI] Enroll a fourth batch of five steps in automatic sharding ( #60858 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 03:46:07 -07:00
MatthewCYM
f7a3a7265f
[Bugfix][Multimodal] Fix Qwen3-ASR audio preprocessing parity ( #57146 )
...
Signed-off-by: MatthewCYM <chenyiming0v0@gmail.com >
2026-10-12 10:42:51 +00:00
Jimmy Wang and Juan Pérez de Algaba
2534a96fcf
[Docs] List additional unauthenticated endpoints ( #61081 )
...
Signed-off-by: Jimmy Wang <53066820+JimmyWang0417@users.noreply.github.com >
Co-authored-by: Juan Pérez de Algaba <124347725+jperezdealgaba@users.noreply.github.com >
2026-10-12 11:47:52 +02:00
Bugen Zhao and Claude Opus 5.5
5a09062632
[Rust Frontend] Compose answer constraints into parser-owned output grammars ( #60781 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 09:43:48 +00:00
0466bd36ea
[Rust Frontend] Build the Kimi K3 full-output grammar from its parser mode ( #59190 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-12 09:43:47 +00:00
Shijin Zhang and Chauncey
83773ecdf3
[Frontend] Support CustomToolCall in the non-Harmony Responses API ( #57322 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-12 17:22:55 +08:00
zhangchengzhucufe-dev and Kunshang Ji
a33d46b471
[Bugfix][XPU] Forward softcap and alibi_slopes in xpu_ops.flash_attn_varlen_func ( #58936 )
...
Signed-off-by: zhangchengzhucufe-dev <zhangchengzhucufe@gmail.com >
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-12 17:10:55 +08:00
Yan Ma
e05201aa2e
[XPU] refine memory release for sleep mode ( #61158 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-10-12 16:59:38 +08:00
Thuongvu Ho and Nicolò Lucchesi
99b44c7a42
[Metrics] Add admission rejection metrics ( #55812 )
...
Signed-off-by: Thuongvu Ho <thuongvuho@gmail.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
2026-10-12 08:31:07 +00:00
Cyrus Leung
5bed289707
[DX] Update CODEOWNERS for entrypoints ( #60984 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-10-12 08:06:47 +00:00
4163a80142
[Rust Frontend] Return weight-version spans on final outputs ( #61070 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 15:56:59 +08:00
Ting SUN
f4114e068e
[Bugfix][Frontend] Preserve token IDs in chat derender responses ( #61065 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-10-12 07:55:06 +00:00
AMD-yanfeiwang and aoshen02
e3bbaf2369
[ROCm][DSv4] Fuse the shared expert into AITER MegaMoE ( #61004 )
...
Signed-off-by: Yanfei Wang <yanfei.wang@amd.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-12 15:53:16 +08:00
Giancarlo Delfin and Rishi Puri
36f3972812
[Spec Decode] Skip metadata build for dflash/dspark when all model backends support it ( #60692 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Rishi Puri <20074092+puririshi98@users.noreply.github.com >
2026-10-12 00:09:10 -07:00
Zhenzhong Xu and Kunshang Ji
6a70913590
[XPU] bump up auto-round-lib to 0.16.0 ( #60976 )
...
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-12 14:47:59 +08:00
74f4404f58
[Bugfix][Distributed] Own SHM out-of-band buffers before releasing read slots ( #61000 )
...
Signed-off-by: Hubert Marek <163992334+hubert-marek@users.noreply.github.com >
Signed-off-by: Ao Shen <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Ao Shen <aoshen@inferact.ai >
2026-10-12 01:44:12 -04:00
Canlin Guo and Claude Opus 5.5
fe4df3df0e
[Bugfix][GLM-5.3-Flash] Restore MTP index sharing across draft iterations ( #61079 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 05:41:52 +00:00
d4e92ccc37
[Bugfix][Frontend] Honor endpoint plugin route overrides ( #58839 )
...
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-12 03:58:15 +00:00
liuzhenwei and Kunshang Ji
bfa7e5b7af
[XPU][CI] Enable XPU graph in graph tests ( #60784 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-12 10:39:40 +08:00
Yan Ma
f147949be8
[XPU] limit FA kernel block size to multipleOf 64 ( #57297 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-10-12 10:25:32 +08:00
dingdangmao and Codex
124b89d5b7
[Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries ( #59962 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-12 02:08:14 +00:00
Michał Ganczarenko and Claude Opus 5.5
03091e28b8
[Bugfix] Read Ling quantization_config from top-level config for Bailing V3 VL ( #59026 )
...
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 09:25:29 +08:00
Laith Sakka
cc9b0f097d
[Fix] MiniMax: fix CUDA IPC handle exchange on cuda-bindings >= 13.4 ( #60917 )
...
Signed-off-by: Laith Sakka <lsakka@meta.com >
2026-10-11 20:38:56 -04:00
Nick Hill
3fe917abb1
[Bugfix][MRV2] Fix FlashInfer attention and ReplaySSM warmups on MRV2 ( #61013 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-11 16:04:41 -07:00
f26d9f432c
[Bugfix][GLM-5.3-Flash] Honor --mamba-ssm-cache-dtype for the KDA recurrent state ( #59147 )
...
Signed-off-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-11 17:55:00 +00:00
Nick Hill
c059bee812
[MRV2] Switch CI tests still running on MRV1 unnecessarily ( #61014 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-11 09:57:07 -07:00
Ting SUN and Cyrus Leung
b99fdbe1f8
[Bugfix] Respect the configured offline chat template ( #61061 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-11 16:27:21 +00:00
93d2cab35e
[Security] Fix P2P ZMQ socket exhaustion via unbounded peer sessions ( #51504 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
2026-10-11 11:39:11 -04:00
aoshen02 and Codex
239a3fa3a8
[Sleep] Retain draft-private weights with existing parameter selector ( #61046 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 23:30:56 +08:00
3e8e1364ff
[Perf][GLM-5.3-Flash] Shard long-context indexer prefill rows across TP ( #54951 )
...
Signed-off-by: Zheng Cai <8370601+zigzagcai@users.noreply.github.com >
Signed-off-by: Zheng Cai <caizheng1993@gmail.com >
Signed-off-by: Codex <noreply@openai.com >
Co-authored-by: root <root@rjob-fb29567b88fec0a9-88c9a4c797eb2ec1-0.caizheng.ailab-sys.svc.pjlab.local >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-11 10:43:37 -04:00
aoshen02
bd5bbce6e4
[RL] Release weight-transfer communicators without stopping inference ( #61060 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: fh3 <a@b>
2026-10-11 21:15:14 +08:00
tc-mb and Cyrus Leung
c85616efc6
[Model] Support MiniCPM-V 4.7 ( #58674 )
...
Signed-off-by: tc-mb <tianchi_cai@icloud.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-11 10:21:10 +00:00
Subhash Polisetti
f425a21208
[Bugfix][Frontend] Support scalar priority and wire priority into LLM.chat() ( #57243 )
...
Signed-off-by: subhash polisetti <subhashr161347@gmail.com >
2026-10-11 10:03:33 +00:00
2339bd39d6
[Bugfix][Mooncake] Use the EAGLE replay boundary when retaining sliding-window tails ( #61012 )
...
Signed-off-by: Stu Cao <stu@primeintellect.ai >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 09:15:18 +00:00
aoshen02 and OpenAI Codex
1789f09588
[Frontend] Explain disabled scale-out endpoints with opt-in hints ( #61042 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-11 08:58:07 +00:00
Juan Pérez de Algaba and Cyrus Leung
fcae5c9887
[Security] Restrict default CORS origins ( #61009 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-11 08:26:02 +00:00
Xun Sun
26a488ad93
[Fast Start][Bugfix] Snapshot GPU memory after external weights are ready ( #60966 )
2026-10-11 07:10:04 +00:00
ef42b4385f
[Bugfix] Reclaim cached GPU memory during weight loading ( #57516 )
...
Signed-off-by: Debanshu Das <debanshu@google.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-10-11 07:03:06 +00:00
Bruno Volpato and Codex
61f084b5f3
[Bugfix][Multimodal] Fix per-video cache option reuse ( #55203 )
...
Signed-off-by: bvolpato <brunocvcunha@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 06:21:55 +00:00
Yongye Zhu and Claude Opus 5.5
2eb491e18b
[Perf][MiniMax-M3] MSA prefill attends with the E4M3 query; bump fmha_sm100 to 968061a ( #60684 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 13:40:45 +08:00
Nick Hill
f8471164bf
[MRV2] Detatch weight-reload logic from MRV1 ( #60937 )
2026-10-10 21:15:05 -07:00
bzsuni
01abe975b5
[Bugfix][KV Connector] Fix speculative block reservations across KV cache groups ( #59880 )
...
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io >
2026-10-10 20:55:41 -07:00
Shuolei Wang and aoshen02
b89852aad0
[RL][Frontend] Return token-level weight-version spans on final outputs ( #59582 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Shuolei Wang <948904026@qq.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-11 11:33:29 +08:00
3c062d391d
[Bugfix] Support 0-dim tensors in packed weight transfer ( #52235 )
...
Signed-off-by: subhash polisetti <subhashr161347@gmail.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 01:41:06 +00:00
Wentao Ye
276fbcff27
[Deprecation] Remove scheduled deprecated items ( #60923 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-10 15:45:33 -04:00
Yufeng He
187a0eb98a
[Bugfix][Frontend] Forward --enable-log-outputs to the Anthropic handler ( #60982 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
2026-10-10 16:36:03 +00:00
f2376cdc4d
[Fast Start] Preload the FlashInfer autotune table on the weight cache daemon ( #60085 )
...
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-10 15:48:06 +00:00
Wentao Ye
3709632ff2
[Refactor] Remove dead code multiple places ( #60120 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-10 15:41:10 +00:00
Wentao Ye
838526c7e5
[Compile] Fix sampler compile warning ( #60680 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-10 08:09:45 -07:00
Mook
bf16c9c139
[Bugfix][Model] K-EXAONE: restore loading of pre-2.0 configs and apply RoPE to the MTP layer ( #60187 )
...
Signed-off-by: godmook <cmoh4135@naver.com >
Signed-off-by: Mook <68294499+Godmook@users.noreply.github.com >
2026-10-10 15:05:55 +00:00
Tony Lin
2bea22002d
[BUG] Fix KV offload shared-memory slot selection using worker rank ( #58497 )
...
Signed-off-by: Tony Lin <tony.lin@intel.com >
2026-10-10 20:25:11 +08:00
Kay Yan and Codex
c41b2639e2
[Bugfix][Responses API] Complete cache-write token accounting ( #60973 )
...
Signed-off-by: Kay Yan <kay.yan@daocloud.io >
Co-authored-by: Codex <noreply@openai.com >
2026-10-10 09:08:36 +00:00
SeongJun Lee and lesj0610
a98247ab4d
[Model] Drop the 4x M-RoPE cache headroom for Qwen3-VL, Qwen3.5 and Qwen3.8 ( #59853 )
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-10-10 06:51:52 +00:00
wang.yuqi
c1692c85f1
[Misc] mv chat_utils.py to renderers. ( #60522 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-10 06:50:34 +00:00
SeongJun Lee and Isotr0py
699c892dd5
[Perf] Skip the M-RoPE warmup when every holder of the RoPE fuses it ( #60246 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-10 06:47:39 +00:00
liuzhenwei and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ff440c7805
[BreakableCG] Add eager break for Mamba-family ops ( #57881 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-10 14:06:57 +08:00
Andreas Karatzas and Claude
a205fb3177
[Bugfix][Model] Restore AXK1 config defaults dropped with the vendored config ( #60954 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-10 05:54:52 +00:00
devpramod and Claude Opus 5.5
46fb84c2d4
[Model] Support EAGLE3/DFlash aux hidden states for Granite ( #60855 )
...
Signed-off-by: Pai, Pramod <pramod.pai@intel.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-10 13:34:10 +08:00
Chauncey
fa5cfd48d5
[Refactor] Deduplicate trace header handling in BaseServing ( #60828 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-10-10 05:27:58 +00:00
Nick Hill
7d0b4e57aa
[ROCm] Remove MRV1 model exceptions ( #56360 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-10 05:09:19 +00:00
8ea646b827
[Rust Frontend][gRPC] Remove GenerateStream producer task so disconnects abort requests ( #60922 )
...
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-10-10 04:59:35 +00:00
linitra24
364bc81a9e
[Bugfix][LoRA] Fix GPT-OSS gate/up interleaving with mixed MoE LoRA formats ( #60812 )
...
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-10 12:52:51 +08:00
Puneet Sharma
5ca26880ad
Optimize ppc64le vLLM runtime image size ( #48074 )
...
Signed-off-by: puneetsharma21 <puneet.sharma21@ibm.com >
2026-10-10 12:39:58 +08:00
Guancheng Fu and Kunshang Ji
acec1b1d84
[Bugfix][XPU] Remove stale W4A8 weight after repacking ( #57511 )
...
Signed-off-by: Guancheng Fu <guancheng.fu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-10 04:27:35 +00:00
Matthew Bonanni and Claude Opus 5.5
d5f356dae0
[HiSparse] Own GPU residency instead of editing core block-table rows ( #60647 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 21:25:44 -07:00
SeongJun Lee and Misha Goin
9bd074a996
[Attention] Decode an NVFP4 KV cache with XQA on SM12x ( #60452 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-09 22:24:00 -06:00
Misha Goin and Claude Opus 5.5
29e4d2d42e
[Frontend][Build] Add vllm download-kernels for FlashInfer precompiled kernels ( #58765 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 21:22:35 -07:00
Miłosz Grunwald and Kunshang Ji
be0823357f
[TEST][XPU][CI] Fix ci uva kernel test instability 2 ( #60404 )
...
Signed-off-by: microslaw <milosz.grunwald@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-10 12:09:13 +08:00
Giulio De Pasquale
f1a3c60558
[Feature][Attention] Cover a whole spec-verify in one DiffKV split-KV program ( #59085 )
...
Signed-off-by: Giulio De Pasquale <git@depasquale.giugl.io >
2026-10-10 11:55:45 +08:00
2d69083c4a
[ROCm][MXFP4] Align the AITER W4A4 MoE intermediate size to 128 ( #55104 )
...
Signed-off-by: xiaofei.zheng <xiaofei.zheng@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
2026-10-10 11:14:01 +08:00
DimensionSTP and OpenAI Codex
10cc2f6ae2
[Bugfix] Bound GDN speculative decode query widths ( #55504 )
...
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-10 11:00:16 +08:00
02aa10539f
[Bugfix][ModelLoader] Make enable_weights_track check quantized layers ( #57108 )
...
Signed-off-by: siliangchen-amd <SiLiang.Chen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-10 10:44:06 +08:00
aoshen02 and Claude Opus 5.5
1dfe9fbdae
[Core] Release FlashInfer allreduce workspace with --enable-nccl-comm-suspend ( #59360 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 02:26:40 +00:00
Fangzhou Ai and Claude Opus 5.5
193922d6ff
[ROCm][Perf][DSv4.1] Mono decode layer for DeepSeek-V4.1-Flash on MI355X ( #60397 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 20:53:18 -05:00
3c802af3cf
[Feat][Frontend] structured decisions: DiffusionGemma read strategy ( #60466 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-10 09:33:22 +08:00
Nick Hill
7d1d8660d6
[MRV2] Miscellaneous code structure updates ( #60754 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-09 17:58:09 -07:00
9f1c783886
[Test] Allow one fp8 step on K bytes in the kpool decode fuzz test ( #60725 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-10 00:26:53 +00:00
f8e6e365ae
[Bugfix][Mooncake] Save the EAGLE attention proof with hybrid Mamba partial tails ( #60695 )
...
Signed-off-by: Dao Le <Dao007forever@gmail.com >
Signed-off-by: Dao007forever <daole@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-10 00:08:08 +00:00
Wei Zhao and Codex
03a7d3776f
[Perf] Support minimum KV splits configuration for TokenSpeed MLA decode ( #60345 )
...
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-09 23:33:47 +00:00
04b02fc8a5
[Core] shm tensor arena for efficient cpu->gpu worker broadcast ( #51207 )
...
Signed-off-by: Bolin Sun <bolins@nvidia.com >
Signed-off-by: <bolins@nvidia.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-09 16:13:24 -07:00
Rohan Potdar and Claude Opus 5.5
fa94ad4143
[Test] Allow one bf16 ulp in the fused mHC post/pre residual check ( #60724 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 17:54:06 -05:00
Matthew Bonanni and Claude Opus 5.5
4c43254d52
[HiSparse] Report host-pool block residency as HiSparse metrics ( #59485 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 16:45:34 -06:00
Kevin H. Luu and Claude Opus 5.5
783ac12071
[CI] Accept whitespace around /ci commands and reply instead of going silent ( #60925 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 15:43:37 -07:00
ed7ae705a1
[Bugfix][MRV2] Use int64 index mappings to fix row-offset overflow ( #60736 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-09 14:38:11 -07:00
51144dd83e
[Bugfix][Spec Decode] Trim token_ids/logprobs left past a stop string under speculative decoding ( #47616 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 14:22:31 -07:00
Bugen Zhao and Claude Opus 5.5
35e920d671
[Rust Frontend] Support --stream-interval and per-request stream_interval ( #60589 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 14:10:42 -07:00
Giancarlo Delfin
ffdde24f54
[Docs] Document adaptive verification without a confidence head ( #60913 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-10-09 13:54:40 -07:00
Kai Hoang and Flora Feng
e0cb3efce0
[Bugfix][Structured Output] Decode choice specs as JSON in outlines and LMFE ( #60748 )
...
Signed-off-by: hpdkhoa <hpdkhoa2311@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-09 20:52:08 +00:00
02e55d2af3
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots ( #57169 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-09 15:48:37 -04:00
Misha Goin and Claude
3766a67a32
[WideEP] Explain how to fix a too-old NCCL for DeepEP v2 ( #60449 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-09 12:34:58 -07:00
04462732a4
[Bugfix] Reject max_num_scheduled_tokens=0 at config time instead of hanging ( #60714 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Hui Li <40770106+untrall@users.noreply.github.com >
2026-10-09 19:15:10 +00:00
Willow Lopez and Artem Perevedentsev
ad8a6065f4
[Bugfix] Validate guidance tokenizer compatibility before EngineCore ( #52298 )
...
Signed-off-by: Oxygen56 <jiangth99@163.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-09 21:59:02 +03:00
Misha Goin and Claude
1dc60dd367
[CI] Pin pydantic in the mypy pre-commit hook ( #60894 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-09 11:42:41 -07:00
Ning Xie and Simon Mo
a0f28d73a8
[sharded state loader] support pp in sharded state loader ( #52110 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
Co-authored-by: Simon Mo <simon.mo@hey.com >
2026-10-09 11:08:31 -07:00
premsurawut and Claude Opus 5.5
c811943137
[Bugfix][Frontend] Reject beam search with echo and logprobs in /v1/completions ( #60876 )
...
Signed-off-by: surawut.j <surawut.jirasaktavee@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 18:06:26 +00:00
Bugen Zhao and Claude Opus 5.5
5dc9bcd8cd
[Rust Frontend] Pass reasoning controls through render -> generate ( #60546 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 11:04:49 -07:00
Bugen Zhao and Claude Opus 5.5
ce4603cad6
[Bugfix][Rust Frontend] Pass through raw characters in byte-level decode ( #59861 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 11:04:00 -07:00
Bugen Zhao and Claude Opus 5.5
9a585c62aa
[Rust Frontend] Box rarely-set fields of per-token output types ( #60588 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 11:01:16 -07:00
MOVZX
ac305c44cb
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-09 17:41:13 +00:00
1a3a7a8f01
[Bugfix] Treat max_tokens as output upper bound, not input-space reservation ( #57035 )
...
Signed-off-by: YuvrajSinghBhadoria2 <245782664+YuvrajSinghBhadoria2@users.noreply.github.com >
Signed-off-by: YuvrajSinghBhadoria2 <yuvrajsinghbhado2030@gmail.com >
Signed-off-by: Yuvraj Singh Bhadoria <yuvrajsinghbhado2030@gmail.com >
Co-authored-by: YuvrajSinghBhadoria2 <245782664+YuvrajSinghBhadoria2@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-10-10 01:15:22 +08:00
Andrey Talman and Andrey Talman
89ac8d971b
[CI][Test] Allow 10% ngram acceptance drop under preemption on CUDA in test_async_scheduling_accuracy ( #60715 )
...
pre-commit / pre-commit (push) Canceled after 0s
pre-commit / pre-run-check (push) Canceled after 0s
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com >
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com >
2026-10-09 16:59:17 +00:00
stefankoncarevic
b74233860d
[CI][ROCm] Wait for VRAM to settle before building LLM in test_full_graph ( #60872 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-09 16:48:43 +00:00
Taewoong Kim and Cyrus Leung
75e3b17c05
[Frontend] Add cache salt support to generative scoring ( #60841 )
...
Signed-off-by: Taewoong Kim <ktw2172@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-09 16:07:50 +00:00
90ba2f34b5
[Build] Find compatible published wheels for precompiled installs ( #58871 )
...
Signed-off-by: greg pereira <grpereir@redhat.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 08:05:54 -07:00
Abhijit Roy
970dc63478
[Docs] Add Giga-Embeddings bidirectional Qwen3 usage to embed docs ( #60854 )
...
Signed-off-by: Abhijit <abroy@redhat.com >
2026-10-09 14:41:38 +00:00
Taneem Ibrahim
a6b39f0262
[Bugfix] Clean up async client sockets after event loop closure ( #57757 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-10-09 10:32:47 -04:00
Warren Deng and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
fc39317392
[Bugfix][Kernel] Remove invalid NC from Kimi-K3 KDA autotune keys ( #60446 )
...
Signed-off-by: Warren Deng <warrdeng@meta.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 14:25:41 +00:00
Juan Pérez de Algaba and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
3d52578e13
[Security] Do not retain Phi-4 MM audio attention matrices ( #60614 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 22:23:35 +08:00
Mikko Tukiainen and Cursor
81c993658e
[ROCm][Perf] Fuse main QK-norm/RoPE/gate and KV-cache write into the AMD QSA prepare launch ( #59732 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-09 21:14:49 +08:00
Li, Jiang
9adba10f3f
[Bugfix] Fix pre-commit check ( #60840 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-10-09 12:48:32 +00:00
Suraj Sharan and linitra24
1ee69dde9c
[Bugfix][LoRA] Reject PEFT adapter features vLLM does not implement ( #60225 )
...
Signed-off-by: suraj <surajsharan85@gmail.com >
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-09 16:39:58 +04:00
Yan Ma
c17fadcfdc
[XPU] fix fused_input_norm for MM ( #59865 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-10-09 20:16:35 +08:00
YiSheng5 and Kunshang Ji
0cc0460870
[XPU]Fix the accuracy issue when meet topk_ids=-1 on DP+EP scenarios ( #57787 )
...
Signed-off-by: yisheng <yi.sheng@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-09 20:01:54 +08:00
Louie Tsai
196483e25b
[Tools] Recipes sweep workflow ( #57307 )
...
Signed-off-by: louie-tsai <louie.tsai@intel.com >
2026-10-09 19:44:09 +08:00
Praneeth Reddy A and Li, Jiang <jiang1.li@intel.com>
1c47ebfc57
[Docs] Fix VLLM_CPU_KVCACHE_SPACE default and units ( #60053 )
...
Signed-off-by: Praneeth <praneethreddyarikatla@gmail.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-10-09 19:43:55 +08:00
Chen Cheng and Isotr0py
d095e61c0b
[Bugfix][Spec Decode] Fix GLM-5.3-Flash DFlash2/EAGLE3 aux hidden-state capture with the upstream config ( #60763 )
...
Signed-off-by: Chen Cheng <85005591+ischencheng@users.noreply.github.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-09 11:35:48 +00:00
wang.yuqi
7368fadebd
[CI Failure] Move MTEB tests to h200_35gb ( #60817 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-09 11:34:38 +00:00
Luca Motz and Claude Opus 5.5
83ed0c7c09
[Bugfix][NIXL] Fix receive post-process for different P/D block sizes ( #58860 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 19:30:11 +08:00
Shreyansh Jain and Artem Perevedentsev
28d76f3bad
[Bugfix][Structured Output] Reject unsupported regex inside groups ( #60359 )
...
Signed-off-by: Shreyansh Jain <shreyanshjain646@gmail.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-09 14:16:13 +03:00
Chaojun Zhang
b027ac8312
[XPU] Support register KV offload mmap region as pinned host memory on XPU ( #51956 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
2026-10-09 18:58:07 +08:00
5cadabfafd
[Docs] Add Qwen2.5-Coder-7B-Instruct to batch invariance tested models ( #46368 )
...
Signed-off-by: Yuval Luria <yluria@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-09 10:41:29 +00:00
Yan Ma
4e318ac563
[XPU] fix online fp8_per_channel quantization ( #56027 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-10-09 18:38:01 +08:00
Thang Nguyen and Claude Opus 5.5
efd141047d
[CI] Enroll a third batch of five steps in automatic sharding ( #60757 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 03:26:47 -07:00
Thang Nguyen and Claude Opus 5.5
b8bd335754
[CI] Enroll five more steps in automatic sharding ( #60662 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 03:26:27 -07:00
Warren Deng
59ffd73e85
[Bugfix][Kernel] Restore FP fusion for indexer Q quantization ( #60716 )
...
Signed-off-by: Warren Deng <warrdeng@meta.com >
2026-10-09 09:46:41 +00:00
Damien Laine and Cyrus Leung
2186b89a9a
[Bugfix][Model] Voxtral realtime: fix boot OOM and engine crash at max_model_len ( #45022 )
...
Signed-off-by: Damien Laine <damien.laine@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-09 17:40:29 +08:00
Harry Mellor and Claude Opus 5.5
2c99ee9333
[Chore] Remove vendored code made redundant by Transformers 5.16.1 ( #60623 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 09:29:03 +00:00
Chao Lei
bc713cf8cc
[Bugfix]MooncakeConnect add query timeout and retry ( #55923 )
2026-10-09 09:19:33 +00:00
Taewoong Kim and Chauncey
18c19eaf23
[Frontend] Add cache salt and tracing to structured decisions ( #60786 )
...
Signed-off-by: Taewoong Kim <ktw2172@gmail.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-09 08:53:50 +00:00
a6f43e57d6
[ROCm] Enable the cuMem CUDA-graph pool offload on ROCm ( #59523 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: indianspeedster <cspandey016@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: indianspeedster <cspandey016@gmail.com >
2026-10-09 08:29:55 +00:00
The Anh Nguyen
bacbbe1878
[Bugfix][NIXL] Guard record_transfer against None telemetry ( #57734 )
...
Signed-off-by: The Anh Nguyen <ntheanh201@gmail.com >
2026-10-09 08:22:18 +00:00
Sriram Kumar
560fae64cd
[ROCm][Bugfix] Pass per-sequence context lengths to the AITER paged MQA-logits kernels ( #59420 )
...
Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com >
2026-10-09 08:16:20 +00:00
Maroon Ayoub and Claude
8a03ccf7b3
[Frontend] Add return_mm_kwargs to render requests ( #60728 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-09 08:14:13 +00:00
f43e4c39e3
[Structured Output] Allow propertyNames with additionalProperties on xgrammar ( #57594 )
...
Signed-off-by: errmakov <ide404@gmail.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-09 10:49:47 +03:00
efaca9547d
[Perf][DSv4.1] Exchange packed FP8 Engram rows across DP and feed wkv MXFP8 ( #59923 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 15:30:18 +08:00
82e89e9944
[Perf][Qwen4Exp] Retune H200 M=4 merged QSA LL-GEMM plan ( #60776 )
...
Signed-off-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-10-09 07:18:50 +00:00
19c0bed584
[Fusion] Support 3D activation quantization fusion and enable it for Pixtral ( #59612 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-09 15:12:03 +08:00
dd30786fef
[CI] Enable Zen CPU image build on AMD CI ( #60016 )
...
Signed-off-by: Chinmay Kulkarni <Chinmay.Kulkarni@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
2026-10-09 01:40:21 -05:00
Paweł Mańczak and Kunshang Ji
f0a5f111f2
[XPU] Add tuned Mamba SSU configs for Intel Arc Pro B60 ( #56765 )
...
Signed-off-by: pmanczak <pawel.manczak@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-09 14:29:29 +08:00
ad73a4740d
[Perf][DSv4.1] Trim the last KV-source layer in decoder replay ( #59894 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Juntian Liu <Juntianl777@gmail.com >
2026-10-09 06:07:49 +00:00
8cbd5d0300
[Frontend] structured decisions: noul and score question types ( #60696 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Co-authored-by: DeepSeek <noreply@deepseek.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-09 04:48:16 +00:00
Rohan Potdar and Claude Opus 5.5
88a419649e
[ROCm][Kimi-K3] Use vLLM's FP8 max in the fused AttnRes quantization ( #60717 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 04:45:13 +00:00
fa312090cb
[Frontend] Accept each connection on the least loaded API worker ( #60723 )
...
Signed-off-by: hao-aaron <aaronhao@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 04:40:13 +00:00
Rajeev Warrier
81d9c45e4b
[Docs] [Multimodal] Note that per-request media_io_kwargs needs --trust-request-mm… ( #60598 )
...
Signed-off-by: Rajeev Warrier <warrier.rajeev.95@gmail.com >
2026-10-09 04:23:07 +00:00
e31b823bf5
[Bugfix][Tool Parser] Use LongCat tags in the LongCat structural tag ( #60747 )
...
Signed-off-by: hpdkhoa <hpdkhoa2311@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 21:20:54 -07:00
9824159d94
[Bugfix] Fix InternLM2 tool parser dropping characters from streamed arguments ( #60709 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 21:20:49 -07:00
Vineeth Sai Varikuntla and Flora Feng
112a282915
[Bugfix][Frontend] Keep streamed completion text_offset in step with tokens ( #60668 )
...
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 21:20:43 -07:00
51ef239768
[Rust Frontend] Carry reasoning controls over gRPC ( #59836 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 11:11:19 +08:00
Rémi Delhaye
a1cd0ec650
[Model] Add zerank-2 reranker ( #60415 )
...
Signed-off-by: rdlhio <contact@rdlh.io >
2026-10-09 03:01:51 +00:00
zzaebok and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
702ce313f4
[Perf][Multimodal] Sort multimodal encoder inputs by modality ( #60196 )
...
Signed-off-by: Jaebok Lee <jaebok9541@naver.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 10:55:36 +08:00
Tiezhen WANG and Kevin H. Luu
295c062db8
Remove sudo from dockerfile command ( #60488 )
...
Signed-off-by: Tiezhen WANG <38108242+xianbaoqian@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-08 19:24:56 -07:00
Mia Ojeda and Lucas Wilkinson
4611c2c3d2
[BugFix][DiffusionGemma][CPU] Copy async output snapshots instead of aliasing reused buffers ( #59829 )
...
Signed-off-by: ojeda-e <eojeda@league.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-08 19:16:51 -07:00
201e537d98
[Bugfix][LoRA] Validate incompatible MoE adapter formats ( #59924 )
...
Signed-off-by: Brian Weston <brianweston@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-09 09:49:13 +08:00
Nishant Revur and Claude
581692cc0f
[Bugfix][Spec Decode] Fix multimodal embedding merge in Llama 4 and Mistral Large 3 EAGLE drafters ( #57633 )
...
Signed-off-by: Nishant Revur <kumrfni@amazon.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-09 01:44:50 +00:00
DGlan
4274ae956c
[Bugfix][EC Connector] Release shared memory on NIXL setup failure ( #59875 )
...
Signed-off-by: DGlan <162426117+DGlan@users.noreply.github.com >
2026-10-08 17:38:16 -07:00
ab905a885d
[CI] Run GH200 test on the shared arm64 CI image ( #60253 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-08 15:22:20 -07:00
Kamran and Codex
8d10c38b73
[Perf] Reduce fallback decode CPU by 69% at 32K ( #60497 )
...
Signed-off-by: spa5k <79936503+spa5k@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
2026-10-08 21:42:56 +00:00
Yifan Qiao and Claude Opus 5.5
240785b82c
[Bugfix] Keep the GLM-5.3 kpool tail and Qwen4 QSA ring out of the null block ( #59528 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 20:44:04 +00:00
sungbin1015 and Flora Feng
2ddde19e8e
[Frontend] Support min_p in the Responses API ( #48084 )
...
Signed-off-by: sungbin1015 <sbin@solbox.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 20:38:17 +00:00
Divakar Verma
c5c1161382
[ROCm][Build] Fail fast when setup.py develop deps aren't pip-preinstalled ( #59447 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
2026-10-08 15:28:38 -05:00
ae9adec905
[Frontend] Expose /v1/systemone without an opt-in flag ( #60677 )
...
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-08 20:21:05 +00:00
72d59adc2c
[Bugfix][KV Connector][NIXL] Transfer the whole PLE short-conv page in disaggregated P/D ( #59997 )
...
Signed-off-by: Alberto Massidda <amassidda@nvidia.com >
Co-authored-by: Alberto Massidda <amassidda@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-08 23:58:13 +04:00
Matthew Bonanni and Claude Opus 5.5
1afb029839
[Bugfix][HiSparse] Route by swap-row capacity, not the index-group workspace ( #60682 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 19:45:44 +00:00
c23ca06b6d
[Bugfix][Structured Output] Feed the token that implicitly ends reasoning to the grammar ( #59627 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Yifan Zong <263738792+yzong-rh@users.noreply.github.com >
2026-10-08 19:38:33 +00:00
dingdangmao and Flora Feng
bab0ac61f8
[Bugfix][Frontend] Forward the reasoning wiring to the engine for batch chat completions ( #59263 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 19:32:46 +00:00
Misha Goin and Claude
e3a29aa9b3
[Bugfix][MoE] Size the permute scratch from its inputs ( #60447 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-08 15:32:07 -04:00
SeongJun Lee and Misha Goin
fff10d2072
[Attention] Allow an NVFP4 KV cache on SM90 with fa2 ( #60620 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 12:16:22 -07:00
Kevin Li and Misha Goin
fc494822ea
[Test][Quantization] Gate CUTLASS W4A8 kernel tests to SM90 ( #58878 )
...
Signed-off-by: Kevin Li <kevinbean99@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 19:14:35 +00:00
Divakar Verma and Andreas Karatzas
18751bc6a7
[ROCm][CI] Add coverage for AiterExperts token padding: inf/nan garbage rows ( #59334 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-08 19:11:45 +00:00
Aarushi Jain and Andreas Karatzas
9bbc581560
[Bugfix] Close MessageQueue zmq resources via finalizer to avoid exit ( #60472 )
...
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-08 13:53:00 -05:00
Yeonwoo Sung and Flora Feng
56e6da2c47
[Bugfix][Frontend] Honor stop_token_ids on the Responses API ( #56592 )
...
Signed-off-by: YeonwooSung <neos960518@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 18:47:30 +00:00
Ning Xie and Simon Mo
c4a5ff5c18
[Frontend] shutdown engine core and its sub processes asap when initializing ( #52299 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
Co-authored-by: Simon Mo <simon.mo@hey.com >
2026-10-08 11:46:01 -07:00
Shijin Zhang
2274e86842
[Spec Decode][Model] Support DFlash2 draft models with GLM-5.3-Flash ( #56983 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
2026-10-08 14:41:10 -04:00
Misha Goin and Claude
30c7405890
[MoE] MXFP4 oracle: add CUTLASS W4A4 backend and BF16 activation fallback ( #60079 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-08 11:39:09 -07:00
d8fa463e2b
[Bugfix][Prefix caching] Fix inconsistent hybrid Mamba prefix cache boundaries ( #60533 )
...
Signed-off-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 18:33:48 +00:00
1664c4f444
[Bugfix][MiniMax-M3] Guard sparse decode sentinel indices ( #54597 )
...
Signed-off-by: Kyle Liang <kylliang@nvidia.com >
Signed-off-by: Kyle Liang <kyleliang-nv@users.noreply.github.com >
Co-authored-by: Kyle Liang <kyleliang-nv@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 18:27:57 +00:00
Tialo and Misha Goin
972195f4f5
[CI][Build] build macos wheels using abi3 ( #60312 )
...
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 14:19:20 -04:00
Francisco Javier Arceo and Misha Goin
705e124c05
[Bugfix] Followup to #59299 : preserve Unicode in structured decision state prompts ( #60651 )
...
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 11:00:03 -07:00
Doug Smith
242e4213fc
[Doc] Require GCC 13 for CUDA source builds ( #60259 )
...
Signed-off-by: Doug Smith <dosmith@redhat.com >
2026-10-08 09:43:39 -07:00
Elvir Crnčević and Misha Goin
8674a794c7
[Perf] Enable Kimi K3 shared expert sharding with DeepEP v2 ( #56875 )
...
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 16:42:15 +00:00
stefankoncarevic
f4dde3132c
[CI][ROCm] Reuse AITER's prebuilt MoE module in the modular-kernel harness ( #60633 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
2026-10-08 11:18:05 -05:00
9f3d70e081
[Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) ( #55686 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: fxmarty-amd <felmarty@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 12:04:46 -04:00
Mikko Tukiainen and Cursor
e2414ec1db
[ROCm][Qwen4Exp] Use fused PLE Triton kernels on the AMD backend ( #60021 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-08 15:47:56 +00:00
Justin Barlow and Harry Mellor
0fc0cefb4e
[Bugfix] Don't int-sort attention-type names in --kv-cache-dtype-skip-layers for packed KV dtypes ( #60364 )
...
Signed-off-by: lobo235 <7@netlobo.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-08 16:34:55 +01:00
e41d5d5543
[Bugfix] Don't warn about missing watermark config for default requests ( #60412 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-08 15:29:20 +00:00
Haokai Ma and Claude
0751f5272f
[Bugfix] Scope deprecated CLI argument warnings to the parser that defines them ( #59917 )
...
Signed-off-by: Tyler Ma <158780265+TylerMa-debugg@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-08 15:29:04 +00:00
Woosuk Kwon
194bd72be7
[Compile] Remove torch.compile-based sequence parallelism and AsyncTP ( #60517 )
2026-10-08 08:28:23 -07:00
wangxiyuan and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
072d49bb6e
[Platform] Support platform-specific MLA prefill backend selection ( #56959 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 15:25:21 +00:00
fd928e5f3c
[Bugfix] Check batch-invariance support for every attention backend ( #60271 )
...
Signed-off-by: yifanFengg <evanfengff@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-08 11:19:57 -04:00
JartX
994b8b77be
[Bugfix][ROCm] RDNA3 W4A16 MoE: handle the N-first CT weight layout ( #60388 )
...
Signed-off-by: JartX <sagformas@epdcenter.es >
2026-10-08 15:06:39 +00:00
kliuae
d80ddaac73
[ROCm][Perf] Kimi-K3 Store only the current rank's shards in latent MoE up-proj ( #59591 )
...
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com >
2026-10-08 14:55:20 +00:00
d33872dfeb
[Feat] Structured decisions endpoint (/v1/systemone) ( #59299 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-08 14:48:12 +00:00
yhcheong (vllmellm)
273e099693
[Bugfix][ROCm] Select page-aligned kernel blocks for pooled indexers so the block table addresses storage pages (GLM-5.3-Flash) ( #59412 )
...
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com >
2026-10-08 14:44:58 +00:00
zzaebok
2f7f05c256
[Bugfix][MoRIIO] Prevent remote KV reload after decoder preemption ( #53250 )
...
Signed-off-by: Jaebok Lee <jaebok9541@naver.com >
2026-10-08 22:28:16 +08:00
daa9085143
[KV Connector][NIXL] Register HiSparse host pool under single TP Rank ( #60161 )
...
Signed-off-by: Pravein Govindan Kannan <pravein.govindan.kannan@ibm.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 07:25:02 -07:00
Matthew Bonanni and Claude Opus 5.5
4ed4c4433b
[Bugfix][HiSparse] Drop the unaccounted, write-only prefill mirror staging buffer ( #60429 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 14:00:49 +00:00
Vineeth Sai Varikuntla and Harry Mellor
194da61de1
[Bugfix] Reject negative --device-ids indices instead of selecting the wrong device ( #49053 )
...
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-08 13:01:25 +00:00
458ba2edf8
[ROCm][Kimi-K3] Fuse AttnRes output with per-token FP8 input quantization ( #59069 )
...
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
2026-10-08 12:01:47 +00:00
73c742b09c
[Bugfix] Make disable_any_whitespace actually disable whitespace on xgrammar ( #58067 )
...
Signed-off-by: Stu Cao <stu@primeintellect.ai >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-08 14:41:14 +03:00
devshah-cohere and Bugen Zhao
b49c921a27
[Bugfix][Rust Frontend] Always include token_ids in completion stream choices ( #60286 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: devshah-cohere <dev.shah@cohere.com >
2026-10-08 19:11:55 +08:00
Andy Lo
ba77c4c130
[Bugfix][Attention] Preserve small FP8 softmax weights in Triton attention ( #60156 )
...
Signed-off-by: Andy Lo <andy@mistral.ai >
2026-10-08 12:35:22 +02:00
Nicolò Lucchesi
e161a0798c
[Perf][Core] Reduce free KV cache queue overhead ( #60269 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-08 10:17:11 +00:00
wang.yuqi
459764104d
[Frontend] Move offline_utils.py to entrypoints/common. ( #58052 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-08 09:56:05 +00:00
Muhammad Usman Mateen and Misha Goin
e73895a5df
[Bugfix] Make suppress_stdout redirect fd 1 instead of sys.stdout.fileno() ( #59336 )
...
Signed-off-by: usmanmateen <usmanmateen255@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 09:52:50 +00:00
Chaojun Zhang
dfaabf91ac
[XPU] [CI] Fix stale ignore path for weight_transfer_metrics tests on Intel CI ( #60558 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
2026-10-08 17:48:47 +08:00
Kevin H. Luu and Claude Opus 5.5
24be8c0bca
[CI] Remove dead in-tree TPU (torch_xla) CI scripts ( #60591 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:42:07 -07:00
Jiangyun Zhu and Claude Opus 5.5
e265b77982
[CI][Mamba] Use well-conditioned prompts for FlashInfer ReplaySSM parity tests ( #60515 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 17:39:56 +08:00
7816afdb25
[Bugfix][Core] Drain async KV loads in pause(mode="wait"), and pause the drain test only once a request is in flight ( #60572 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 09:33:41 +00:00
Kevin H. Luu and Claude Opus 5.5
c4cd88d9fe
[CI] Make (H100) Distributed DP + EP optional ( #60540 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:31:13 -07:00
Harry Mellor and Claude Opus 5.5
21d9758459
Remove Python 3.10 support now that it is EOL ( #60402 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:30:52 -07:00
liuzhenwei and Kunshang Ji
0004536dd0
[XPU][Graph] Replace boolean-mask assignment with torch.where ( #60529 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-08 17:13:48 +08:00
Nicolò Lucchesi
70bdce6133
[Model][GLM-5.3-Flash] Warm up the DCP top-k merge kernels for the kpool indexer ( #60032 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-08 11:05:52 +02:00
7d47ac2ddb
[CI] Enroll five steps in automatic sharding ( #60492 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: khluu <khluu000@gmail.com >
2026-10-08 01:52:59 -07:00
Paweł Mańczak
4cf1307e72
[XPU] Add tuned Mamba SSU configs for 2 more B70 shapes ( #57565 )
...
Signed-off-by: pmanczak <pawel.manczak@intel.com >
2026-10-08 16:36:05 +08:00
Ting SUN
0f112d1800
[Bugfix][Pooling] Preserve cache_salt for chunked embeddings ( #47696 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-10-08 07:33:28 +00:00
Alec
29ae81fe83
[Rust Frontend] Preserve explicit zero sampling values in gRPC requests ( #60115 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
2026-10-08 07:27:47 +00:00
Rehan Khan and Bob
fd0216d3b5
[Bugfix][CPU] Fix fused Gumbel-max wrapping-window bias ( #60221 )
...
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com >
Co-authored-by: Bob <bob@ibm.com >
2026-10-08 15:15:53 +08:00
ARAVINDHAN T and linitra24
e58f8dde7f
[Bugfix] Use FusedMoEWithLoRA for shared MoE LoRAs on 3D models ( #60098 )
...
Signed-off-by: ARAVINDHAN T <arvindhant01@gmail.com >
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-08 15:04:32 +08:00
xiangdong
032d39e9b4
[XPU][CI] Split Intel jobs to improve B50 utilization ( #58817 )
...
Signed-off-by: zengxian <xiangdong.zeng@intel.com >
Signed-off-by: xiangdong <40376367+zxd1997066@users.noreply.github.com >
2026-10-08 14:53:37 +08:00
stefankoncarevic and Andreas Karatzas
814301b268
[Bugfix][Quant] Read safetensors headers of checkpoints with other file names ( #60411 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-08 01:38:41 -05:00
Vadim Gimpelson and Claude
ee406adf0c
[Bugfix] Fix ZeroDivisionError in Qwen3-VL video sampling when the source fps is unknown ( #60501 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-08 06:28:44 +00:00
f226af0e11
[Bugfix] Eliminate frontend ZMQ port TOCTOU with inherited listeners ( #54113 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 06:25:27 +00:00
fb2ac82458
[ROCm][DSv4.1][Perf] Replace the Torch prefill combine with the shared combine_topk_swa_indices kernel ( #56638 )
...
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Ren Yuzhou <54501155+yuzhouo7@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-08 14:15:18 +08:00
Shanshan Shen
95307ac6d4
[Doc] Update ViT CUDA graph design doc ( #60519 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-10-08 14:00:58 +08:00
Jiahan Chang (Cyrus)
816173ca9d
[Kernel] Remove WAR for LL CuTeDSL BF16 GEMM ( #60434 )
...
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com >
2026-10-08 05:38:07 +00:00
81198e97ba
[Refactor] Abstract DeepGemm MegaMOE backend ( #58053 )
...
Signed-off-by: Hui Shi <kalinshi@tencent.com >
Co-authored-by: LuneExplorer <liyihan23@mails.tsinghua.edu.cn >
Co-authored-by: ArmageddonKnight <bojian@cs.toronto.edu >
2026-10-08 05:05:39 +00:00
4ac0179f8a
[ROCm][Perf] Use native merge_attn_states kernel on gfx942/gfx950 ( #60159 )
...
Signed-off-by: mghanimi <mghanimi@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-08 04:49:49 +00:00
775792f2d5
[Bugfix] Reland #53020 : tie lm_head.weight for Nemotron Parse when checkpoint omits it ( #60272 )
...
Signed-off-by: Anish Kumar <aniskumar@nvidia.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-08 04:40:55 +00:00
4187ade7b8
[Frontend] Consolidate rl entrypoints. ( #57849 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 21:37:11 -07:00
1fbbfa1657
[Bugfix][Kimi-K3] Declare max_tp_shards on the DSpark MLA KV cache spec ( #59733 )
...
Signed-off-by: hyukjlee <hyukjlee@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
2026-10-08 00:33:32 -04:00
Matt and Claude Opus 5.5
5dd4a628dd
[ROCm][Bugfix] Map sleep-mode cuMem memory for the host ( #60367 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-08 04:26:12 +00:00
Isotr0py and Shanshan Shen
8352b2427f
[Multimodal] Optimize encoder cudagraph padding logic ( #58925 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-08 03:38:09 +00:00
Simon Danielsson and Douglas Lehr
87d9996abe
[ROCm][Perf][GLM-5.3-Flash] BF16 splitk kernel for sparse MLA decode ( #58584 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-07 21:37:51 -05:00
bnkaiteam and Flora Feng
3722645539
[Bugfix][Frontend] Accept successful 2xx responses for batch output uploads ( #60507 )
...
Signed-off-by: bnkaiteam <bnkaiteam@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 02:32:35 +00:00
caff2b44cb
[BugFix] Pick the DP world-group port at bind time via the coordination store ( #51018 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-08 02:30:35 +00:00
wangxiyuan and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a4b14e57b6
[Platform] Allow BackendEnum dynamic register ( #41091 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 09:56:19 +08:00
Matt and Claude Opus 5.5
9dba339d07
[Bugfix][ROCm] Keep MLA derived weights in place on weight reload ( #60389 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-07 20:38:06 -05:00
Nicolò Lucchesi
8733a21d55
[Core] Bound execute_dummy_batch RPC waits by the execute-model timeout ( #60398 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-08 01:22:04 +00:00
Junyi Yao and sfeng33
fdfc171a74
[Bugfix][ToolParser] Stream every tool call when one delta completes several ( #60354 )
...
Signed-off-by: Junyi Yao <j.yao@wustl.edu >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-08 01:07:35 +00:00
2dd887426e
[Bugfix][Frontend] Do not modify caller params in LLM.score() ( #59959 )
...
Signed-off-by: 叮当猫 <2756823972@qq.com >
Signed-off-by: apex-mochen <2756823972@qq.com >
Co-authored-by: DeepSeek Harness <noreply@deepseek.com >
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-08 09:03:08 +08:00
9e3e37cb3c
[TEST][XPU][CI] disable tests of unsupported quantization ( #60383 )
...
Signed-off-by: microslaw <you@example.com >
Co-authored-by: microslaw <you@example.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-08 08:56:28 +08:00
Kang Nahoon and Flora Feng
1995c0fcd0
[Bugfix][Frontend] Apply output_config and thinking in Anthropic count_tokens ( #59180 )
...
Signed-off-by: Kang Nahoon <ejrtks102020@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 00:24:20 +00:00
Kevin H. Luu and Sherlock
1283362e76
[CI] Add headroom and selection coverage for H200 fast lanes ( #56775 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: Sherlock <sherlock@raft.ai >
2026-10-07 17:17:56 -07:00
Nicolò Lucchesi
a58bdd0d4e
[Perf] JIT monitor: hook TileLang JITImpl.compile instead of __call__ ( #58685 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-08 00:11:59 +00:00
8faac3dd2a
[Bugfix] [Spec Decode] DeepSeek V4/V4.1 NVFP4: keep ignored MTP/DSpark experts on MXFP4 ( #59655 )
...
Signed-off-by: Shobhit Verma <shobhitv@nvidia.com >
Signed-off-by: v-shobhit <161510941+v-shobhit@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
2026-10-08 00:10:32 +00:00
YueshenZ and sfeng33
ed825aafc0
[Bugfix][Frontend] Fix Responses namespace aliases in named tool calling ( #59723 )
...
Signed-off-by: Shen <312058264+YueshenZ@users.noreply.github.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-08 00:07:52 +00:00
www6v and sfeng33
38dc8ee500
[Bugfix] Validate tool-parser/tokenizer compatibility at startup ( #59749 )
...
Signed-off-by: www6v <www6v@126.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-07 23:46:41 +00:00
Dhairyashil R G and sfeng33
d6fe5dca68
[Bugfix][Parser] Emit buffered post-reasoning text when no tool parser is configured ( #58911 )
...
Signed-off-by: Dhairyashil R G <18738735+dhairyashilRG@users.noreply.github.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-07 23:21:51 +00:00
Artem Perevedentsev
4ac5a36a8e
[CI/Build][NVIDIA] Build NIXL with NIXL EP from source in Rubin image ( #60090 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-08 02:44:17 +04:00
ef63c23d35
[Bugfix][DSv4.1] Skip SWA bounded replay for KV loads that carry the window ( #59197 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 15:28:42 -07:00
Nicolò Lucchesi and Mistral Vibe
d169c5a903
[Docs] Add disagg PD flow-example guidance to /pr-checklist skill ( #60111 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Mistral Vibe <noreply@mistral.ai >
2026-10-07 22:16:05 +00:00
f38c6778ce
[quantization] Add per-token NVFP4 quantization MoE support for ReLU2 ( #56740 )
...
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-07 22:01:09 +00:00
b803edbc9f
[Model] Streamline EmbeddingGemma 2 config resolution and harmonize Triton prefill attention ( #60289 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-07 14:56:40 -07:00
Simon Danielsson
947dd62c07
[CI] Fix failing test_deepseek_v4_rocm_dspark* tests ( #60462 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-10-07 14:18:19 -07:00
Thang Nguyen and Kimi
b47ec440bc
[CI] Stamp provenance labels on the torch-nightly image so Initialized Snapshot E2E can run ( #59487 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-10-07 14:15:39 -07:00
Misha Goin and Claude Opus 5.5
a982c81e3e
[Bugfix] Limit GlmMoeDsaForCausalLM fp8 KV cache default to SM100 ( #60281 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 13:37:13 -07:00
Rui
08567505b0
[Bugfix][KV Offload] Restore KVCR construction through the secondary-tier factory ( #58088 )
2026-10-07 21:36:57 +02:00
e572e02d21
[UX] Skip checkpoint shards an MTP head does not load ( #60152 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 15:16:19 -04:00
Moein Khazraee
17c70c8953
[KV Offload] Update KVCR adapter to pool-and-index API ( #59899 )
2026-10-07 21:14:07 +02:00
b9767299db
[Bugfix] Respect Inductor deterministic mode in combo-kernel defaults ( #59074 )
...
Signed-off-by: cheesecake <farzanaman99@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 20:08:11 +01:00
Jinzhen Lin and Kevin H. Luu
32fbfa15e8
[Bugfix][Quantization] Accept folded NVFP4 scales for Humming A16 ( #60337 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-07 11:04:43 -07:00
Giancarlo Delfin
6ee9be6b3c
[Spec Decode] Rename AR speculators to StandaloneAR / TargetDependentAR ( #60335 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-10-07 14:01:59 -04:00
fba462e008
[Bugfix]Fix FlashInfer warmup crash: autotune(tuning_buckets=...) must round M up, not down (Invalid MXFP8 split-K tactic with spec-decode) ( #58165 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 14:01:21 -04:00
554340f3d3
[Attention] Support NVFP4 KV cache on SM8x and SM12x with FlashInfer ( #46963 )
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-07 13:24:21 -04:00
3d6853b32c
[Bugfix] Seed hybrid mamba state index with mamba_block_size on prefix-cache hits ( #55601 )
...
Signed-off-by: pondzikk <pondzik@gmail.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: ptorsten <1652174+ptorsten@users.noreply.github.com >
2026-10-07 17:09:50 +00:00
Harry Mellor and Claude Opus 5.5
a8260cbc5c
[CI] Bump Transformers version to 5.19.0 ( #60381 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 10:01:41 -07:00
d64a18832b
[ROCm][Triton] Migrating MiniMax-M3 kernels from make_block_ptr to tensor descriptors ( #59285 )
...
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Co-authored-by: jpvillam <juan.villamizar@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Rohan Potdar <rohan.potdar@amd.com >
Co-authored-by: Daniel <danichan@amd.com >
2026-10-07 11:49:59 -05:00
MohitAMD and Cursor
240ee8de9e
[Bugfix][KV Connector][ROCm] Fix MoRI-IO KV block-offset for speculative decoding (MTP) ( #55053 )
...
Signed-off-by: Mohit Deopujari <Mohit.Deopujari@amd.com >
Signed-off-by: MohitAMD <mohit.deopujari@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-07 11:33:30 -05:00
Andy Lo
c741bfca70
[Docs] Add review areas for andylolu2 ( #60416 )
...
Signed-off-by: Andy Lo <andy@mistral.ai >
2026-10-07 16:05:29 +00:00
pmanczak
f7f62d1916
[XPU][MoE] Tune Triton fused MoE for Intel XPU ( #53065 )
2026-10-08 00:02:13 +08:00
xaguilar-amd and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
9367d8b969
[Bugfix][MLA] Scope DSpark non-causal capability to builder layers ( #57992 )
...
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-07 10:22:32 -05:00
stefankoncarevic
71f9a74920
[CI][ROCm] Re-enable the DSv4.1 decoder replay graph test on ROCm ( #60392 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-07 09:57:52 -05:00
Alec
14e3902d51
[Frontend][Rust] Add gRPC forbidden token sequences and cache usage ( #59837 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
2026-10-07 07:46:44 -07:00
Mikko Tukiainen and Cursor
548597367e
[ROCm][Perf] Add AITER FlyDSL GDN prefill backend ( #57560 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-07 09:44:22 -05:00
Micah Williamson and Fangzhou Ai
f6e3944e31
[ROCm][DSv4] AITER MegaMoEV2 Integration For DeepSeek V4 ( #59685 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
2026-10-07 09:35:53 -05:00
c83935b300
[Bugfix][Model Loader] Support reload_weights with runai_streamer load format ( #58124 )
...
Signed-off-by: Viktor Koukouliev <vkoukouliev@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-07 14:04:35 +00:00
00CC
5281e49908
[ROCm][Perf] Enable medium-skinny dispatch in RDNAHybridW4A16LinearKernel ( #52619 )
...
Signed-off-by: tangzzycc <3081129260@qq.com >
2026-10-07 13:21:43 +00:00
Divakar Verma and Andreas Karatzas
efb8d85f24
[ROCm][CI] Add FP32 rounding slack to accuracy test near-tie upper bounds ( #60292 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-07 08:19:22 -05:00
2a54f6b625
[ROCm][Perf][GLM-5.3-Flash] Enable AITER fused shared experts on gfx950 ( #59221 )
...
Signed-off-by: Jin Tao <jin.tao@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
2026-10-07 20:53:40 +08:00
d9503dc503
[Bugfix][Qwen4Exp] Derive the QSA head_dim from the unsharded head count ( #59945 )
...
Signed-off-by: lesj0610 <lesj0610@godoiksan.org >
Co-authored-by: lesj0610 <lesj0610@godoiksan.org >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-07 12:24:40 +00:00
stefankoncarevic
3ca00a8261
[CI] Skip EmbeddingGemma2 pooling tests needing transformers >= 5.19.0 ( #60371 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
2026-10-07 10:22:31 +00:00
SeongJun Lee
6b75dfb8a1
[Misc] Report a missing DeepSelect extension only when it is requested ( #60242 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
2026-10-07 09:53:17 +00:00
Kevin H. Luu and Claude Opus 5.5
02d8a94e61
[CI][ROCm] Skip DSv4.1 decoder replay graph test on ROCm ( #60378 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 02:38:31 -07:00
df417f780a
[Bugfix][MoRIIO] Exclude synchronous READ destinations from KV zeroing ( #59164 )
...
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com >
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
Signed-off-by: Hexiang Wang <56632993+whx-sjtu@users.noreply.github.com >
Co-authored-by: Yichao Zhu <Yichao.Zhu@amd.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-07 16:35:56 +08:00
42ecc8eebe
[CI][RL] Consolidate RL entrypoint tests under tests/entrypoints/rl ( #59948 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 08:29:09 +00:00
Bugen Zhao and Claude Opus 5.5
bc8dd4ad27
[Rust Frontend] Share SchemaRoot between argument coercion and grammars ( #59408 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 08:28:27 +00:00
stefankoncarevic
90f4936d9e
[CI/Build] Widen the ragged prefill scoring tolerance, drop batch invariance ( #60192 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-07 16:21:47 +08:00
Bugen Zhao and Claude Opus 5.5
7436a7f119
[Rust Frontend] Backtrack from safe text at a complete marker ( #59563 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 07:45:15 +00:00
e6fc81bc78
[Bugfix][Scheduler] Let pooling chunked prefill use full context ( #48039 )
...
Signed-off-by: Tommy <tran.tommy@hotmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-07 07:32:00 +00:00
cbee82324c
[Docs][RL] Move pause/resume and batch invariance examples to examples/rl ( #59949 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 06:22:10 +00:00
Sriraj and Claude
43b4aaea3e
[Bugfix][Frontend] Return 4xx for client errors in /cohere/v2/chat ( #60309 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-07 04:55:34 +00:00
Turner Jabbour and Claude Sonnet 5
aecb717f10
[CI] Allowlist-shrink batch 2: wire 9 tests/models/ files into Buildkite, fix 2 stale bugs found ( #59050 )
...
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-10-06 21:16:53 -07:00
Mohammad Hijjawi and Cyrus Leung
ef014bdf06
[Bugfix][Frontend] Return logprobs in Cohere Chat v2 responses ( #59072 )
...
Signed-off-by: Mohammad Hijjawi <mohammad.hijjawi1997@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-07 04:13:52 +00:00
Hyukjoon Lee and Claude Opus 5.5
2342343746
[ROCm][DCP] Default MLA DCP verify to the round-robin asm decode where it applies ( #59965 )
...
Signed-off-by: hyukjlee <hyukjlee@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 03:10:54 +00:00
8a26869bca
[Bugfix] Index Mamba align state copies by request slot in MRV2 ( #60210 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Karl0007 <44745209+Karl0007@users.noreply.github.com >
Co-authored-by: Yongqin Wang <yongqinwang@roblox.com >
2026-10-07 02:32:46 +00:00
956960f85f
[Perf][MRV2] Avoid stream syncs in NonUvaBuffer host-to-device copies ( #60215 )
...
Signed-off-by: cheesecake <farzanaman99@gmail.com >
Co-authored-by: William Joyner <133391209+joynerwk03@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 01:51:12 +00:00
Robert Shaw and Claude Code
283d76d56a
[Bugfix] Widen ep_gather output offset to int64 ( #58373 )
...
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-10-07 01:45:04 +00:00
fe51194e2d
[Bugfix][MiniMax-M3] Fence decode indexer shared-memory stage reuse ( #60194 )
...
Signed-off-by: Vladimir Anisimov <vanisimov@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-07 01:35:12 +00:00
Kevin H. Luu and Claude Opus 5.5
4b4eb6277a
[CI] Remove AMD mirror of CPU-only Rust frontend cargo steps ( #60323 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 18:30:52 -07:00
Micah Williamson and Kevin H. Luu
13f8c453ee
[CI][Releases] Trigger perf-eval Through Release Pipeline ( #52213 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-06 18:27:27 -07:00
Miłosz Grunwald
703c29179c
[TEST][XPU][CI] Generalize model runner prefill tail classification tests to all platforms ( #60201 )
...
Signed-off-by: microslaw <milosz.grunwald@intel.com >
2026-10-07 08:59:20 +08:00
Sriraj and Flora Feng
db6e3cd8c4
[Bugfix][Frontend] Log the full request body when max_log_len is unset ( #60310 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-07 00:42:17 +00:00
jxj and OpenAI Codex
6c2925fe5f
[Bugfix][Core] Preserve speculative metrics when coalescing outputs ( #55708 )
...
Signed-off-by: git-jxj <65210887+git-jxj@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-07 08:35:14 +08:00
Sriraj and Flora Feng
d77cf81156
[Bugfix][Frontend] Honor default penalties in batch chat completions ( #60307 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-07 00:21:00 +00:00
5ab4e4445b
[Perf][MoE] Skip padded work in block-FP8 DeepGEMM experts ( #59128 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-06 18:49:57 -05:00
Cheese Cake and Claude Opus 5.5
b6d8e8afd9
[Bugfix][MRV2][Spec Decode] Accept num_speculative_tokens in NgramGPUSpeculator.propose ( #60300 )
...
Signed-off-by: cheesecake <farzanaman99@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 15:02:04 -07:00
ae19af883a
[CI] Require transformers 5.19 for the EmbeddingGemma2 registry test ( #60295 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-06 14:50:10 -07:00
aditi-amd
385d86a6cf
[Attention] Add UltraQuant 4-bit KV cache backend (FlyDSL D=256) ( #57057 )
...
Signed-off-by: Aditi Rana <aditi.rana@amd.com >
2026-10-06 20:42:52 +00:00
819852df6b
[ModelRunner V2] Speculative Decoding NGram GPU Implementations ( #40704 )
...
Signed-off-by: PatchouliTaisa <patchychen@tencent.com >
Signed-off-by: PatchouliTaisa <pyramkar@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Signed-off-by: Xuanan Chen <xuananchenc@nvidia.com >
Co-authored-by: PatchouliTaisa <patchychen@tencent.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Xuanan Chen <xuananchenc@nvidia.com >
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com >
2026-10-06 13:32:59 -07:00
Benjamin Chislett and Claude Opus 5.5
dac35c5b66
[Misc] Add chat-completion options to gsm8k_eval.py CLI ( #60114 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 20:13:44 +00:00
PatrykSaffer and Misha Goin
e43db1f5e2
Pad to 64 not 128 Q heads in flashMLA sparse ( #60029 )
...
Signed-off-by: Patryk Saffer <patryk.saffer@mistral.ai >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-06 16:03:05 -04:00
d1f3d8b870
[Quantization] Prefer Humming before Marlin backends on SM90 ( #56997 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 12:31:35 -07:00
Roger Wang and Kimi Code
bb87d227d4
[Bugfix][Model] Fix mypy safe-super error in EmbeddingGemma2Model ( #60280 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-10-06 15:10:00 -04:00
e37e51dd24
[Metrics] Expose cached prompt tokens by cache tier ( #56318 )
...
Signed-off-by: Cam Quilici <cjquilici@gmail.com >
Signed-off-by: Cam Quilici <cameron@semianalysis.com >
Co-authored-by: Cam Quilici <cameron@semianalysis.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-06 14:52:05 -04:00
Nishant Revur and Claude
ad0f67a7ec
[Bugfix][Spec Decode] Keep heterogeneous-vocab draft models on Model Runner V1 ( #59541 )
...
Signed-off-by: Nishant Revur <kumrfni@amazon.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-06 18:03:44 +00:00
3403e0f176
[Bugfix] Persist FlashInfer autotune cache per rank to fix the rank-0-only cache-hit deadlock ( #57635 )
...
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com >
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-06 17:51:01 +00:00
Luciano Martins and Luciano Martins
02b83919aa
[Model] Support EmbeddingGemma2 multimodal pooling architecture ( #60254 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
2026-10-06 13:43:38 -04:00
7b665f7a58
[Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) ( #58181 )
...
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-06 17:22:47 +00:00
Matthew Bonanni and Claude Opus 5.5
e82b80099a
[Perf][HiSparse] Cache per-request residency instead of rescanning every page each step ( #60083 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 13:04:55 -04:00
Wentao Ye
fba31f31a9
[GLM5.3 Perf] Switch to fp8 kv cache by default for GLM, 2.3%~5.5% E2E Throughput Improvement ( #60140 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-06 12:54:20 -04:00
f8d92cae41
[Quantization] Layer re-quantization for linear layers through online quantization API (MXFP8 -> FP8 PTPC showcase) ( #55684 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com >
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com >
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-06 11:38:18 -05:00
eky-amd and Claude Fable 5
29f955cce0
[ROCm] Fuse MLA dual RMSNorm + FP8 group quant for DeepSeek-R1 ( #54857 )
...
Signed-off-by: Ethan Ky <ethan.ky@amd.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-10-06 11:26:38 -05:00
stefankoncarevic
4516896626
[CI/Build] Warm the DP engines before measuring load balance in test_load ( #60241 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-06 11:23:37 -05:00
Giancarlo Delfin
fcf53d240c
[Spec Decode] Remove eager metadata rebuild during MTP fused multi-step decode ( #58463 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-10-06 12:17:30 -04:00
SeongJun Lee
4ea0c28bc4
[Bugfix] Warn only on per-request do_normalize/do_rescale overrides ( #60240 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
2026-10-06 15:55:27 +00:00
Jeff-Tseng-dev
aad4579bbe
Upgrade tpu-inference to v0.31.0 ( #60190 )
2026-10-06 15:44:56 +00:00
Chris Leonard and Shengqi Chen
cc73cca9c8
[Build] Migrate vendored DeepGEMM from pybind to TORCH_LIBRARY (abi3) ( #48962 )
...
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-06 08:39:04 -07:00
ovidiusm
cbe1f9740d
[NIXL] Bump NIXL version to 1.5.0 ( #56907 )
...
Signed-off-by: Ovidiu Mara <ovidium@nvidia.com >
2026-10-06 16:55:03 +02:00
Bugen Zhao and Claude Opus 5.5
0719975cee
[CI][Rust Frontend] Pass --locked to cargo binstall ( #60182 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 22:53:17 +08:00
8bd7737a03
[Docs] Clarify that vLLM does not isolate tenants sharing a server ( #59811 )
...
Signed-off-by: SunnyR <sunny.rangnani@chainguard.dev >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Juan Pérez de Algaba <124347725+jperezdealgaba@users.noreply.github.com >
2026-10-06 22:39:59 +08:00
Samuel Nordmann and Misha Goin
4a99ca40d6
[Perf][MoE] Eliminate staging from NCCL symmetric reduce-scatter ( #49194 )
...
Signed-off-by: samnordmann <snordmann@nvidia.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-06 07:37:01 -07:00
Jiangyun Zhu and Claude Opus 5.5
bf049ddb8f
[Docs] Run pr-checklist before opening a PR ( #60235 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 14:12:18 +00:00
Matthias Gehre
b267fe4881
[ROCm][Perf] W4A16: pad gfx11 weight and activation strides ( #56301 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
2026-10-06 08:22:33 -05:00
Artem Perevedentsev and Shashank Varma
049507aa76
[Bugfix][Structured Output] Flag multi-branch allOf as unsupported for xgrammar ( #59061 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Shashank Varma <324153016+shashankvarma499@users.noreply.github.com >
2026-10-06 17:01:04 +04:00
Roy Wang
1a86733c7b
[Docs] Add model recipes skill ( #57786 )
2026-10-06 20:40:10 +08:00
Rodrigo Garcia
3faf214051
[vllm-bench, feature] Added support for Mooncake-style, timed-traces replay. ( #55937 )
2026-10-06 20:39:35 +08:00
Sherif Waly
25385102fe
[Bugfix][KV Offload] Honor speculative cacheability in SimpleCPU ( #60071 )
...
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai >
2026-10-06 14:25:08 +02:00
Wentao Ye
3e182185aa
[Deprecation] Deprecate sonet dataset as scheduled ( #60129 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-06 08:15:12 -04:00
Mia Ojeda
3313825983
[Bugfix][DiffusionGemma] Read the causal mask as int32 and compile the sample step ( #59992 )
...
Signed-off-by: ojeda-e <eojeda@league.com >
2026-10-06 19:25:51 +08:00
09e0ce3d82
[Bugfix][Frontend] Apply model-default reasoning parser in GPU-less render server ( #54835 )
...
Signed-off-by: Shimi Bandiel <shimib@google.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-06 18:46:24 +08:00
zhangchengzhucufe-dev
68088ed392
Fix grammar in BlockPool.reset_prefix_cache docstring: "to invalid" → "to invalidate" ( #59886 )
...
Signed-off-by: zhangchengzhucufe-dev <zhangchengzhucufe@gmail.com >
2026-10-06 10:19:54 +00:00
Louie Tsai
888074b42d
[CPU][Recipes] Detect model head constraints for automatic tensor parallel selection in vLLM Recipes Tool ( #60170 )
...
Signed-off-by: louie-tsai <louie.tsai@intel.com >
2026-10-06 18:15:40 +08:00
Bugen Zhao and Claude Opus 5.5
6bbad6acdf
[Rust Frontend] Pass tool defer_loading through to chat templates ( #60203 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-10-06 18:00:15 +08:00
aoshen02 and Claude Opus 5.5
31e2443c90
[Model] LongCat-Flash: scale MLA norms while loading and drop the post-load sweep ( #60080 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 08:47:02 +00:00
roikoren755 and zjy0516
e11962fc1c
[Feat][Mamba2] Enable internal prefill checkpoints ( #57329 )
...
Signed-off-by: Roi Koren <roik@nvidia.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-06 07:53:51 +00:00
edfccc9469
[ROCM][Perf] Add ROCM_SEGMENTED_ATTN attention-backend works across RDNA3/3.5/4 ( #59132 )
...
Signed-off-by: root <jeffaw99@hotmail.com >
Signed-off-by: big-yellow-duck <jeffaw99@hotmail.com >
Co-authored-by: Tan Pin Siang <1716735+tanpinsiang@users.noreply.github.com >
Co-authored-by: vllmellm <190700713+vllmellm@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-06 07:53:47 +00:00
15c1cc12ac
[Bugfix][EPLB] Use group-local ranks for torch P2P transfers ( #55804 )
...
Signed-off-by: LostFox11 <wangziyue17@huawei.com >
Co-authored-by: LostFox11 <wangziyue17@huawei.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: YangShuai52 <178856220+YangShuai52@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-06 07:44:57 +00:00
Tarun Kumar
6ad2e06c4a
fix: exclude flashinfer-jit-cache from wheel install_requires ( #60052 )
...
Signed-off-by: Tarun Kumar <takumar@redhat.com >
2026-10-06 06:31:13 +00:00
sungbin1015 and Claude
1e5d0ea888
[CI/Build] Fix incorrect CLI arg validation tests in test_cli_args.py ( #46779 )
...
Signed-off-by: sungbin1015 <sbin@solbox.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-06 05:58:47 +00:00
40c86ae1ce
[Kernel] Enable OAI Triton MXFP4 MoE on SM12x ( #58877 )
...
Signed-off-by: lazymio <mio@lazym.io >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-06 01:49:13 -04:00
Bugen Zhao and Claude Opus 5.5
b3082d3044
[Rust Frontend] Remove the PyO3 tool-parser bridge ( #59744 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 05:46:27 +00:00
Bugen Zhao and Claude Opus 5.5
5145e35e50
[Frontend] Port the MiniMax M3 tool parser to the parser engine ( #59743 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 05:46:27 +00:00
czhu-cohere and aoshen02
6fddc9b21e
[Profiler] Add per-session profiling controls to Python and Rust frontends ( #57875 )
...
Signed-off-by: czhu <conway.zhu@cohere.com >
Signed-off-by: Conway Zhu <conway.zhu@cohere.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-06 05:21:19 +00:00
aoshen02 and Claude Opus 5.5
9a1f6fd4cb
[Model][DeepSeek-V4.1] Permute mega-attention wq_b / wo_a while loading ( #60064 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 05:20:09 +00:00
Chinmay-Kulkarni-AMD and Cursor Agent
533d81b4b0
[ZenCPU][Test] Add zentorch/torch version pin consistency test ( #52888 )
...
Signed-off-by: Chinmay Kulkarni <Chinmay.Kulkarni@amd.com >
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2026-10-06 13:12:19 +08:00
eb2901b735
[Perf][DSv4.1] Run decoder replay layers in CUDA graphs and trim in PIECEWISE graphs ( #59532 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 22:11:45 -07:00
21d93d0d8c
[Attention][Model] FP8 KV cache for Triton DiffKV and MiMo-V2.6-Flash ( #58128 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-05 21:18:29 -07:00
Tianyu Guo
30ab428235
[Bugfix][Attention] Restrict FlashInfer sparse MLA FULL graphs to decode ( #59751 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-10-06 04:13:23 +00:00
lzhan011 and Flora Feng
a3e0243b1c
[Bugfix][Parser] Mistral pre-v11: don't fail requests on unexpected tool call JSON ( #54844 )
...
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-06 03:37:30 +00:00
Tianyu Guo and OpenAI Codex
8c417a2dae
[Bugfix][Model] Fix Qwen3-Omni interleaved M-RoPE boundary positions ( #59842 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-06 03:35:53 +00:00
Tianyu Guo
48eb44b719
[Bugfix][KVConnector][NIXL] Fix TP handshake test after per-region FA desc flags ( #60157 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-10-06 03:19:41 +00:00
6fa67afed9
[Bugfix][Core] Retire stale Mamba spec blocks in prefill checkpoint steps ( #59759 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-06 11:01:19 +08:00
2b32d9ac6a
[Bugfix][Core] Drop stale block hashes when a session truncates tokens ( #59103 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Co-authored-by: Brian Woods <me@brian.biz >
Co-authored-by: Change72 <changg@nvidia.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-06 02:51:03 +00:00
b0b23a48d4
[Bugfix] Fix Qwen4Exp FP8 PLE pinned lookup kernel failing to compile below SM89 ( #60142 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-06 01:56:44 +00:00
9465e1c783
[Bugfix][NIXL] Recover locally invalidated pull peer metadata ( #55471 )
...
Signed-off-by: xijiade.aihemaiti <3146335281@qq.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 18:46:53 -07:00
lzhan011 and Flora Feng
9bf4e51030
[Bugfix][Frontend] Malformed $defs in a tool schema should be a 400, not a 500 ( #54850 )
...
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-06 01:37:20 +00:00
amd-xavierwang and TJian
efd001756b
[ROCm] Disable AITER attention query quantization on RDNA3 ( #57750 )
...
Signed-off-by: xawang <xawang@amd.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-10-06 01:37:18 +00:00
2537629eb2
[Bugfix][KVConnector][NIXL] Size per-region replicate flags by each region's block count ( #60107 )
...
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-06 01:27:50 +00:00
0e8f8ecba8
[Bugfix][KVConnector][NIXL] Return NumPy descriptors from mixed-memory local registration ( #60108 )
...
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-06 01:23:20 +00:00
4e198cef83
[TEST][XPU][CI] Fix ci uva kernel test instability ( #60041 )
...
Signed-off-by: Your Name <you@example.com >
Co-authored-by: Your Name <you@example.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-06 09:19:01 +08:00
97bd8c74eb
[ROCm][DSv4.1][Perf] One-pass dequantize and a gather-sized grid for the compressed K cache ( #56720 )
...
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 17:57:46 -07:00
b2863c0de9
[Kimi-K3] Use the fused K/V pack kernel on the decode-context-parallel prefill path ( #56881 )
...
Signed-off-by: Brandon Zhang <31413216+brandonfzhang@users.noreply.github.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-06 00:13:55 +00:00
Mai Hoàng Đức and Flora Feng
5cd8a2626b
[Bugfix][Structured Output] Cap JSON schema nesting to prevent 500s and API-server stalls ( #60036 )
...
Signed-off-by: Mazukiri <maihoangduc04@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-06 00:06:59 +00:00
Aarushi Jain and Andreas Karatzas
fa423b614c
[CI][ROCm] Wait for engine teardown between LM Eval models ( #60100 )
...
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-05 23:39:59 +00:00
Mason and Yongye Zhu
6dd813e0b7
[Bugfix][MoE] Allow MiniMax2 routing in TRTLLM BF16 monolithic backend ( #59571 )
...
Signed-off-by: Mason Pitt <bradley.b.pitt@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-10-05 16:24:14 -07:00
0eac152707
[Bugfix] Route every speculation-capable row through the speculative path so recurrent state stays consistent ( #56531 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-06 03:03:44 +04:00
Ting SUN and Flora Feng
4a705c7b54
[Bugfix][Frontend] Preserve logprobs when top_logprobs is null ( #47838 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 22:43:31 +00:00
Kevin H. Luu and Claude Opus 5.5
89a00995c9
[CI] Read each profiler round's trace as it stops in test_gpu_profiler ( #59769 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 15:25:11 -07:00
RomeJay and Flora Feng
2e06395fd3
[Bugfix][Frontend] Build tool-call grammars from the tools the prompt renders ( #59879 )
...
Signed-off-by: Rongjie Jiang <j1870831863@outlook.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 22:01:03 +00:00
Martin Hickey and Flora Feng
3fb1cd796b
[Frontend] Extract parse and assembly out of batch chat derender ( #60073 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 21:57:45 +00:00
2e3154aa18
[HiSparse] Log steady-state max concurrency and expose host-tier utilization gauges ( #58949 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-05 14:20:32 -07:00
Matthew Bonanni and Claude Opus 5.5
8542c3eff5
[CI] Fix test_mixed_warmup_gate after #57053 ( #60116 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 21:17:00 +00:00
Yida Weng and Flora Feng
18c8a65edd
[BugFix][Frontend] Pass reasoning_ended through render → generate ( #60059 ) ( #60062 )
...
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 20:39:25 +00:00
Wentao Ye
74c5cbcd7b
[Bugfix][MRV2][Spec Decode] Honor dynamic K in autoregressive speculators, 1.2~1.3x kernel perf improvement ( #57053 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-05 14:54:11 -04:00
Raphaël Rialland and Simon Veitner
c32495d58f
[watermark] compatibility validation ( #56801 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
2026-10-05 14:37:13 -04:00
Giancarlo Delfin
88f3c601b3
[Docs] Add reviewer area of interest for TheEpicDolphin ( #60092 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-10-05 18:35:19 +00:00
aoshen02 and Claude Opus 5.5
d09ff77637
[Bugfix] Give each data-parallel engine its own global RNG streams ( #59788 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 17:38:18 +00:00
877ddcc456
[Bugfix][Spec Decode] Reserve the bonus KV slot for fill-in DSpark ( #59105 )
...
Signed-off-by: Evgeny Savinov <notime.sea@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
2026-10-05 17:35:51 +00:00
akii96
182247257f
[ROCm][Perf] Use a stacked hipBLASLt GEMM for the bf16x3 router ( #52668 )
...
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com >
2026-10-05 17:28:07 +00:00
Matthew Bonanni and Claude Opus 5.5
54d93af9fb
[Bugfix][HiSparse] Reject cudagraph_mode=FULL at startup ( #59688 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 12:42:27 -04:00
Sohaib Ahmed
64cb683842
[Model] Add Nemotron 3.5 ASR transcription support ( #59827 )
...
Signed-off-by: Sohaib-Ahmed21 <sohaibahmed1919@gmail.com >
2026-10-05 16:28:09 +00:00
stefankoncarevic and Andreas Karatzas
87954c5b04
[CI/Build] Declare the video modality on the CohereCompass reference model ( #60063 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-05 16:25:42 +00:00
Mikko Tukiainen and Cursor
d23af12cb1
[ROCm][Perf] Reach the fused QSA pre-indexer from the AMD path ( #57947 )
...
Signed-off-by: mjkvaak-amd <mjkvaak-amd@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-05 16:22:29 +00:00
bzsuni and cjackal
528772a4bf
[Bugfix][Core] Retain encoder cache references for repeated multimodal inputs ( #59942 )
...
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io >
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-10-05 15:52:46 +00:00
charlesli640
edde9d2ce4
Remove redundant dependency requirement for TPU ( #59977 )
...
Signed-off-by: Charles Li <licharles@google.com >
2026-10-05 08:45:37 -07:00
stefankoncarevic
f2d8fbf459
[CI/Build] Run the prefill token scoring test under batch invariance ( #60056 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-05 10:32:33 -05:00
Matthew Bonanni and Claude Opus 5.5
9e931a0c6c
[CI] Add CODEOWNERS for HiSparse ( #60060 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 15:18:29 +00:00
60932a6401
[Bugfix][Structured Output] Preserve literal values in Guidance disable_additional_properties ( #58709 )
...
Signed-off-by: Abhay Joshi <105213625+abhayjoshi201@users.noreply.github.com >
Co-authored-by: Antigravity <noreply@google.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-05 18:12:40 +03:00
Nicolò Lucchesi
30d4032363
[Model][GLM-5.3-Flash] Support DCP for the kpool sparse indexer ( #59211 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-05 16:44:14 +02:00
Jee Jee Li
f9c9e8ac24
[LoRA] Remove tensorizer ( #60024 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-05 07:01:56 -07:00
Abhay Shukla and Claude Opus 5.5
d3547f9d03
[Test] Re-enable Voxtral HF reference test on Transformers v5 ( #59771 )
...
Signed-off-by: AbhayShuklaIIT <abhay.shukla@live.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 13:43:49 +00:00
cjackal
0e468adb43
[Model] Extend device-side mm normalization to Kimi K2.5 / K3 ( #59278 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-10-05 21:28:13 +08:00
Nicolò Lucchesi and Leoyzen
ff53f32409
[Attention][MLA] Support fp8_ds_mla KV cache for NoPE-512 models on SM90 ( #59246 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Leoyzen <leoyzen@gmail.com >
2026-10-05 13:12:00 +00:00
Vadim Gimpelson and Claude
d0d6e5f3a2
[Bugfix] Fix Mamba page size AssertionError with spec decoding on GraniteMoeHybrid, FalconH1 and Zamba2 ( #59975 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-05 12:49:47 +00:00
roikoren755 and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
51eeb0c58f
[Bugfix] Load stacked expert weights for non-gated MoE ( #59031 )
...
Signed-off-by: Roi Koren <roik@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 15:26:55 +03:00
Stefano Castagnetta and Thien Tran
4ff028d77e
[Bugfix][Qwen4Exp] Honor --kv-cache-dtype-skip-layers in QSA attention ( #60023 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-05 12:13:30 +00:00
Stefano Castagnetta and Thien Tran
55b80221ed
[Perf][Qwen4Exp] Keep the HC up projection on the skinny GEMM path ( #60027 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-05 20:09:36 +08:00
Juan Pérez de Algaba
710ac56e69
[Security] Restrict Pillow image formats on untrusted media paths ( #60022 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
2026-10-05 10:40:15 +00:00
edca360f13
[Bugfix][Qwen4Exp] Load PLE tables unquantized under Quark checkpoints ( #59443 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-10-05 10:40:02 +00:00
c04c79e50d
[Performance] Add SM121 TP=2 skinny-GEMM plans ( #59632 )
...
Signed-off-by: sudhanshu shukla <sudhanshu112233shukla@users.noreply.github.com >
Co-authored-by: sudhanshu shukla <sudhanshu112233shukla@users.noreply.github.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-05 18:27:51 +08:00
Maroon Ayoub
b1401e0aa7
[Bugfix][NIXL] Count heartbeats as remote engine activity ( #59873 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
2026-10-05 09:36:13 +00:00
ae53b06898
[Bugfix][GLM-5.3-Flash] Keep batch x heads out of gridDim.z in the GLM-5.3-Flash fused recurrent KDA kernel ( #56974 )
...
Signed-off-by: NolenLiang <nliang@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com >
2026-10-05 09:31:45 +00:00
Ian Eaves and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
b1f229fb75
[Model] Declare SupportsEagle3 on Qwen3ASRForConditionalGeneration ( #52824 )
...
Signed-off-by: Ian Eaves <ian.k.eaves@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 09:23:47 +00:00
Jee Jee Li
e3af5bf3ce
[LoRA] Code cleanup ( #60017 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-05 09:15:53 +00:00
aoshen02 and Claude Opus 5.5
4a30c4cad0
[Model][DeepSeek-V4] Make MegaMoE shared-expert finalize independent of linear post-load order ( #59927 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 08:39:49 +00:00