alp and mottopanikeiku
ff01a6b14f
[Bugfix][Frontend] Stop beam search on generation config EOS tokens ( #62072 )
...
pre-commit / pre-run-check (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
2026-10-18 16:49:42 +00:00
aoshen02 and Claude Opus 5.5
fe8f2140ee
[Doc] Document /inference/v1/generate request/response fields and DP rank header ( #62305 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 15:26:36 +00:00
e80b6c642f
[ROCm][DSv4] Overlap the unfused shared expert with AITER MegaMoE ( #61819 )
...
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-18 22:39:38 +08:00
aoshen02 and Claude Opus 5.5
371745ebd6
[Doc][RL] Document routed experts return for MoE routing replay (R3) ( #62302 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 21:11:45 +08:00
072c0886d5
[Build] Keep plain CUDA arch when its suffixed variant is also requested ( #56549 )
...
Signed-off-by: Andre Lustosa <alustosa@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-18 20:13:17 +08:00
6b24947628
[CI/Build] Build DeepEP for SM103 (B300/GB300) in the Docker image ( #61829 )
...
Signed-off-by: Muhammad Ibrahim <muhammad.ibrahim@multiversecomputing.com >
Co-authored-by: Muhammad Ibrahim <muhammad.ibrahim@multiversecomputing.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-18 20:12:34 +08:00
Harry Huang and Thien Tran
494663940b
[Qwen4Exp] Clean up HC and PLE implementations for Qwen3.8-Flash-Next ( #61454 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-18 19:38:21 +08:00
Rémi Delhaye
3c13653738
[Frontend] Score with the chat template declared by Sentence Transformers CrossEncoders ( #60877 )
...
Signed-off-by: rdlhio <contact@rdlh.io >
2026-10-18 11:30:12 +00:00
Kevin H. Luu and Claude Opus 5.5
953fa442fa
[CI] Per-test timeout for Language Models (Extended Pooling) ( #62279 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 04:00:16 -07:00
Isotr0py
3f1b04e045
[Bugfix] Fix Opencv video backend frames count missing for WebM video inputs ( #61759 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-10-18 10:37:39 +00:00
aoshen02 and Claude Opus 5.5
2365177ddd
[Doc][RL] Document the weight-transfer HTTP protocol for trainers without vLLM ( #62267 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 09:26:56 +00:00
Louie Tsai
1b2a666391
[Doc][Recipe] Fix for redhat fp8 xeon support issue ( #62245 )
2026-10-18 16:46:56 +08:00
83c9a547bf
[CI][Release] Fix release wheel builds: stop excluding tracked recipes/ from Docker context ( #62221 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-18 00:48:00 -07:00
b510857d37
[Bugfix][Build] Build fused GDN MTP decode for SM80-SM90 with CUDA < 13 ( #60896 )
...
Signed-off-by: kimseunghyun <incongnito12@gmail.com >
Signed-off-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com >
2026-10-18 15:14:06 +08:00
7c9042c05a
[Bugfix][MRV2] Fix speculative warmup deadlock with pipeline parallelism ( #62129 )
...
Signed-off-by: chenjr97 <chenjr97@mail2.sysu.edu.cn >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 23:51:32 -07:00
da9134bc89
[ROCm][CI] Give the MoE parallel-launch watchdog room for AITER JIT builds ( #62161 )
...
Signed-off-by: Andreas Karatzas <AndreasKaratzas@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <AndreasKaratzas@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-17 21:49:27 -05:00
b2456c622b
[ROCm][CI] Make ROCm 10.0 the default image; keep ROCm 7.2 as -rocm72 ( #58761 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-17 21:43:59 -05:00
03ee2e134d
[CI][ROCm] Fix order-dependent test_rocm_aiter_sampler_defers_import (CPU tensors hit Triton) ( #62213 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-18 01:10:08 +00:00
djramic and Andreas Karatzas
9c778e9ff8
[Bugfix][EPLB] Wait for pending expert buffer copies before NIXL READs ( #62081 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-17 18:18:02 -05:00
7879f3687b
[NIXL] Drop unreachable producer-side notif counting in push mode ( #62210 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 21:24:35 +00:00
Kevin H. Luu and Claude Opus 5.5
255f14599e
[Bugfix][Rust Frontend] Fix macOS build: read inherited listener family from its address ( #62177 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 11:56:28 -07:00
03e8300d2c
[Bugfix] Enable inductor batch-invariant mode under VLLM_BATCH_INVARIANT ( #61118 )
...
Signed-off-by: Yifan Feng <evanfengff@gmail.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-17 14:53:06 -04:00
Afonso Ribeiro and Flora Feng
5c1c903501
[Perf][Parser] Stream Qwen3 tool argument values incrementally (12 KB call: 2.1 s to 11 ms) ( #62027 )
...
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-17 18:49:06 +00:00
Tyrone
bafb263664
[Bugfix][Benchmarks] Name the real cause when the /tokenize alignment probe fails ( #59888 )
2026-10-17 14:47:52 -04:00
Nick Hill and Claude Opus 5.5
0c5dd18e7a
[MRV2] Collect KV transfer results before the drafter runs ( #62041 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-17 10:27:57 -07:00
763e3f7c91
[Bugfix][Spec Decode] Disable dynamic speculative decoding for fixed-K proposers instead of crashing ( #46399 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-17 12:41:04 -04:00
Alec and Claude Opus 5.5
e744eacf68
[Rust Frontend] Validate bad_words limits during request lowering ( #61933 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 09:10:40 -07:00
c0b6f74bee
[Benchmark] Add per-case kernel profiles to vllm bench forward ( #61692 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 08:27:11 -07:00
siyu and Xun Sun
c61a1e73c6
[Feature] Skip MLA weight absorption when weights are pre-processed ( #57320 )
...
Co-authored-by: Xun Sun <UNIDY2002@outlook.com >
2026-10-17 23:10:04 +08:00
aoshen02 and Claude Opus 5.5
237d99b4af
Revert "[Docs] Add train-inference consistency agent skill" ( #61551 ) ( #62186 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 13:54:30 +00:00
aoshen02 and Claude Opus 5.5
2d8efee9a7
[Docs] Add train-inference consistency agent skill ( #61551 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 20:49:50 +08:00
Roy Wang and Claude Opus 5.5
21e09ea959
[Doc] Sync DeepSeek-V4.1-Flash recipe with vllm-project/recipes#1136, #1137 ( #62179 )
...
Signed-off-by: esmeetu <jasonailu87@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 12:27:52 +00:00
837e87e1c5
[Feature][LoRA][MoE] Expert LoRA for FlashInfer TRTLLM MXFP4 experts ( #61979 )
...
Signed-off-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 20:17:56 +08:00
liuzhenwei and Kunshang Ji
4e964b2623
[XPU] Fix Event init failure w/ external ( #61743 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-17 19:10:39 +08:00
Vadim Gimpelson and Claude
ec8b20fbf0
[Bugfix] Fix Gemma 4 encoder CUDA graph capture OOM at startup ( #61365 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-17 11:04:49 +00:00
2b85d9cb65
[Perf][Bugfix] Prefer MNNVL for single-node FlashInfer allreduce again ( #61989 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-17 02:55:28 -07:00
026f6ab268
[Model] DeepSeek-V4/V4.1 VL: stream language-model weights instead of sorting the checkpoint ( #58857 )
...
Signed-off-by: git bisector <gitbisector@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-17 09:37:56 +00:00
Roy Wang and Claude Opus 5.5
b515648641
[Doc] Host model recipes in recipes/ ( #61302 )
...
Signed-off-by: esmeetu <jasonailu87@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 09:14:36 +00:00
f98ae6a9fd
[Bugfix][Core] Keep resumable sessions that stop while preempted ( #58778 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Yifan Zong <yzong@redhat.com >
2026-10-17 09:08:14 +00:00
2067297b2f
[ROCm][Perf] Use AITER tuned configs to select a4w4 gemm kernel instead of env var ( #61278 )
...
Signed-off-by: Andy Friedrich <afriedri@amd.com >
Signed-off-by: afriedri <afriedri@amd.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-17 04:00:28 -05:00
Aarushi Jain and Andreas Karatzas
3bf25c5097
[CI][Bugfix] Ignore AMD CI webhook no-op builds in /amd-ci run and retry ( #62087 )
...
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-17 03:54:31 -05:00
Jee Jee Li
e56b1603af
[Kernel]Add norm fused kernel for Step-5 Preview and Muse Spark Open ( #62132 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-17 01:53:00 -07:00
Ting SUN
eb8de8d1a1
[Bugfix][OpenAI] Respect allowed_token_ids in beam search ( #47630 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-10-17 08:50:40 +00:00
dskwe
b05ede5616
[Bugfix][Frontend] Preserve token IDs in completion derender responses ( #62114 )
...
Signed-off-by: Yuan Li <dskwelmcy@163.com >
2026-10-17 08:27:48 +00:00
61bb590877
[Bugfix][Spec Decode] Clear stale shared block-table rows ( #61174 )
...
Signed-off-by: Cheng Rui <286040359@qq.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com >
2026-10-17 01:25:02 -07:00
c3de0b37e2
[Core][Spec Decode] Replace EAGLE group heuristics with explicit draft-layer tags ( #61455 )
...
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 08:22:33 +00:00
08a65e287e
[Bugfix][Kernel] hadacore: don't load past the input on a partial last warp ( #61944 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 00:52:17 -07:00
znzryb and Woosuk Kwon
ad2688ab30
[Bugfix] Validate draft-token capacity in forward benchmark ( #61809 )
...
Signed-off-by: znzryb <znzryb34@gmail.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
2026-10-17 00:51:36 -07:00
Jovan and OpenAI Codex
5be4266f5c
[Bugfix][Frontend] Preserve Cohere and Anthropic streaming errors ( #61947 )
...
Signed-off-by: Kevin <jia__kuan@163.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-17 06:10:16 +00:00
Wei Zhao
90d6d871d6
[Attention] Add CuTe DSL backend selection for MLA context prefill ( #62069 )
...
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-10-17 01:21:55 -04:00
Cy Cheng and Cy
d74f4180cd
[Bugfix][Frontend] Detokenize echoed prompts after truncation ( #61876 )
...
Signed-off-by: Cy <15118312+cycancode@users.noreply.github.com >
Co-authored-by: Cy <15118312+cycancode@users.noreply.github.com >
2026-10-17 05:20:38 +00:00
Jee Jee Li
39983a5e7a
[Kernel] Add FP32 router GEMM support for Step-5 Preview and Muse Spark Open ( #62126 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-16 22:17:55 -07:00
41b8c15809
[Feature][MoRIIO] Support hybrid WRITE transfers with DSpark ( #60544 )
...
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com >
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-17 04:57:58 +00:00
Micah Williamson and Cursor
a0e7137c4b
[CI][ROCm][Bugfix] DeepSeek V4.1: don't pass expert intermediate padding to AITER ( #61960 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-17 04:25:39 +00:00
7b6310c109
[Perf][Sampler] Speed up the small-batch split top-p pipeline ( #61996 )
...
Signed-off-by: Bowen Wang <bowen@inferact.ai >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 21:22:11 -07:00
Fadi Arafeh and vllm-agent
556ac15eb9
[Bugfix] Remove extra args from CPU's compute slot mapping kernel ( #61992 )
...
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 12:11:33 +08:00
Gabriel Wu and Claude Opus 5.5
dcd07967ca
[CPU] Keep the hybrid block size across the engine core's config rerun ( #61805 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 12:06:03 +08:00
ce49c47cbb
[Bugfix][Sampler] Carry the boundary tie count across blocks in the split top-p mask kernel ( #62046 )
...
Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com >
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com >
Co-authored-by: Bowen Wang <bowen@inferact.ai >
2026-10-17 02:26:57 +00:00
Sherif Waly and vllm-agent
ce8853d27a
[Bugfix][KV Offload] Allow explicit SimpleCPUOffload block counts ( #61674 )
...
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 02:11:12 +00:00
71c20f73f2
[FlashInfer] Migrate autotune warmup to flashinfer.autotune_v2 managed cache ( #55671 )
...
Signed-off-by: Yang Xu <yanxu@nvidia.com >
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Signed-off-by: YangXu <yanxu@nvidia.com >
Co-authored-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 02:08:28 +00:00
Shuolei Wang and aoshen02
b3a7e2ff6e
[RL][Frontend] Return weight-version spans on OpenAI completions and chat ( #61347 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-17 02:08:07 +00:00
6b6a6c5c98
[RL] Support per-parameter source layouts for NCCL M2N ( #60934 )
...
Signed-off-by: pkousha <pkousha@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-16 21:41:17 -04:00
Harry Mellor and Claude Opus 5.5
f8112704d5
[Chore] Use upstream Transformers InklingProcessor ( #61883 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 01:04:54 +00:00
f019e3e105
[ROCm] attn res + sigmoid_mul + conv fusions ( #52968 )
...
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com >
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-17 00:54:54 +00:00
Vadim Gimpelson and Claude
783380af39
[Bugfix] Fix NaN output of the Mamba2 chunk scan kernel on sm90 ( #61718 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-16 17:24:51 -07:00
Wei Zhao and Codex
be25977532
[CI][Tests] Account for FP8 KV-cache rounding in MLA reference ( #62067 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-16 17:10:08 -07:00
523d74e2ac
[Bugfix][LoRA] Fix garbage/NaN lora_expand output on SM90 with fp32 i… ( #61906 )
...
Signed-off-by: BeichenHuang <polarishuang0.0@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-16 19:42:30 -04:00
a94c315711
[DP] Engines own wave wake-ups; drop the front-end FIRST_REQ ( #59559 )
...
Signed-off-by: fh3 <a@b>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 19:41:54 -04:00
b544587408
[Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning ( #53558 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-16 23:16:47 +00:00
alp and mottopanikeiku
2268d83bd9
[Bugfix][Multimodal] Honor RGB PNG transparency when loading images ( #61986 )
...
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
2026-10-16 23:09:12 +00:00
28e92eeff2
[ROCm][Perf][Kimi-K3] Use AITER Gluon chunk_kda for KDA prefill ( #59552 )
...
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com >
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 22:58:56 +00:00
Afonso Ribeiro and Flora Feng
b088eba944
[Perf][Parser] Stream MiniMax M2 tool argument values incrementally (16 KB call: 5.9 s to 26 ms) ( #62035 )
...
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-16 22:46:02 +00:00
596e530675
[Kernel] Add router GEMM kernels for (5120, 256) and BF16 output to ll_bf16_gemm ( #62042 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Lain <fusiyuan2000@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 15:39:46 -07:00
153a17166e
[Frontend] Add the Beam reasoning and tool parsers ( #62052 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com >
Co-authored-by: neilkg <33635204+neilkg@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 14:08:29 -07:00
Dan Blanaru and Claude Opus 5.5
9731f30a03
[Bugfix] Keep VLLM_RAY_BUNDLE_INDICES out of the compile cache key ( #61211 )
...
Signed-off-by: DanBlanaru <48605845+DanBlanaru@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 20:59:48 +00:00
68ebea789a
[Perf][MoE] Skip padded work in SM100 FP4 DeepGEMM experts ( #59044 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-16 16:20:52 -04:00
cc2e4fecf2
[ROCm] [fusion] support ROCm aiter fusion of per tensor quant + rmsnorm ( #52831 )
...
Signed-off-by: Andy Friedrich <afriedri@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 15:19:07 -05:00
9eb7fb9157
[Build] Remove unused tools/install_deepgemm.sh ( #61265 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 15:57:49 -04:00
cjackal
663e535640
[Perf][Spec Decode] Skip bonus-logit reduction for short drafts in rejection sampling ( #61451 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-10-16 12:42:48 -07:00
Oxana Korzh and Claude Opus 5.5
5bf3d9f665
[ROCm][Kimi-K3] Support DSpark adaptive verification on MI355X ( #59824 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-16 13:53:49 -05:00
djramic
307c5844ad
[ROCm][CI] Enable batch invariance for the ngram scheduling accuracy test ( #62048 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
2026-10-16 18:36:14 +00:00
Guan-Ming Chiu and vllm-agent
3dc6def360
[Attention][Spec Decode] Enable varlen Mamba2 decode for adaptive verification ( #53036 )
...
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 11:23:59 -07:00
djramic
e7a5bdb454
[ROCm][CI] Move Spec Decode Speculators + MTP to MI355 ( #62053 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
2026-10-16 18:22:38 +00:00
alberto
ae8f41238c
[Bugfix][Parser] Honor a prompt-closed reasoning block in non-streaming parse and reasoning token counts ( #61857 )
...
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com >
2026-10-16 12:54:42 -04:00
cffc0cfa31
[CI] Fix B200 FusedMoE Layer Kernels hang on NVSHMEM IB setup ( #61695 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 12:00:12 -04:00
Andreas Karatzas and Claude
706791d516
[CI][AMD] Move 17 test groups from MI300 to MI355 and test large DeepSeek/Nemotron evals on AITER ( #61955 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-16 10:58:59 -05:00
Sawyer Bowerman and Codex
76081b9ee4
[Bugfix][Model] Honor quantization config in Inkling MTP ( #61921 )
...
Signed-off-by: Sawyer Bowerman <sbowerma@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-16 15:49:26 +00:00
bnellnm
1a5b187066
Add @bnellnm to CODEOWNERS ( #62025 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-10-16 11:16:03 -04:00
Wentao Ye
8e25bbca19
[Refactor] Remove dead code multiple places ( #61843 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-16 15:01:07 +00:00
8a6c0449b3
[Bugfix][ROCm][GLM-5.2] Stop forcing skip_final_all_reduce on AMD DSA model path ( #58589 )
...
Signed-off-by: avininjamay8 <Avinash.Paul@amd.com >
Signed-off-by: avininjamay8 <avpaul@amd.com >
Co-authored-by: avininjamay8 <avpaul@amd.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 21:38:55 +08:00
Ayushman Singh
de7eeaa87b
[Perf] Share Triton JIT warmup compilation pool across registered kernels ( #61422 )
2026-10-16 08:59:42 -04:00
Martin Hickey
64175d3827
[Bugfix][Frontend] Keep the parser's tool call IDs in batch derender ( #61852 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-16 10:27:17 +00:00
alp and mottopanikeiku
4ab99ff136
[Bugfix][Frontend] Honor the constructor chat template in offline scoring ( #61990 )
...
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
2026-10-16 10:20:46 +00:00
024c5188bf
[Frontend] Resolve logprobs on the parsed streaming derender path ( #60437 )
...
Signed-off-by: Shimi Bandiel <shimib@google.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 18:04:15 +08:00
Hexiang Wang
9a16d43ef0
[Bugfix][Mooncake] Fix MultiConnector stats and proxy SSE responses ( #61218 )
...
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com >
2026-10-16 17:54:48 +08:00
zofia
07f3857fb9
[XPU][Linear][MXFP4] enable torch as mxfp4 linear backend on xpu ( #51965 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Signed-off-by: zofia <110436990+zufangzhu@users.noreply.github.com >
2026-10-16 17:07:05 +08:00
fa75d3456d
[Feature][PCP] Add DeepSeek-V4.1-Flash support ( #59857 )
...
Signed-off-by: wangyicong <wangyicong@bytedance.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 23:17:51 -07:00
Chaojun Zhang and vllm-agent
074c83a09a
[CI][Intel] Skip ROCm-only test_topk_gating_enabled_by_cdna on XPU ( #61932 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 13:44:32 +08:00
Wei Zhao
e365792997
[Dependency] Upgrade TokenSpeed MLA to 2.0.3 ( #61689 )
...
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-15 22:28:59 -07:00
dskwe
49cb2bbf71
[Bugfix] Honor code_revision in config-based tokenizer loading ( #61611 )
...
Signed-off-by: Yuan Li <dskwelmcy@163.com >
2026-10-16 05:21:12 +00:00
4ac1f8d2ac
[Bugfix][Structured Output] Mask grammar-invalid implicit reasoning ends ( #61574 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-16 12:53:41 +08:00
maithilijoshi20 and Cyrus Leung
92dc3dd393
[Bugfix] Honor OPT token embedding width for prompt embeddings ( #61736 )
...
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-16 04:41:50 +00:00
Andreas Karatzas and Claude
b466fbacd2
[ROCm][Perf] Any-shape low-latency MoE router GEMM on MI355 ( #61895 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-15 23:22:31 -05:00
642c18d64c
[Perf] Kimi K3 Specialized Decode: CuTeDSL Optimized GB300 (1.5-2x) ( #61867 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-15 22:50:41 -04:00
3ddc6702a8
[XPU][Docker] Bump nixl to 1.5.0 ( #60356 )
...
Signed-off-by: Sandeep Maddipatla <sandeep.maddipatla@intel.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 10:35:29 +08:00
Shuolei Wang
05ad0a8ace
[Perf][DSv4.1] Split fused_wqa_wkv by token across TP ranks on SM90 ( #58872 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
2026-10-16 02:31:34 +00:00
dd9e07a25f
[Feature] Add External LB Mode Support for Elastic EP Scaling ( #43202 )
...
Signed-off-by: lqh-hh <liuqihang6@huawei.com >
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: lqh-hh <liuqihang6@huawei.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: 2017403603 <705328646@qq.com >
Co-authored-by: Julien Mancuso <jmancuso@nvidia.com >
Co-authored-by: Itay Alroy <75032521+itayalroy@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-15 21:29:02 -04:00
Monishver and Kevin H. Luu
fb3fec970e
[Bugfix][Kernel] silu_and_mul_quant: run the vectorized loop ( #61716 )
...
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 18:16:53 -07:00
Andreas Karatzas and Claude
29345f42a6
[ROCm][Qwen4Exp] Support HC and MoE sequence parallelism on ROCm ( #61471 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-15 20:11:17 -05:00
Kevin H. Luu and Claude Opus 5.5
a956a8126e
[Bugfix][MoE] Keep the gate's hash_indices_table out of RoutedExperts parameters ( #61930 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 18:00:17 -07:00
Jee Jee Li
bee95149e2
Update DeepGEMM commit ( #61744 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-15 17:54:40 -07:00
Tessa Pham and Benjamin Chislett
8f93c64866
[Bugfix] remove duplicate hash-routing capability method ( #61924 )
...
Signed-off-by: Tessa Pham <tepham@redhat.com >
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com >
2026-10-15 17:40:23 -07:00
a8be1500ba
[Bugfix][MoE] Restore TP-sharded routed-expert capture for DP=1 ( #61388 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Signed-off-by: Ao Shen <281812371+aoshen02@users.noreply.github.com >
Signed-off-by: Ao Shen <aoshen@inferact.ai >
Co-authored-by: Ao Shen <281812371+aoshen02@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Ao Shen <aoshen@inferact.ai >
2026-10-16 00:21:24 +00:00
Jack Hu and Douglas Lehr
ef38e8d006
[Model][Hardware][AMD][Kernel] Part 2/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_ATTN ( #42755 )
...
Signed-off-by: Jack Hu <Jack.Hu@amd.com >
Signed-off-by: Jack Hu <jingxujaster@hotmail.com >
Signed-off-by: Jack Hu <jack.hu@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-15 17:10:14 -07:00
Anerudhan Gopal and OpenAI Codex
af28c171f5
[Dependency] Upgrade FlashInfer to 0.7.2 and restore KDA auto backend ( #61913 )
...
Signed-off-by: Anerudhan Gopal <agopal@nvidia.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-15 17:04:24 -07:00
Juntian Liu and Claude Opus 5.5
19b0efce30
[Perf][Core] Skip the null head of sliding-window block tables ( #61724 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 16:14:51 -07:00
Woosuk Kwon and Claude Opus 5.5
4a89316efa
[Core] Drop the Ray driver-node GPU pin left over from the legacy executor ( #61713 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 16:00:18 -07:00
3f1e57225a
[ROCm][Perf][Kimi-K3] Use AITER Gluon KDA kernels for decode and spec decode ( #57392 )
...
Signed-off-by: omuhamma <omar.muhammad@amd.com >
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
2026-10-15 17:54:24 -05:00
de7d70676d
[Perf] Low-latency CPU all-reduce for DP metadata syncs ( #61767 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: hjjq <50634613+hjjq@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-15 15:29:13 -07:00
Stefano Castagnetta and Kevin H. Luu
784d1fff0c
[MRV2][Spec Decode] Quantized draft lm_head for drafters that share the target head ( #59973 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 22:06:40 +00:00
d92a4f25f2
[watermarking] Add DFlash support to dual-key gumbel-max watermarking ( #59554 )
...
Signed-off-by: Shern Shiou Tan <shernshiou@gmail.com >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Raphael Rialland <raphael.rialland@mistral.ai >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-15 22:06:06 +00:00
e711723352
[ROCm] gfx1250 optimizations and model enablement ( #58800 )
...
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com >
Signed-off-by: Juan Villamizar <jpvillam@amd.com >
Signed-off-by: jpvillam <jpvillam@amd.com >
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com >
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com >
Co-authored-by: jpvillam <jpvillam@amd.com >
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com >
Co-authored-by: Daniel <danichan@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com >
2026-10-15 14:51:57 -07:00
Thang Nguyen and Claude Opus 5.5
9b99ccb56b
[CI] Shard the AMD mirrors of the DP, RayExecutorV2 and batch invariance steps automatically ( #61861 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 16:48:19 -05:00
DimensionSTP and Kevin H. Luu
9edcc7fa7e
[Bugfix][Spec Decode] Avoid reusing random draws across block verification rounds ( #61161 )
...
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 14:46:38 -07:00
Mikko Tukiainen
0c25849292
[ROCm][Perf][Bugfix] Probe the preshuffled FP4 GEMM config by logical K ( #59406 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
2026-10-15 21:11:33 +00:00
f8460ef236
[Kernel][DSv4.1] Bump FlashMLA to the 2026.09.30 release and its interleaved V4.1 KV cache ( #61371 )
...
Signed-off-by: juntianl <juntianl@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-15 13:58:25 -07:00
33212db2c2
[KV Connector][NIXL] Reject NVFP4 KV cache with heterogeneous TP head split ( #61865 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 13:57:37 -07:00
Thang Nguyen and Claude Opus 5.5
c414bf0295
[CI] Shard four AMD mirrors automatically ( #61817 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 13:57:35 -07:00
Wentao Ye
54d9b1cb48
[GLM5.3 Perf] Inplace update draft decode metadata, 1.7% E2E Throughput Improvement ( #61873 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-15 20:20:12 +00:00
SeongJun Lee and Wei Zhao
ee8c47961b
[CI] Use an ungated model as the attention test config default ( #61732 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-15 20:19:09 +00:00
21f4b6dc84
[KV Offload] Split the CPU offload region across shm files ( #59546 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 20:17:40 +00:00
Thang Nguyen and Claude Opus 5.5
16532876e7
[CI] Shard four more AMD mirrors automatically ( #61860 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 15:13:51 -05:00
d12ea8e309
[Bugfix][MRV2] Keep input_ids on later PP stages for raw-token models ( #61016 )
...
Signed-off-by: booth-algo <kevinlauofficial01@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 12:54:21 -07:00
d486219fc2
[Bugfix][Core] Uncount stop-trimmed spec tokens from num_computed_tokens ( #58668 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-15 12:50:06 -07:00
49f944be3e
[Perf][HiSparse] Reuse pinned host pool after graph profiling ( #61668 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 12:33:03 -07:00
Rohan Potdar and Claude Opus 5.5
105605b8e9
[ROCm] Support DeepSeek-V4 hash routing in AITER Triton W4A16 MoE ( #60282 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 14:10:12 -05:00
Thien Tran
82b9ccaa79
[Perf][MoE] Skip the TRTLLM fp8-block activation-scale transpose copy ( #61773 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-15 12:09:07 -07:00
4d58fa254a
[MoE] Unify DeepGEMM MegaMoE experts behind one feature-matrix layer; add NVFP4 ( #61538 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 19:02:01 +00:00
Misha Goin and Claude Opus 5.5
4d9a8d6a41
[Bugfix] Make Humming DeviceInfo lookups traceable under torch.compile ( #61858 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 11:12:21 -07:00
Mikko Tukiainen and Cursor
47fd76e4ce
[ROCm][Perf] Route softmax MoE top-k, including fused shared-expert sigmoid, through AITER topk_gating ( #56226 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-15 17:53:18 +00:00
4f49b55c1f
[MoE] Shape-based padding for FlashInfer TRT-LLM MXFP8 experts (gated and non-gated) ( #61293 )
...
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-15 13:44:11 -04:00
7cd7982dfd
[Spec Decode] Don't reserve DFlash/DSpark draft slots in the scheduler token budget ( #59468 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-16 01:23:58 +08:00
bzsuni and Flora Feng
338dab077c
[Bugfix] Fix Apertus streaming scientific notation ( #61812 )
...
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-15 16:49:57 +00:00
957d512e96
[Bugfix][Frontend] Preserve streaming chunk boundaries in RequestOutputCollector ( #58941 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Co-authored-by: Benedikt Falk <bfalk@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-15 16:38:28 +00:00
Jovan
a9e90b9d1d
[Bugfix][Frontend] Reject Cohere REQUIRED tool choice without tools ( #61825 )
...
Signed-off-by: Kevin <jia__kuan@163.com >
2026-10-15 16:37:08 +00:00
d6f0b7a416
[CI] Skip exited descendants when finding the worker in the FT e2e test ( #61648 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-15 16:08:14 +00:00
ed3050add4
[Core] Apply prefill_schedule_interval outside data parallelism ( #54627 )
...
Signed-off-by: lifulu <fululi12@amd.com >
Signed-off-by: lifulu <lifulu@users.noreply.github.com >
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
Co-authored-by: lifulu <lifulu@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-15 11:03:13 -05:00
c825e1d235
[Perf] Fuse the mamba align block-table gather into one Triton kernel, 7 kernels -> 1, 6x speedup on all shapes for this kernel ( #58737 )
...
Signed-off-by: lifulu <fululi12@amd.com >
Signed-off-by: lifulu <lifulu@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: lifulu <lifulu@users.noreply.github.com >
2026-10-15 11:02:53 -05:00
CharlesWu and Yichao Zhu
a97d19d792
[UMBP][1/10] Add shared connector, runtime contracts, and request lifecycle ( #61789 )
...
Signed-off-by: CharlesWu <Charles.Wu@amd.com >
Co-authored-by: Yichao Zhu <Yichao.Zhu@amd.com >
2026-10-15 15:12:39 +00:00
3dcfaeb4e8
[Frontend] Apply each endpoint's top_logprobs cut in derender ( #59518 )
...
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-15 15:04:04 +00:00
Matthias Gehre
011fa68ca9
[ROCm][Perf] Pad fp16/bf16 GEMM weight row stride for L2 cache ( #55090 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
2026-10-15 14:26:32 +00:00
Ronald and aoshen02
d12d5460a5
[Bugfix][MoE] Keep routing rows during DeepSeek V4.1 decoder replay ( #61643 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-15 14:02:06 +00:00
24f632ca7e
[Bugfix] Engage indexer TP row sharding on total rows, not per-rank rows ( #61647 )
...
Signed-off-by: Zheng Cai <caizheng1993@gmail.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-15 13:54:14 +00:00
mosafariuk
69425846f8
[Bugfix][CPU] Report the real cause when the worker outgrows its memory reservation ( #59472 )
...
Signed-off-by: Mo <mosafariuk@gmail.com >
2026-10-15 21:52:05 +08:00
b7b20b1b8f
[Bugfix][Frontend] Leave the final stop token out of chat derender text ( #61757 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-15 21:51:34 +08:00
littlecircle0730 and Shanshan Shen
71536bd5f8
[MM][CG] Support ViT full CUDA graph for DeepSeek-VL2 ( #46005 )
...
Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com >
Signed-off-by: littlecircle0730 <43994952+littlecircle0730@users.noreply.github.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-15 21:38:52 +08:00
0951fc183c
[ROCm][DSv4.1] Disambiguate ROCm attention class names ( #58689 )
...
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 21:38:39 +08:00
1463501553
[Bugfix] Terminate streaming sessions on ABORT/ERROR and prevent zombie sessions ( #58942 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Benedikt Falk <bfalk@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-15 13:32:19 +00:00
Xunzhuo
766c837bd5
[Model] Support every Decision 3.0 size (d3-flash, d3-mini, d3-nano, d3-lite, d3-edge) ( #61778 )
...
Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com >
2026-10-15 13:29:24 +00:00
Laith Sakka and Nick Hill
9cede34a67
[Bugfix] Create cuBLAS handles before memory profiling to avoid OOM ( #61044 )
...
Signed-off-by: Laith Sakka <lsakka@meta.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-15 08:59:22 -04:00
Martin Hickey
252be28c18
[Frontend] Replace return_mm_kwargs with an mm_payload enum on render requests ( #61427 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-15 12:41:46 +00:00
Martin Hickey
c6b772cf6b
[Bugfix][Frontend] Emit kv/ec transfer params at the top level of render output ( #61241 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-15 12:34:15 +00:00
Micah Williamson
e8e1e96196
[ROCm][DSv4] AITER MegaMoE Fixed-Slot Dispatch For DeepSeek-V4 ( #61262 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
2026-10-15 19:10:59 +08:00
8b27d27402
[Bugfix][KV Offload] Always attempt NIXL agent removal during peer teardown ( #60279 )
...
Signed-off-by: nej1gotnochill <divyanshlalotra2@gmail.com >
Co-authored-by: Codebuff <noreply@codebuff.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 14:07:23 +03:00
Talor Abramovich
81e42c9987
[Feature][Frontend] Control the log level of request JSON body using env var ( #60630 )
...
Signed-off-by: talora <talora@nvidia.com >
2026-10-15 11:07:19 +00:00
jimmy-adams
365aaa77bd
[Bugfix][DSv4] Remap Quark MTP layer_quant_config onto draft prefixes ( #56054 )
...
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com >
2026-10-15 18:51:57 +08:00
e10c10a644
[Core] Add PROMOTION_LATENCY histogram metric for tiering offload ( #53910 )
...
Signed-off-by: Wen Zhou <wenzhou@redhat.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-15 13:47:41 +03:00
Pei Li and Kevin H. Luu
6fe7401fac
[Perf] Read VLLM_COMPUTE_NANS_IN_LOGITS once per IterationStats ( #61710 )
...
Signed-off-by: Pei Li <peili@nvidia.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 18:35:54 +08:00
Timofey and Codex
716c5225ae
[Model] Add AliceAI support ( #57963 )
...
Signed-off-by: timka <timka.byzov@yandex.ru >
Co-authored-by: Codex <noreply@openai.com >
2026-10-15 10:15:03 +00:00
7020600b1d
[KV Connector][NIXL] Register packed MLA KV rows as a single region ( #61074 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-10-15 09:18:15 +00:00
Flora Feng and Chauncey
80dd705c1d
[Frontend] Make finish_reason and Responses item status truthful ( #59324 )
...
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-15 10:12:02 +01:00
Denis Khachyan and Claude Code
427915bed2
[Build][CPU] Install Run:ai Model Streamer in vllm-openai-cpu image ( #61382 )
...
Signed-off-by: Denis Khachyan <khachyanda@gmail.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-10-15 09:11:51 +00:00
Kevin H. Luu and Claude Opus 5.5
7f730b4cea
[CI] Run 4xH100 steps nightly only ( #61782 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 01:32:43 -07:00
wang.yuqi
0f0bc4dbb1
[Test] Split the chat_utils tests. ( #61765 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-15 08:26:22 +00:00
b965ee564d
[Chore] Use upstream Transformers DeepseekV4Config ( #61392 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 01:08:08 -07:00
Misha Goin and Claude Opus 5.5
1c4c319792
[Frontend] Advertise --enable-x/--disable-x for boolean CLI flags ( #61479 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 01:08:02 -07:00
48f17b77bb
[Bugfix] Reject mismatched float16/bfloat16 KV cache dtype ( #60885 )
...
Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com >
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 08:06:59 +00:00
fc8e08a2a6
[Bugfix][KV Offload] Do not let mark_miss() flip a pending or in-flight async lookup probe ( #59096 )
...
Signed-off-by: Sohom Chakraborty <16609933+sohom-cs@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 10:57:46 +03:00
Bugen Zhao and Claude Opus 5.5
8fa5ae844a
[Frontend] Let an unset --tool-strict-level fall back to the tool parser's default ( #61328 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 15:48:38 +08:00
91ed2859c7
[CPU][Zen] Update zentorch to 2.14.0.1 & enable float16 on Venice and later ( #61411 )
...
Signed-off-by: Ganesh R <ganesh.r@amd.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-15 00:23:50 -07:00
85c94c1706
[Tests][XPU] Skip modelopt_fp4 online MoE re-quantization test on XPU ( #61723 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-15 00:23:45 -07:00
Kevin H. Luu and Claude Opus 5.5
8f111b22ee
[CI] Run 4xB200 steps nightly only ( #61541 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 00:08:48 -07:00
Kevin H. Luu and Claude Opus 5.5
32feecd91f
[CI] Pin DeepSeek-V4.1-Flash-NVFP4 eval to its FP8-Engram revision ( #61763 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 00:05:18 -07:00
Chaojun Zhang and Kevin H. Luu
ab593678b5
[CI][XPU] Ignore test_reload.py on Intel CI ( #61740 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 15:03:34 +08:00
a9fe30e9f6
[Bugfix][Multimodal] Recover PyNvVideoCodec decoder slots after failures ( #59597 )
...
Signed-off-by: Daniel Afrimi <dafrimi@nvidia.com >
Signed-off-by: danielafrimi <45691845+danielafrimi@users.noreply.github.com >
Signed-off-by: Daniel Afrimi <dafrimi@oci-hsg-cs-001-login-02.cm.cluster >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Daniel Afrimi <dafrimi@oci-hsg-cs-001-login-02.cm.cluster >
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com >
2026-10-15 09:17:19 +03:00
liranschour and Kevin H. Luu
c5f8adc0b1
[KV Offload] Expose OffloadingManager.lock()/unlock() and service the tiering control plane off the scheduler thread ( #58168 )
...
Signed-off-by: Liran Schour <lirans@il.ibm.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 08:50:56 +03:00
Harshil and Cyrus Leung
c54dca96bd
[Bugfix][Benchmark] Fix probe latency and duration in bench serve ( #60511 )
...
Signed-off-by: QHarshil <harshil_c@hotmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-15 05:50:30 +00:00
5341b5ba7d
[KV Offload] Pin only each worker's own offload slots ( #59544 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-15 08:49:42 +03:00
6605802904
[Perf][KV Connector][Mooncake] Send the KV layout once per peer ( #59825 )
...
Signed-off-by: staryxchen <staryxchen@tencent.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-10-15 05:36:31 +00:00
a400804ce4
[CI] Relax MTP sharded sampling cross-boot noise bound ( #57026 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 22:31:45 -07:00
3f1bcfd82b
[Bugfix][KV Offload] Bound unanswered P2P lookup waits ( #56836 )
...
Signed-off-by: jacklin78911-collab <jacklin78911@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 08:23:34 +03:00
81e1c3f183
[XPU][MLA] Integrate XPU MLA support into the existing FLASH_ATTN_MLA backend ( #58297 )
...
Signed-off-by: baodii <di.bao@intel.com >
Signed-off-by: Zhouxiaoya25 <xiaoya.zhou@intel.com >
Signed-off-by: ZhouXiaoya25 <xiaoya.zhou@intel.com >
Co-authored-by: baodii <di.bao@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-15 12:47:37 +08:00
Vivaan Sharma and Kevin H. Luu
96b5b0ee4b
[Model] Enable torch.compile and CUDA graphs for EmbeddingGemma2 ( #61572 )
...
Signed-off-by: Vivaan Sharma <99265792+Vivaansharma2003@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 12:32:39 +08:00
Haiyu Zuo and Kevin H. Luu
ff5a91a456
[Feature][Model Loader] Apply model-owned MTP shard selection to RunAI ModelStreamer ( #60759 )
...
Signed-off-by: 浩韵 <zuohaiyu.zhy@alibaba-inc.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 12:23:11 +08:00
Chaitanya Sri Krishna Lolla and Cursor
833665cded
[ROCm][DI] Enable MoRI FP4 dispatch for mxfp4 w4a4 moe ( #59596 )
...
Signed-off-by: lcskrishna <lollachaitanya@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-14 23:20:14 -05:00
Tuukka Sarvi and Kevin H. Luu
10d2c6083d
[ROCm][DSv4.1] Enable DSpark adaptive verification ( #58151 )
...
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 12:05:43 +08:00
Shanshan Shen
f46021a7fa
[ROCm][Perf] Enable DPA+ETP prefill compute/comm dual-batch-overlap for DeepSeek-V4 ( #57773 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-10-15 11:16:48 +08:00
Feanor and Taneem Ibrahim
5d5c3d7c43
[Bugfix] Preserve padding token types for cross encoders ( #58143 )
...
Signed-off-by: zhaoye <772971548@qq.com >
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-10-15 03:09:24 +00:00
429a3da665
[Bugfix][ROCm][Disagg] Allow MoRIIO HMA per-layer block_len ( #58585 )
...
Signed-off-by: avininjamay8 <Avinash.Paul@amd.com >
Signed-off-by: avininjamay8 <avpaul@amd.com >
Co-authored-by: avininjamay8 <avpaul@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-15 09:35:31 +08:00
Kai and Kunshang Ji
de2b869ee9
[Bugfix][Kernel] Register int8 scaled_mm as a custom op ( #57331 )
...
Signed-off-by: Kai <limkc.reply@gmail.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-15 09:27:49 +08:00
Aarushi Jain
2abd479aee
[ROCm][CI] Add AMD job for DeepSeek-V4.1 Mono Decode Numerics ( #61720 )
...
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com >
2026-10-15 01:23:25 +00:00
Woosuk Kwon and Claude Opus 5.5
e57bb4ace7
[Core] Remove the legacy Ray compiled-graph executor ( #61712 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 18:13:27 -07:00
bfdbfa1632
[Bugfix][PCP] Align MoE routing padding with gathered tokens ( #60946 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: wangyicong <wangyicong@bytedance.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-14 21:04:00 -04:00
Woosuk Kwon and Claude Opus 5.5
8530aed2b7
[Spec Decode] Remove Medusa and MLP speculator support ( #61701 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 17:29:23 -07:00
josh-schell-amd and Claude Opus 5.5
273a04964e
[ROCm] Bump AITER to v0.1.25 and relax the DSv4.1 mono MoE tolerance ( #61697 )
...
Signed-off-by: josh schell <josh.schell@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-14 23:50:24 +00:00
Aarushi Jain and Andreas Karatzas
7840bede98
[ROCm][CI] Add AMD jobs for test groups that only exist in tes… ( #59678 )
...
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-14 23:00:44 +00:00
Kevin H. Luu
4b61f300c0
[CI] Fix /ci run reporting a failed merge on fork PRs ( #61684 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-10-14 15:26:15 -07:00
Micah Williamson
a4d553a46a
Revert "[Build] Update deep_gemm git tag for locality_domain update" ( #61703 )
2026-10-14 14:57:44 -07:00
Rohan Potdar and Claude Opus 5.5
9a648b4649
[ROCm][Kimi-K3] Don't read W_UK_T before weight loading in the MLA wrapper ( #61654 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 21:44:55 +00:00
Ben Browning and Flora Feng
4c5ea86f53
[Bugfix][Parser] Stream Gemma4 string tool arguments incrementally ( #61494 )
...
Signed-off-by: Ben Browning <56071+bbrowning@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-14 21:25:41 +00:00
984c242ede
[Quantization] Support compressed-tensors MXFP4 MoE with FP8 activations via Mxfp4MoEMethod ( #59815 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 13:33:35 -07:00
fe98d90a9d
[Dependency] Update Humming and remove NVFP4 scale workaround ( #61556 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 13:32:36 -07:00
Woosuk Kwon and Claude Opus 5.5
2e8a5930f6
[Benchmark] Remove the temporary bench forward dummy loader ( #61676 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 13:30:18 -07:00
Juntian Liu
7c3f7e5500
[Perf][DSv4.1] Length-aware candidate block selection in the decode indexer ( #61039 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
2026-10-14 20:15:58 +00:00
rasmith and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2622ee21aa
[ROCm][BugFix] Don't use AITER rms_norm as an IR OP on gfx90a when AITER always on. ( #59290 )
...
Signed-off-by: Randall Smith <Randall.Smith@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 12:48:37 -07:00
alp and mottopanikeiku
e73fa00981
[Bugfix][Frontend] Apply the JSON schema in Cohere Chat v2 response_format ( #60586 )
...
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
2026-10-14 19:42:20 +00:00
Chris Leonard and Shengqi Chen
ded7ae35e5
[Build] Update deep_gemm git tag for locality_domain update ( #61491 )
...
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Signed-off-by: Shengqi Chen <harry-chen@outlook.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-14 15:41:16 -04:00
Juntian Liu and Claude Opus 5.5
388a610dfb
[Perf][DFlash] Capture context combine and anchor in the draft CUDA graph ( #59511 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-14 15:40:52 -04:00
Wei Zhao and Jee Jee Li
4975fda882
[Dependency] Upgrade FlashInfer to 0.7.1 ( #60948 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-10-14 12:31:08 -07:00
Ayushman Singh
9b5c6707a3
[Bugfix][MRV2] Capture DCP FULL cudagraphs with KV context ( #61496 )
...
Signed-off-by: Ayushman Singh <40520701+ayush1399@users.noreply.github.com >
2026-10-14 19:20:48 +00:00
Stefano Castagnetta and Misha Goin
222e00fc44
[CI/Build] Fix test_deepseek_v41_mxfp8_scale_loading after #57797 ( #61678 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-14 12:17:59 -07:00
Andreas Karatzas and Claude
e96c822660
[Bugfix][ROCm][MiniMax-M3] Reject unbuilt index-head counts in the AITER indexer ( #60990 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-14 14:13:48 -05:00
Sandeep Agrawal and Kevin H. Luu
21e207efa1
[Bugfix][MRV2][Spec Decode] Honor EAGLE3 single-state draft configuration ( #61563 )
...
Signed-off-by: Sandeep Agrawal <sandeepa@nvidia.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 12:09:06 -07:00
amd-xavierwang and JartX
06265beb25
[ROCm] Enable fp8 query for Triton attention ( #60296 )
...
Signed-off-by: xawang <xawang@amd.com >
Co-authored-by: JartX <sagformas@epdcenter.es >
2026-10-14 15:03:20 -04:00
857ddac4e5
[Perf][MRV2] Skip empty draft-token resets on request admission ( #61640 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-14 12:01:01 -07:00
c1407a7e86
[Spec Decode][MRV2] Warm up top-k/top-p and greedy rejection kernels for speculative decoding ( #59260 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 14:59:21 -04:00
simon-veitner-redhat and Claude
ae94e9c9e8
[Bugfix] Don't crash DeepGEMM MegaMoE warmup when get_bf16_mega_gate_config is missing ( #61657 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-14 18:58:35 +00:00
Simon Danielsson
3f729dafcc
[Perf][ROCm][GLM-5.3-Flash] AITER prefill indexer topk ( #60753 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-10-14 13:48:56 -05:00
lucas-santos-amd and Douglas Lehr
e534024e49
[ROCm][Bugfix][MLA] Fix FP8 MLA prefill unpad for interleave-padded heads ( #60907 )
...
Signed-off-by: Lucas Santos <Lucas.Santos@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-14 14:34:28 -04:00
d1d8e6c096
[Perf][Nemotron-Parse] Do image rescale/normalize on the device ( #60500 )
...
Signed-off-by: DiegoCao <DiegoCao@users.noreply.github.com >
Co-authored-by: DiegoCao <DiegoCao@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 18:33:22 +00:00
Kevin H. Luu and OpenAI Codex
0b6d07cbe2
[CI/Build] Use FileStore for DeepEP test rendezvous ( #56595 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-14 14:29:12 -04:00
Robert Shaw and Claude
d3f6b2a7b3
[Frontend] Clean Client Facing 500 Report ( #58043 )
...
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-14 14:27:58 -04:00
Ganesh R and Cyrus Leung
47232f3c7f
[Bugfix][CI] Fix renderers tests broken by tool name removal from warn logs ( #61652 )
...
Signed-off-by: Ganesh R <ganesh.r@amd.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-14 18:14:58 +00:00
Rohan Potdar and Claude Opus 5.5
52a8cc5316
[ROCm][CI] Add dtype to the KDA metadata test's fake model config ( #61650 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 18:04:14 +00:00
Woosuk Kwon and Claude Opus 5.5
9d3dfa2886
[Benchmark] Add vllm bench forward: exact-shape forward timing ( #61339 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 10:52:26 -07:00
Taimys and OpenAI Codex
2c74fd1873
[Model Runner V2] Enable batch-sharded sampling for GPT-OSS ( #61257 )
...
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-14 10:11:11 -07:00
Xunzhuo
89cab55086
[Model] Support Decision 3.0 decision models (vllm-sr/d3) ( #61565 )
...
Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com >
2026-10-14 17:10:55 +00:00
zexplorerhj and Codex
3895a8babe
[Bugfix] Fix TP loading of small block-FP8 MLA gates in BailingMoeV3 ( #61642 )
...
Signed-off-by: zexplorerhj <zhjoneson@163.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-14 10:09:23 -07:00
SeongJun Lee and Wei Zhao
e2e3e2b834
[CI] Use an ungated model in the DGX Spark XQA test ( #61416 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-14 17:04:07 +00:00
Harry Mellor and Claude Opus 5.5
3e851a7803
[Bugfix][Attention] Make the query contiguous for TRTLLM ragged MLA prefill ( #61634 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 16:38:52 +00:00
Zheng Gong and Cursor
355553ac22
[ROCm][Perf] Drop two dead per-step ops from the AITER sparse MLA metadata build ( #58354 )
...
Signed-off-by: ZhengGong-amd <zgong@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-14 11:15:51 -05:00
13dbf96093
[Model Loader] Apply EP weight filtering in InstantTensor ( #61167 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-14 09:10:10 -07:00
Nick Hill and Ilia Yastrebov
eaf25f675a
[NIXL] Prepare KV transfer descriptors as NIXL 1.5 strided lists ( #61032 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Ilia Yastrebov <iyastrebov@nvidia.com >
2026-10-14 18:07:24 +02:00
Nan Zhang and Robert Shaw
9e2b579aab
[Model] Support EAGLE3/DFlash aux hidden states for GraniteMoe and GraniteMoeShared ( #61110 )
...
Signed-off-by: ZN1010 <neilzhang1010@126.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-14 12:01:14 -04:00
e2aabc8b15
[Bugfix][KV Offload][P2P] Keep store pins and AbortAck until NIXL transfers stop ( #59790 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 18:31:31 +03:00
Juan Pérez de Algaba and OpenAI Codex
5d88c034f6
[Security] Reject local verifier references in Speculators metadata ( #61587 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-14 15:30:20 +00:00
bf6c80c7e8
[Misc] Remove tool name from warn logs ( #59294 )
...
Signed-off-by: Benjamin Merkel <benjamin.merkel@tngtech.com >
Co-authored-by: Benjamin Merkel <benjamin.merkel@tngtech.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-14 08:03:11 -07:00
Itay Etelis and Codex
efad400092
[HiSparse] Prefault the shared host pool in parallel ( #60869 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-14 10:28:26 -04:00
02653fa22c
[Bugfix][KV Offload] Fix inaccurate proportional store dropping ( #59140 )
...
Signed-off-by: Thuongvu Ho <thuongvuho@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-14 17:05:21 +03:00
Ashish Rao and Vadim Gimpelson
b4b7eca0ef
[Bugfix][Quantization] Clamp FP8 scales before UE8M0 rounding ( #58032 )
...
Signed-off-by: Ashish Rao <asrao@nvidia.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
2026-10-14 13:56:42 +00:00
7115b70c01
[Bugfix][KV Offload][P2P] Drop a timed-out store job's blocks from the round's parked supply ( #59329 )
...
Signed-off-by: Sohom Chakraborty <16609933+sohom-cs@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-14 13:22:28 +00:00
Luca Motz and Claude Opus 5.5
2d09fdcdc2
[Quantization] Auto-disable DeepGEMM for Kolibri 1 on SM12x ( #61274 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 12:59:50 +00:00
3723a1ea96
[Bugfix][Transformers backend] Fix CUDA graph capture for dynamic-rope models ( #58064 )
...
Signed-off-by: YongfengX <erkang040804@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 05:56:54 -07:00
hcl and Harry Mellor
2631afd3fe
[Bugfix][Platform] Fall back to torch when device-level NVML memory is unsupported (GB10) ( #57378 )
...
Signed-off-by: hclsys <chenglunhu@gmail.com >
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-14 12:45:29 +00:00
Kapil Arya and Or Ozeri
a68dbbad05
[Bugfix][KV Offload] Retain offload mappings after unsafe KVCR startup failure ( #57500 )
...
Signed-off-by: Kapil Arya <kapila@nvidia.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-14 15:43:04 +03:00
775fefa804
[Metrics][KV Offload] Add vllm:kv_offload_config_info - generic info metrics for offloading the offloading connector ( #56867 )
...
Signed-off-by: Amir Friedman <Amir.Friedman1@ibm.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-14 12:34:37 +00:00
ssingh231219
3a5a6b11a3
[Perf][Model] Batch PaddleOCR-VL vision encoder and projector ( #61458 )
...
Signed-off-by: ssingh231219 <simran.singh@artificialy.com >
2026-10-14 19:59:44 +08:00
d39c2fb05a
[Bugfix][Gemma4] Quantize audio towers independently ( #59935 )
...
Signed-off-by: sudhanshu shukla <sudhanshu112233shukla@users.noreply.github.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: sudhanshu shukla <sudhanshu112233shukla@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-14 11:27:44 +00:00
Lazurite
23fbc2ec2f
[Model] Support precomputed image embeddings for LightOnOCR ( #61585 )
...
Signed-off-by: Rapisurazurite <43494437+Rapisurazurite@users.noreply.github.com >
2026-10-14 10:54:09 +00:00
xuebwang
81503550f3
[ROCm] Disable AITER FP8/FP4 BMM for MLA by default ( #53123 )
...
Signed-off-by: xuebwang-amd <xuebwang@amd.com >
2026-10-14 10:46:02 +00:00
linshi wang and Or Ozeri
c67c69e657
[Bugfix][KV Offload] Track allocated block IDs when num_external_tokens is 0 ( #47046 )
...
Signed-off-by: linshi-w <linshiw.edu@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-14 13:34:10 +03:00
wang.yuqi
6ecb39eaa7
[Docs] Update structured decisions docs. ( #61588 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-14 10:28:22 +00:00
Ronald and aoshen02
19aba3e4d5
[Bugfix][RL] Load attention sinks through weight_loader for layerwise reload ( #57797 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-14 10:17:58 +00:00
f9b65c30c7
[ROCm][Perf] Make AITER FlyDSL GDN prefill default on gfx942/gfx950 ( #60645 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 10:11:57 +00:00
3f466c852c
[Bugfix][Parser] Recover genuine tool calls after unclosed Qwen3 <function= opener in prose ( #59928 )
...
Signed-off-by: Abhay Joshi <105213625+abhayjoshi201@users.noreply.github.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Gemini <noreply@google.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-14 13:01:25 +03:00
8a8e2baa02
[Bugfix][Rust Frontend] Inject VLLM_USE_RUST_FRONTEND=1 into vllm-rs managed engine envs ( #61586 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-14 17:09:08 +08:00
Sherif Waly
be4ce2e505
[Bugfix] Share PCP-aware local GPU indexing for NUMA binding ( #61237 )
2026-10-14 10:54:03 +02:00
Thien Tran and Kimi Code
937d6c19df
[Perf][GDN] Skip FLA chunk metadata when FlashInfer GDN prefill is active ( #61569 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-10-14 01:46:54 -07:00
Kevin H. Luu and Misha Goin
d36158f919
[CI] Merge the target branch into stale PRs on /ci run ( #61592 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-14 01:43:03 -07:00
Artem Perevedentsev
bb1438a93f
[CI] Fix Unlimited-OCR loading after upstream model_type rename ( #61590 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-14 01:24:24 -07:00
467b024f21
[Rust Frontend] Report vllm-proto schema version in ServerInfo ( #61533 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-14 16:21:48 +08:00
aa0c433a3e
[Frontend] Accept --reasoning-parser none as the unset default ( #61525 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-14 15:50:35 +08:00
c9bfa574ce
[Online quantization] Extend re-quantization API to MOE (showcase NVFP4 modelopt/CT/quark to any online MOE method) ( #60276 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com >
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com >
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com >
2026-10-14 15:49:37 +08:00
Chauncey and Cyrus Leung
e1726d9142
[Frontend] Add cache salt and tracing to openai decisions ( #61550 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-14 00:48:44 -07:00
zofia
4dc5add6ea
[XPU] [Linear] [MXFP8] enable torch as mxfp8 linear backend ( #51933 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Signed-off-by: zofia <110436990+zufangzhu@users.noreply.github.com >
2026-10-14 00:48:38 -07:00
Ronald
ec8a372566
[ROCm][Bugfix] Pin AITER back to v0.1.24.post1 for DSv4.1 mono decode numerics ( #61559 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
2026-10-14 00:19:38 -07:00
30e79c3a76
[Core] Update PyTorch to 2.14.0, torchvision to 0.29.0, triton to 3.8.0 (release channel) ( #52183 )
...
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com >
Signed-off-by: Andrey Talman <atalman@fb.com >
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 00:17:59 -07:00
ec317a4618
[CI] Keep vLLM log propagation on through the first LLM init in caplog_vllm ( #61535 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-14 15:10:35 +08:00
Jiangyun Zhu and Claude Opus 5.5
badfcdbf91
[Perf][GDN] Internal prefill checkpoints for the FlashInfer GDN prefill backend ( #60050 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 14:49:46 +08:00
a2fb482732
[KVConnector][P2P] Register peers off the scheduler thread ( #57139 )
...
Signed-off-by: Liran Schour <lirans@il.ibm.com >
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-10-14 08:21:57 +03:00
77154ca611
[Perf][Model][Qwen4Exp] QSA: decode four FP8 e4m3 bytes per register below sm_89 ( #60992 )
...
Signed-off-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-14 13:06:16 +08:00
Akshan Krithick
02d69c663a
[Bugfix][Benchmark] Preserve SSE output across transport chunk boundaries ( #51172 )
...
Signed-off-by: Akshan Krithick <akshankrithick305@gmail.com >
2026-10-13 21:58:58 -07:00
a78d050eff
[Bugfix] Bump tokenspeed-mla to 2.0.2 to fix nanobind conflict with Triton 3.9 ( #61125 )
...
Signed-off-by: Warren Deng <warrdeng@meta.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-13 21:28:07 -07:00
3ccc0a90b6
[KV Connector] Support sleep mode with MooncakeConnector over RDMA ( #59625 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: Dakai An <77474977+andakai@users.noreply.github.com >
2026-10-14 04:18:47 +00:00
Lanze Liu and Ning Xie
f4b8d43a68
[Bugfix] Keep force-killing the process tree when a child exits mid-walk ( #59984 )
...
Signed-off-by: Lanze Liu <lanzetech@gmail.com >
Co-authored-by: Ning Xie <andy.xning@gmail.com >
2026-10-13 20:33:18 -07:00
Shanshan Shen
2521446ac8
[ROCm][Perf] Enqueue the default chain first in CSA/HCA at small batch sizes for DeepSeek-V4 ( #61378 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-10-14 10:57:26 +08:00
2308b6f00b
[Perf][MoE] Cheaper triton_kernels 3.8 routing build in the OAI Triton MoE ( #58897 )
...
Signed-off-by: Zheng Gong <zgong@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-14 10:48:05 +08:00
xaguilar-amd
a1f51a0226
[ROCm][MLA] Route gfx942 DCP causal multi-token verify to Triton split-KV decode ( #58555 )
...
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com >
2026-10-14 10:17:15 +08:00
John Calderon and Artem Perevedentsev
449c05a0ca
[CI][Tests] Upgrade torch version for Rubin prerelease ( #61468 )
...
Signed-off-by: John Calderon <jcalderon@nvidia.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-14 10:10:00 +08:00
Kevin H. Luu and Codex
d5e62627a3
[CI] Shard chat APIs, model evals and model tests automatically ( #61475 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-13 18:16:27 -07:00
Stefano Castagnetta and Kartikey Vyas
4e0681ed56
[Quantization] Support packed NVFP4 Qwen4Exp PLE embeddings ( #56273 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Kartikey Vyas <kartikey@artificialanalysis.ai >
2026-10-14 08:24:55 +08:00
Almog Tavor and Itay Alroy
87e7dce3a7
[MRV2] Keep warmup out of the EPLB load stats ( #56716 )
...
Signed-off-by: almogtavor <almogtavor@gmail.com >
Co-authored-by: Itay Alroy <75032521+itayalroy@users.noreply.github.com >
2026-10-14 00:00:48 +00:00
Ryan Rock and Micah Williamson
f43d7e1a0f
[CI][AMD] Gate Aiter MHA FA on dtype ( #60679 )
...
Signed-off-by: Ryan Rock <ryan.rock@amd.com >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
2026-10-13 18:55:19 -05:00
josh-schell-amd and Micah Williamson
7a9d3db896
[ROCm] Bump ROCm base images to AITER v1.25.0 ( #61230 )
...
Signed-off-by: josh schell <josh.schell@amd.com >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
2026-10-13 18:55:09 -05:00
c923e7769b
[Feat] Add text-only OpenAI Decisions API MVP ( #60465 )
...
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-14 07:51:08 +08:00
Chauncey and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ad8d39cbab
[Bugfix] Fix DSpark graph-padding handling in the DeepSeek-V4.1 MoE fused top-k ( #59166 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 07:49:52 +08:00
Giancarlo Delfin and Woosuk Kwon
791f92135a
[Attention] Set ALWAYS cudagraph support for FA4 ( #60908 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
2026-10-13 16:22:52 -07:00
Kevin H. Luu and Codex
6a03954b45
[CI] Shard quantization, pooling, multimodal APIs, core kernels and PCP eval automatically ( #61477 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-13 16:15:30 -07:00
Kevin H. Luu and Codex
0da66a6e46
[CI] Shard batch invariance, Kimi prefix cache, EP eval and generate APIs automatically ( #61474 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-13 16:14:18 -07:00
8082b62f32
[Mooncake] Allow Mamba cache mode none for capacity-only workers ( #60712 )
...
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 23:13:58 +00:00
Tim002 and Thien Tran
96fc2da315
[Perf][Qwen4Exp] Add SM121 TP=2 merged QSA projection plans ( #61437 )
...
Signed-off-by: TimRen01 <HolaTim@hotmail.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-13 22:58:44 +00:00
Nick Hill and Claude Opus 5.5
78b32b3fdc
Revert "[Build] DeepGEMM migration to torch Stable ABI ( #59482 )" ( #61510 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-13 15:33:50 -07:00
Wenbin Chen and Misha Goin
6463d04601
Fix write-after-write hazard in xpu KV RoPE triton kernel ( #46685 )
...
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-13 22:17:14 +00:00
698e79f3bd
[Model] Kolibri 1 via the Transformers modeling backend ( #60827 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-13 22:17:52 +01:00
Anirudh Ravichandran
e90158d912
[Bugfix][MoE] Clamp BLOCK_SIZE_K to the FP8 quantization block in the batched Triton MoE ( #60733 )
2026-10-13 17:02:17 -04:00
Matthew Bonanni and Claude Opus 5.5
292da21f9d
[HiSparse] Read GPU-resident KV directly for mixed and BF16 prefills ( #60417 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 16:40:20 -04:00
Wentao Ye
c19c263ac8
[GLM 5.3 Perf] Handle two elements at the same time for per token group quant, 2.8% E2E ITL improvement ( #60927 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-13 13:06:41 -07:00
Andreas Karatzas and Claude
67516b6c42
[Bugfix][ROCm][Kimi-K3] Stage fused KDA decode weights after loading ( #61207 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-13 14:55:57 -05:00
djramic
93d0bd9d50
[ROCm][CI] Drop the -inf padding check from the AITER MQA logits test ( #61420 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
2026-10-13 15:49:30 -04:00
a31054f5ef
[NIXL] Bump CUDA 12 NIXL to 1.5.0 ( #60428 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 12:47:41 -07:00
68fc395554
[Bugfix] Keep GateLinear batch invariant under VLLM_BATCH_INVARIANT ( #59985 )
...
Signed-off-by: yifanFengg <evanfengff@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-13 15:43:54 -04:00
yifanFengg and Claude Fable 5.1
b33cb81237
[Bugfix] Keep compressed-tensors wNa16 linear layers batch invariant ( #60524 )
...
Signed-off-by: yifanFengg <evanfengff@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-13 15:43:40 -04:00
Wentao Ye
2f18d13cf0
[GLM5.3 Perf] Enable fp8 kv cache for GLM5.3 Flash by default, 9.8% E2E Throughput Improvement ( #61228 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-13 15:29:17 -04:00
483e0c24c7
[KV Offload] Split replicated-layout pre-faulting across workers ( #59545 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 19:18:56 +00:00
c068db0e1d
[Bugfix][Compile] Replace Transformers AOT caching with graph caching ( #58918 )
...
Signed-off-by: fangyao fan <2281693949@qq.com >
Signed-off-by: Taimys <ffy2281693949@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Taimys <ffy2281693949@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-13 19:17:29 +00:00
Anish Shanbhag and OpenAI Codex
3f550129e8
[Perf][Attention][MiniMax-M3] Select CUTLASS MSA by EAGLE3 decode shape ( #55838 )
...
Signed-off-by: Anish Shanbhag <ashanbhag@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-13 15:05:28 -04:00
GuanLuo and sfeng33
6d1daa9fe5
[Bugfix][Frontend] Expose completion response schemas in OpenAPI ( #60926 )
...
Signed-off-by: Guan Luo <gluo@nvidia.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-13 18:54:06 +00:00
c5bda658d2
[Model][MoE] Load MegaMoE experts directly into the DeepGEMM layout ( #60096 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Aaron Hao <hao-aaron@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Aaron Hao <hao-aaron@users.noreply.github.com >
2026-10-13 11:52:08 -07:00
Matthew Bonanni and Claude Opus 5.5
624a5ddafd
[Attention] Fetch MLA prefill workspace at use instead of init ( #60654 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 17:25:03 +00:00
e83222396f
[ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches ( #57199 )
...
Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-13 17:21:30 +00:00
Roberto L. Castro and Claude Opus 5.5
7fad7db330
[Perf][Feat] Add generic cuteDSL LL FP32 router (GEMM) ( #44343 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 12:38:46 -04:00
6849ca696a
[Quantization] Add ModelOpt IQ2_XS, IQ2_XXS, and Q8_0 support via b12x ( #59604 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Signed-off-by: Luke Alonso <lalonso@gmail.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Luke Alonso <lalonso@gmail.com >
2026-10-13 09:06:33 -07:00
Woosuk Kwon and Claude Opus 5.5
ac71b17517
[Model Loader] Keep norm layers neutral under dummy weight init ( #61323 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 08:54:24 -07:00
Taksh Kothari
12b005eefa
[Bugfix] Don't split a multimodal item that a prefill chunk starts exactly on ( #52412 )
...
Signed-off-by: Taksh <takshkothari09@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-13 15:32:38 +00:00
Elvir Crnčević
b9b401b2ef
[Kernel] Fuse DeepEP-v2 decode globalize+align into one launch ( #53911 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
2026-10-13 11:10:38 -04:00
wang.yuqi
669f85e686
[Misc] Break chat_utils into more maintainable modules. ( #60972 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-13 15:08:07 +00:00
Rishabh Agarwal
f395a6b1ad
[ROCM][Bugfix] Deferring to TRITON_ATTN backend for Prefix lm models instead of ROCM_ATTN and fixed minor bug in Gemma3 Multimodal code ( #46754 )
...
Signed-off-by: GodRishUniverse <rishabh.agarwal@amd.com >
Signed-off-by: Rishabh Agarwal <risagarw@amd.com >
Signed-off-by: GodRishUniverse <risagarw@amd.com >
2026-10-13 10:03:48 -05:00
Chris Leonard and Shengqi Chen
f5fefbffe5
[Build] DeepGEMM migration to torch Stable ABI ( #59482 )
...
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Signed-off-by: Shengqi Chen <harry-chen@outlook.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-13 08:01:44 -07:00
Jiangyun Zhu and Claude Opus 5.5
61d9c777bf
[CI] Run the Clef pooler test on GPU ( #61409 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 14:42:53 +00:00
de8291e24f
[Core] Add lifecycle hooks for worker extensions ( #59463 )
...
Signed-off-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-13 21:51:05 +08:00
Thien Tran
0fae39a014
[Perf] per_token_group_quant_8bit: multiply by reciprocal instead of dividing per element ( #61343 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-13 21:37:10 +08:00
31e90ac370
[Attention] Enable FA4 sparse MQA decode on Blackwell ( #55866 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-13 09:31:37 -04:00
Nathan Sala and Shaun Kotek - Nvidia
0838cd6593
[Bugfix] Load and apply Nemotron Super final vision LayerNorm ( #61401 )
...
Signed-off-by: Shaun Kotek <skotek@nvidia.com >
Co-authored-by: Shaun Kotek - Nvidia <skotek@nvidia.com >
2026-10-13 12:42:29 +00:00
Hexiang Wang
7342dc0317
[MoRIIO] Require pending remote prefill for READ cache hits ( #60638 )
...
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
2026-10-13 20:19:53 +08:00
253e02b95e
[Perf] Warm up DeepGEMM MegaMoE kernels before serving ( #60468 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
2026-10-13 20:17:48 +08:00
Aditya Ganesh Kumar
d9a543c500
[Bugfix][Frontend] Forward --enable-log-outputs to the Cohere Chat v2 ( #61089 )
...
Signed-off-by: Aditya Ganesh Kumar <adty910@gmail.com >
2026-10-13 12:01:59 +00:00
818f70c295
[Bugfix] Mask special added tokens in xgrammar JSON, regex and EBNF grammars ( #60749 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-13 14:38:23 +03:00
Vadim Gimpelson and Claude
cd7517fb17
[Bugfix] Fix ReplaySSM ring shape assertion when TP does not divide Mamba2 n_groups ( #60704 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-13 11:31:53 +00:00
Linze Shi
e6e84f350c
[Feature][Pooling] Add offline fixed-length late chunking ( #60164 )
...
Signed-off-by: Linze-Shi <linzeshi0@gmail.com >
2026-10-13 11:16:21 +00:00
noone-zhhh and Isotr0py
426f3bd36b
[Bugfix][Model][Rust Frontend] Qwen3_ASR: Handle ragged Qwen audio batches from Rust frontend ( #60979 )
...
Signed-off-by: zh98 <workallday.zhh@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-13 10:49:39 +00:00
samayp01 and Artem Perevedentsev
54db759eea
[Bugfix][Structured Output] Reject structural_tag and unhandled specs in backend validation ( #58862 )
...
Signed-off-by: Samay <samay.piduru@gmail.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-13 13:27:34 +03:00
Juan Pérez de Algaba and OpenAI Codex
7173c64e2e
[Security] Bound decoded and downloaded media per request ( #61226 )
...
Signed-off-by: juanpas97 <juan.perezalgaba@gmail.com >
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-13 10:17:14 +00:00
JackRao123 and Claude Opus 5.5
877f6df889
[Bugfix][DSv4] Don't bake the C128 compressor skip into CUDA graphs ( #59709 )
...
Signed-off-by: Jack Rao <jack.rao@baseten.co >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 10:15:11 +00:00
c7a305ec10
[Bugfix][Mamba] Store conv_kernel - 1 inputs in Mamba2 prefill checkpoints ( #60797 )
...
Signed-off-by: Sungsoo Ha <sungsooh@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com >
2026-10-13 10:09:17 +00:00
Kevin H. Luu and Claude Opus 5.5
206e45d470
[CI] Shard V1 Attention automatically instead of by fixed pytest shards ( #61200 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 02:24:06 -07:00
Thien Tran and OpenAI Codex
656d0d4506
[Perf] Tune Triton GDN decode tiles on SM100-family GPUs ( #61355 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-13 17:01:12 +08:00
Vadim Gimpelson and Claude
183625d4f9
[Bugfix] Fall back to auto for unquantized MoE layers when --moe-backend is marlin ( #60219 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-13 08:48:13 +00:00
Harry Mellor and Claude Opus 5
6f11c220b5
[Transformers backend] Fuse MoE routers that score after the top-k ( #57410 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-13 08:41:23 +00:00
stefankoncarevic
24aa1191f4
[ROCm][Bugfix] Run the AITER AR+RMSNorm fallback test on the platform… ( #61350 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-13 08:35:57 +00:00
Jiangyun Zhu
ff305934f1
[Tests][Mamba] Wait for accepted counts in fake MTP proposer ( #61342 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-13 16:15:47 +08:00
zhouyou
3ad096faf8
[Bugfix][EC Connector] Don't create hash directories on cache-miss lookups in ECExampleConnector ( #60826 )
...
Signed-off-by: zhouyou9505 <zhouyou9505@gmail.com >
2026-10-13 15:59:48 +08:00
2cc2e68d1c
[Core][Mamba] Remove leftovers of mamba_cache_mode="all" ( #60043 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 00:58:59 -07:00
bc3ddb198a
[Model][Qwen4Exp] FP8 QSA main KV cache on Ampere (sm_80, sm_86) ( #60602 )
...
Signed-off-by: kimseunghyun <hellonypcteam@gmail.com >
Signed-off-by: kimseunghyun <incongnito12@gmail.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-13 15:30:19 +08:00
Kevin H. Luu and Claude Opus 5.5
4e4eab73ed
[CI] Shard Model Executor and L4 Pipeline + Context Parallelism automatically ( #61202 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 23:57:32 -07:00
e2ad0ce5f5
[Bugfix][LoRA] Slice expert LoRA by the unpadded per-rank size under TP ( #61235 )
...
Signed-off-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-13 14:25:16 +08:00
Yongye Zhu and Claude Opus 5.5
a6009ea85b
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer ( #61149 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-12 23:24:43 -07:00
Nicolò Lucchesi and Claude Opus 5.5
86b1c2ef15
[Attention][DCP] Support a2a DCP comm backend for FlashMLA sparse ( #61285 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 13:51:22 +08:00
380638afbb
[Model] Add Clef-Flash support ( #61308 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-13 05:46:37 +00:00
0d2689e0df
[ROCm][Perf] Skip redundant sparse index remap on non-indexer layers ( #51309 )
...
Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
2026-10-13 05:25:56 +00:00
2ce8012fa0
[ROCm][Perf] Skip cleaning sparse prefill MQA logits ( #51314 )
...
Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 00:21:52 -05:00
Divakar Verma and Claude Sonnet 5
f7ba007b5e
[ROCm][MoE] Split AiterExperts by weight-quantization key ( #60322 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-10-13 00:13:33 -05:00
b337523c11
[XPU] [Test] Extend spawn-safe sitecustomize monkeypatch to XPU ( #57911 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
Co-authored-by: xiangdong <40376367+zxd1997066@users.noreply.github.com >
2026-10-13 13:12:10 +08:00
Mook and cjackal
5555a2d8f7
[Bugfix][Model] Fix EXAONE-4.5 image inputs: ViT received un-normalised pixels ( #61104 )
...
Signed-off-by: godmook <cmoh4135@naver.com >
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-10-13 13:02:12 +08:00
Yoni Gozlan and Bugen Zhao
69d00bc264
[Bugfix] Accept whole-number floats for integer tool arguments in the response parser ( #61297 )
...
Signed-off-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-13 12:55:20 +08:00
dskwe and Cyrus Leung
28d679e762
[Bugfix] Preserve independent tokenizer code_revision in snapshot resolution ( #61272 )
...
Signed-off-by: Yuan Li <dskwelmcy@163.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-13 04:23:45 +00:00
Yoni Gozlan and Bugen Zhao
657a2e9417
[Rust Frontend] Accept whole-number floats for integer tool arguments ( #61299 )
...
Signed-off-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-13 12:22:48 +08:00
RAHUL KUMAR and Claude Sonnet 5.5
c5422f4c46
[Test][Renderer] Cover the reasoning -> reasoning_content mirror in chat message parsing ( #56129 )
...
Signed-off-by: pctablet505 <55033230+pctablet505@users.noreply.github.com >
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com >
2026-10-13 03:58:32 +00:00
df6c7ad031
[Core][Refactor] Move KV cache planning logic to kv_cache_planning.py ( #61183 )
...
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 03:05:39 +00:00
frida-andersson
c077dc663d
[ROCm][Perf][DSv4.1] Enable decoder SWA bounded replay ( #59039 )
...
Signed-off-by: Frida Andersson <fanderss@amd.com >
2026-10-13 02:19:14 +00:00
xiangdong and Kunshang Ji
69d2b6f82d
[CI][XPU] Standardize test job labels for intel jobs ( #51755 )
...
Signed-off-by: zengxian <xiangdong.zeng@intel.com >
Signed-off-by: xiangdong <40376367+zxd1997066@users.noreply.github.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-13 10:18:25 +08:00
Fangzhou Ai and Claude Opus 5.5
cc2a6eea1a
[ROCm][Perf][DSv4.1] Mono decode on every backbone layer ( #61082 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 10:16:50 +08:00
Woosuk Kwon and Claude Opus 5.5
f47792af7f
[Model Loader] Make dummy weight init dtype-aware ( #61134 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 18:33:26 -07:00
c6566bb9f6
[ROCm][MLA] Keep DCP prefill context in FP8 through AllGather ( #59070 )
...
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
2026-10-13 09:20:38 +08:00
98f7211162
[Attention][Spec Decode] Multi-query Triton split-K for spec-decode verify, with empty-segment/padding hardening ( #56148 )
...
Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com >
Signed-off-by: Venky <23023424+venkywonka@users.noreply.github.com >
Co-authored-by: OpenAI GPT-6 Astra <noreply@openai.com >
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-13 04:37:56 +04:00
189bbfb9f4
Add MoNe experts pruning support ( #50415 )
...
Signed-off-by: Ilya Boytsov <ieboytsov@nebius.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Ilya Boytsov <ieboytsov@nebius.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 23:18:43 +00:00
1484266938
[Perf] Improve _topk_topp_kernel() for masked logits ( #61213 )
...
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-12 16:01:11 -07:00
1bc6e072b0
[Rust Frontend] Add per-request timing metrics to chat and completions ( #59314 )
...
Signed-off-by: wxin <wxin@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 15:39:10 -07:00
802feb6770
[Bugfix][KVConnector] Fix MoRI-IO WRITE-mode requests stranded in WAITING_FOR_REMOTE_KVS ( #57705 )
...
Signed-off-by: Akshay Viswanathan <4953343+akshayv@users.noreply.github.com >
Co-authored-by: Akshay Viswanathan <4953343+akshayv@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com >
2026-10-12 22:29:31 +00:00
Maroon Ayoub and Flora Feng
44f0ddfd7b
[Bugfix][Parser] Stop re-parsing the whole GLM tool argument per chunk ( #61253 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-12 21:28:35 +00:00
Almog Tavor and Robert Shaw
69d54eb078
[Docs] Fix broken llm-d.ai links on the llm-d integration page ( #61022 )
...
Signed-off-by: Almog Tavor <almogtavor@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-12 21:26:43 +00:00
Matthew Bonanni and Claude Opus 5.5
fe341a53dd
[HiSparse] Report host pool in KV cache capacity logs ( #60285 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 20:49:49 +00:00
deedc7be00
[Perf][Humming] Skip the redundant MoE input copy before the w13 GEMM ( #59340 )
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-12 20:24:12 +00:00
Andrey Talman
654b725ea9
[CI] Pace the CRCR nightly report under the relay rate limit ( #61269 )
...
Signed-off-by: Andrey Talman <atalman@fb.com >
2026-10-12 13:23:53 -07:00
Mikko Tukiainen
abad32a718
[ROCm][Perf] Fix Gemma AR+RMS prefill fallback ( #58547 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Signed-off-by: Mikko Tukiainen <mikko.tukiainen@amd.com >
2026-10-12 19:53:08 +00:00
Jiangyun Zhu
fabaecbfa5
[Misc] Expand @ZJY0516 CODEOWNERS to Mamba and KV cache ( #61181 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-12 19:32:25 +00:00
Martin Hickey
7039c7f17b
[Bugfix][Rust Frontend] Omit unset sampling params in render output ( #61220 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-12 19:31:03 +00:00
ztysth and OpenAI Codex
1bdd3fa2d5
[Bugfix][Core] Check multimodal chunk boundaries before encoder cache reuse ( #59721 )
...
Signed-off-by: ztysth <rnk.liulw@gmail.com >
Signed-off-by: ztysth <166926442+ztysth@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-12 19:25:35 +00:00
Martin Hickey
08582942d8
[Bugfix][Frontend] Honor request_id and X-Request-Id in chat and completions render ( #61236 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-12 19:17:12 +00:00
Wentao Ye
d0948f754d
[Bug] Honor spec decode draft model config ( #61240 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-12 15:13:23 -04:00
Maroon Ayoub and Claude
d6aca837c3
[Attention] Make the sparse MLA prefill workspace size configurable ( #61023 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 14:56:51 -04:00
SeongJun Lee and Ning Xie
56e3b081ae
[Bugfix][weight loader] Stop warning when a parameter only widens the weight ( #60020 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Ning Xie <andy.xning@gmail.com >
2026-10-12 14:33:31 -04:00
8055293353
[ROCm][MLA] Cut ~21x host dispatch from the MLA metadata build ( #58381 )
...
Signed-off-by: lifulu <fululi12@amd.com >
Signed-off-by: lifulu <lifulu@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: lifulu <lifulu@users.noreply.github.com >
2026-10-12 13:11:20 -05:00
c9e62f13c2
[CI][ROCm] Add AITER nightly overlay scripts ( #60883 )
...
Signed-off-by: josh schell <josh.schell@amd.com >
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
2026-10-12 13:04:37 -05:00
Woosuk Kwon and Claude Opus 5.5
e1a5086256
[Attention] Remove cascade attention ( #61126 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 10:41:07 -07:00
5b0b8f251e
[Bugfix][HiSparse] Fix TP group hang on steps without scheduled tokens ( #60941 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 17:38:12 +00:00
a7b7bc1fbe
[Metrics] Count finished requests by watermark status ( #58715 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-12 17:21:22 +00:00
Misha Goin and Claude Opus 5.5
45bb710eca
[CI] Add nightly Python 3.14 smoke test ( #60456 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 10:11:26 -07:00
Laith Sakka and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
12e6a585ae
[Fix][Compile] Declare group_shape in the static FP8 quant fusion … ( #61108 )
...
Signed-off-by: Laith Sakka <lsakka@meta.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 01:09:57 +08:00
Chester
c599b5971f
[Bugfix][Frontend] Return all logprobs when completions requests logprobs=-1 ( #55717 )
...
Signed-off-by: Chester <42577861+superdiaodiao@users.noreply.github.com >
2026-10-12 17:05:55 +00:00
haotiaa
6e6e86078a
[Bugfix][Platform] Don't resolve current_platform during import vllm ( #60766 )
...
Signed-off-by: Haotian An <haotiaa@amazon.com >
2026-10-12 17:05:34 +00:00
fa39582756
[Frontend] Serve LiquidAI/d1-3B on /v1/systemone ( #60676 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 13:00:47 -04:00
Misha Goin and Claude
d79b52b182
[Misc] Consolidate csrc shared headers and prune dead code ( #60721 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 09:46:22 -07:00
e0317b28ce
[Feature] Support batch invariance on ROCm ( #52231 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-12 10:53:26 -05:00
TJian
50b21a2667
[Perf] [Feat] [ROCm] Add densemha support to ROCm AITER Sparse MLA ( #49263 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-10-12 15:50:28 +00:00
Nick Riasanovsky
a686c8e1b7
[Triton] Migrate Triton kernels off tl.make_block_ptr for triton-main compat ( #60502 )
...
Signed-off-by: Nick Riasanovsky <njriasan@meta.com >
2026-10-12 11:38:43 -04:00
Shang Wang and Claude Opus 5.5
ffec221276
[Bugfix][DSv4.1] Match decoder-replay input_ids dtype for FlashInfer MegaMoE ( #61041 )
...
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 14:23:00 +00:00
Hoai Vo and Claude Opus 5.5
f3cb10b690
[Model][LoRA] Enable LoRA support for BERT and RoBERTa embedding models ( #61033 )
...
Signed-off-by: hoaivoke <vokehoai@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 14:09:53 +00:00
Shang Wang and Claude Opus 5.5
716961cf1f
[Bugfix] Drop enable_in_kernel_fc2_reduce from FlashInfer CuteDSL MegaMoE config ( #59900 )
...
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 09:05:09 -05:00
Ilya Markov
a281ad32f0
[EPLB] Add EPLB support to dspark draft models (DeepSeekV4) ( #57549 )
...
Signed-off-by: Markov Ilya <markovilya197@gmail.com >
2026-10-12 10:03:09 -04:00
Ilya Markov
b3ee560ae8
[Bugfix][MoE] Apply expert_map remapping for CUTLASS FP4 MOE expert parallel ( #59042 )
...
Signed-off-by: Markov Ilya <markovilya197@gmail.com >
2026-10-12 10:02:56 -04:00
bzsuni
f022975964
[Bugfix] Reject beam search in completion render ( #61212 )
...
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io >
2026-10-12 13:49:46 +00:00
4c36d21280
[Bugfix][LoRA] Apply PEFT rank_pattern / alpha_pattern scaling per module ( #59801 )
...
Signed-off-by: suraj <surajsharan85@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-12 21:49:30 +08:00
3facf822b7
[Performance] Load HF tokenizers and processors from a local snapshot ( #60825 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-12 13:45:54 +00:00
092c35dd15
[Model] Support Cloudflare/clef-omni on /v1/systemone ( #61064 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-12 21:27:34 +08:00
DimensionSTP and Artem Perevedentsev
314ab8cffd
[Bugfix][Structured Output] Drain compilation tasks before releasing the backend ( #61047 )
...
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-12 16:25:36 +03:00
Chauncey
ac28877f0e
[Bugfix] Fix out-of-bounds lookahead in adaptive block verification ( #60945 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-10-12 09:21:35 -04:00
stefankoncarevic
8c175f403a
[CI][ROCm] Compare greedy output only in the AITER attn+quant test ( #61185 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-12 08:20:35 -05:00
Rohan Potdar and Claude Opus 5.5
7900d2eabc
[Test] Accept tied tokens at the k-th value in the Triton top-k test ( #60726 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 08:18:44 -05:00
Harry Huang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
c75cb63cd2
[Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next ( #56322 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-12 20:56:44 +08:00
Tianyu Guo
258b97ff6b
[Bugfix][EC Connector] Size transfers by measured encoder output widths ( #57904 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-10-12 12:32:39 +00:00
Liana Koleva and Bugen Zhao
aa1d44df16
[Bugfix][Rust Frontend] Alias qwen3_xml to qwen3_coder and add qwen3_json for the Qwen JSON format ( #60136 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-12 20:20:53 +08:00
8d7b93fec2
[Frontend] Report stop_reason on /inference/v1/generate and pass it through derender ( #60251 )
...
Signed-off-by: Hyeonki Hong <hhk7734@gmail.com >
Signed-off-by: Shimi Bandiel <shimib@google.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Hyeonki Hong <hhk7734@gmail.com >
Co-authored-by: Mimir <mimir@nousresearch.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-12 19:37:53 +08:00
32b05addc1
[Bugfix][Rust Frontend] Restore DP wave delivery after XPUB reconnects ( #60249 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 19:12:43 +08:00
Reid and Bugen Zhao
f5cda48c90
[Bugfix][Rust Frontend] Export corrupted request metric ( #59937 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-12 19:12:39 +08:00
3b9e462774
[Bugfix][Rust Frontend] Correct DP routing after request removal ( #59631 )
...
Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 19:12:35 +08:00
21445806bd
[Rust Frontend] Support logprob_token_ids on OpenAI endpoints ( #59549 )
...
Signed-off-by: Wang Zupeng <zupenwang@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 19:12:30 +08:00
Thang Nguyen and Claude Opus 5.5
ee8f7ea449
[CI] Enroll a fourth batch of five steps in automatic sharding ( #60858 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 03:46:07 -07:00
MatthewCYM
f7a3a7265f
[Bugfix][Multimodal] Fix Qwen3-ASR audio preprocessing parity ( #57146 )
...
Signed-off-by: MatthewCYM <chenyiming0v0@gmail.com >
2026-10-12 10:42:51 +00:00
Jimmy Wang and Juan Pérez de Algaba
2534a96fcf
[Docs] List additional unauthenticated endpoints ( #61081 )
...
Signed-off-by: Jimmy Wang <53066820+JimmyWang0417@users.noreply.github.com >
Co-authored-by: Juan Pérez de Algaba <124347725+jperezdealgaba@users.noreply.github.com >
2026-10-12 11:47:52 +02:00
Bugen Zhao and Claude Opus 5.5
5a09062632
[Rust Frontend] Compose answer constraints into parser-owned output grammars ( #60781 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 09:43:48 +00:00
0466bd36ea
[Rust Frontend] Build the Kimi K3 full-output grammar from its parser mode ( #59190 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-12 09:43:47 +00:00
Shijin Zhang and Chauncey
83773ecdf3
[Frontend] Support CustomToolCall in the non-Harmony Responses API ( #57322 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-12 17:22:55 +08:00
zhangchengzhucufe-dev and Kunshang Ji
a33d46b471
[Bugfix][XPU] Forward softcap and alibi_slopes in xpu_ops.flash_attn_varlen_func ( #58936 )
...
Signed-off-by: zhangchengzhucufe-dev <zhangchengzhucufe@gmail.com >
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-12 17:10:55 +08:00
Yan Ma
e05201aa2e
[XPU] refine memory release for sleep mode ( #61158 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-10-12 16:59:38 +08:00
Thuongvu Ho and Nicolò Lucchesi
99b44c7a42
[Metrics] Add admission rejection metrics ( #55812 )
...
Signed-off-by: Thuongvu Ho <thuongvuho@gmail.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
2026-10-12 08:31:07 +00:00
Cyrus Leung
5bed289707
[DX] Update CODEOWNERS for entrypoints ( #60984 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-10-12 08:06:47 +00:00
4163a80142
[Rust Frontend] Return weight-version spans on final outputs ( #61070 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 15:56:59 +08:00
Ting SUN
f4114e068e
[Bugfix][Frontend] Preserve token IDs in chat derender responses ( #61065 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-10-12 07:55:06 +00:00
AMD-yanfeiwang and aoshen02
e3bbaf2369
[ROCm][DSv4] Fuse the shared expert into AITER MegaMoE ( #61004 )
...
Signed-off-by: Yanfei Wang <yanfei.wang@amd.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-12 15:53:16 +08:00
Giancarlo Delfin and Rishi Puri
36f3972812
[Spec Decode] Skip metadata build for dflash/dspark when all model backends support it ( #60692 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Rishi Puri <20074092+puririshi98@users.noreply.github.com >
2026-10-12 00:09:10 -07:00
Zhenzhong Xu and Kunshang Ji
6a70913590
[XPU] bump up auto-round-lib to 0.16.0 ( #60976 )
...
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-12 14:47:59 +08:00
74f4404f58
[Bugfix][Distributed] Own SHM out-of-band buffers before releasing read slots ( #61000 )
...
Signed-off-by: Hubert Marek <163992334+hubert-marek@users.noreply.github.com >
Signed-off-by: Ao Shen <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Ao Shen <aoshen@inferact.ai >
2026-10-12 01:44:12 -04:00
Canlin Guo and Claude Opus 5.5
fe4df3df0e
[Bugfix][GLM-5.3-Flash] Restore MTP index sharing across draft iterations ( #61079 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 05:41:52 +00:00
d4e92ccc37
[Bugfix][Frontend] Honor endpoint plugin route overrides ( #58839 )
...
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-12 03:58:15 +00:00
liuzhenwei and Kunshang Ji
bfa7e5b7af
[XPU][CI] Enable XPU graph in graph tests ( #60784 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-12 10:39:40 +08:00
Yan Ma
f147949be8
[XPU] limit FA kernel block size to multipleOf 64 ( #57297 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-10-12 10:25:32 +08:00
dingdangmao and Codex
124b89d5b7
[Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries ( #59962 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-12 02:08:14 +00:00
Michał Ganczarenko and Claude Opus 5.5
03091e28b8
[Bugfix] Read Ling quantization_config from top-level config for Bailing V3 VL ( #59026 )
...
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 09:25:29 +08:00
Laith Sakka
cc9b0f097d
[Fix] MiniMax: fix CUDA IPC handle exchange on cuda-bindings >= 13.4 ( #60917 )
...
Signed-off-by: Laith Sakka <lsakka@meta.com >
2026-10-11 20:38:56 -04:00
Nick Hill
3fe917abb1
[Bugfix][MRV2] Fix FlashInfer attention and ReplaySSM warmups on MRV2 ( #61013 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-11 16:04:41 -07:00
f26d9f432c
[Bugfix][GLM-5.3-Flash] Honor --mamba-ssm-cache-dtype for the KDA recurrent state ( #59147 )
...
Signed-off-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-11 17:55:00 +00:00
Nick Hill
c059bee812
[MRV2] Switch CI tests still running on MRV1 unnecessarily ( #61014 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-11 09:57:07 -07:00
Ting SUN and Cyrus Leung
b99fdbe1f8
[Bugfix] Respect the configured offline chat template ( #61061 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-11 16:27:21 +00:00
93d2cab35e
[Security] Fix P2P ZMQ socket exhaustion via unbounded peer sessions ( #51504 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
2026-10-11 11:39:11 -04:00
aoshen02 and Codex
239a3fa3a8
[Sleep] Retain draft-private weights with existing parameter selector ( #61046 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 23:30:56 +08:00
3e8e1364ff
[Perf][GLM-5.3-Flash] Shard long-context indexer prefill rows across TP ( #54951 )
...
Signed-off-by: Zheng Cai <8370601+zigzagcai@users.noreply.github.com >
Signed-off-by: Zheng Cai <caizheng1993@gmail.com >
Signed-off-by: Codex <noreply@openai.com >
Co-authored-by: root <root@rjob-fb29567b88fec0a9-88c9a4c797eb2ec1-0.caizheng.ailab-sys.svc.pjlab.local >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-11 10:43:37 -04:00
aoshen02
bd5bbce6e4
[RL] Release weight-transfer communicators without stopping inference ( #61060 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: fh3 <a@b>
2026-10-11 21:15:14 +08:00
tc-mb and Cyrus Leung
c85616efc6
[Model] Support MiniCPM-V 4.7 ( #58674 )
...
Signed-off-by: tc-mb <tianchi_cai@icloud.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-11 10:21:10 +00:00
Subhash Polisetti
f425a21208
[Bugfix][Frontend] Support scalar priority and wire priority into LLM.chat() ( #57243 )
...
Signed-off-by: subhash polisetti <subhashr161347@gmail.com >
2026-10-11 10:03:33 +00:00
2339bd39d6
[Bugfix][Mooncake] Use the EAGLE replay boundary when retaining sliding-window tails ( #61012 )
...
Signed-off-by: Stu Cao <stu@primeintellect.ai >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 09:15:18 +00:00
aoshen02 and OpenAI Codex
1789f09588
[Frontend] Explain disabled scale-out endpoints with opt-in hints ( #61042 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-11 08:58:07 +00:00
Juan Pérez de Algaba and Cyrus Leung
fcae5c9887
[Security] Restrict default CORS origins ( #61009 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-11 08:26:02 +00:00
Xun Sun
26a488ad93
[Fast Start][Bugfix] Snapshot GPU memory after external weights are ready ( #60966 )
2026-10-11 07:10:04 +00:00
ef42b4385f
[Bugfix] Reclaim cached GPU memory during weight loading ( #57516 )
...
Signed-off-by: Debanshu Das <debanshu@google.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-10-11 07:03:06 +00:00
Bruno Volpato and Codex
61f084b5f3
[Bugfix][Multimodal] Fix per-video cache option reuse ( #55203 )
...
Signed-off-by: bvolpato <brunocvcunha@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 06:21:55 +00:00
Yongye Zhu and Claude Opus 5.5
2eb491e18b
[Perf][MiniMax-M3] MSA prefill attends with the E4M3 query; bump fmha_sm100 to 968061a ( #60684 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 13:40:45 +08:00
Nick Hill
f8471164bf
[MRV2] Detatch weight-reload logic from MRV1 ( #60937 )
2026-10-10 21:15:05 -07:00
bzsuni
01abe975b5
[Bugfix][KV Connector] Fix speculative block reservations across KV cache groups ( #59880 )
...
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io >
2026-10-10 20:55:41 -07:00
Shuolei Wang and aoshen02
b89852aad0
[RL][Frontend] Return token-level weight-version spans on final outputs ( #59582 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Shuolei Wang <948904026@qq.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-11 11:33:29 +08:00
3c062d391d
[Bugfix] Support 0-dim tensors in packed weight transfer ( #52235 )
...
Signed-off-by: subhash polisetti <subhashr161347@gmail.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 01:41:06 +00:00
Wentao Ye
276fbcff27
[Deprecation] Remove scheduled deprecated items ( #60923 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-10 15:45:33 -04:00
Yufeng He
187a0eb98a
[Bugfix][Frontend] Forward --enable-log-outputs to the Anthropic handler ( #60982 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
2026-10-10 16:36:03 +00:00
f2376cdc4d
[Fast Start] Preload the FlashInfer autotune table on the weight cache daemon ( #60085 )
...
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-10 15:48:06 +00:00
Wentao Ye
3709632ff2
[Refactor] Remove dead code multiple places ( #60120 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-10 15:41:10 +00:00
Wentao Ye
838526c7e5
[Compile] Fix sampler compile warning ( #60680 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-10 08:09:45 -07:00
Mook
bf16c9c139
[Bugfix][Model] K-EXAONE: restore loading of pre-2.0 configs and apply RoPE to the MTP layer ( #60187 )
...
Signed-off-by: godmook <cmoh4135@naver.com >
Signed-off-by: Mook <68294499+Godmook@users.noreply.github.com >
2026-10-10 15:05:55 +00:00
Tony Lin
2bea22002d
[BUG] Fix KV offload shared-memory slot selection using worker rank ( #58497 )
...
Signed-off-by: Tony Lin <tony.lin@intel.com >
2026-10-10 20:25:11 +08:00
Kay Yan and Codex
c41b2639e2
[Bugfix][Responses API] Complete cache-write token accounting ( #60973 )
...
Signed-off-by: Kay Yan <kay.yan@daocloud.io >
Co-authored-by: Codex <noreply@openai.com >
2026-10-10 09:08:36 +00:00
SeongJun Lee and lesj0610
a98247ab4d
[Model] Drop the 4x M-RoPE cache headroom for Qwen3-VL, Qwen3.5 and Qwen3.8 ( #59853 )
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-10-10 06:51:52 +00:00
wang.yuqi
c1692c85f1
[Misc] mv chat_utils.py to renderers. ( #60522 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-10 06:50:34 +00:00
SeongJun Lee and Isotr0py
699c892dd5
[Perf] Skip the M-RoPE warmup when every holder of the RoPE fuses it ( #60246 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-10 06:47:39 +00:00
liuzhenwei and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ff440c7805
[BreakableCG] Add eager break for Mamba-family ops ( #57881 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-10 14:06:57 +08:00
Andreas Karatzas and Claude
a205fb3177
[Bugfix][Model] Restore AXK1 config defaults dropped with the vendored config ( #60954 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-10 05:54:52 +00:00
devpramod and Claude Opus 5.5
46fb84c2d4
[Model] Support EAGLE3/DFlash aux hidden states for Granite ( #60855 )
...
Signed-off-by: Pai, Pramod <pramod.pai@intel.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-10 13:34:10 +08:00
Chauncey
fa5cfd48d5
[Refactor] Deduplicate trace header handling in BaseServing ( #60828 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-10-10 05:27:58 +00:00
Nick Hill
7d0b4e57aa
[ROCm] Remove MRV1 model exceptions ( #56360 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-10 05:09:19 +00:00
8ea646b827
[Rust Frontend][gRPC] Remove GenerateStream producer task so disconnects abort requests ( #60922 )
...
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-10-10 04:59:35 +00:00
linitra24
364bc81a9e
[Bugfix][LoRA] Fix GPT-OSS gate/up interleaving with mixed MoE LoRA formats ( #60812 )
...
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-10 12:52:51 +08:00
Puneet Sharma
5ca26880ad
Optimize ppc64le vLLM runtime image size ( #48074 )
...
Signed-off-by: puneetsharma21 <puneet.sharma21@ibm.com >
2026-10-10 12:39:58 +08:00
Guancheng Fu and Kunshang Ji
acec1b1d84
[Bugfix][XPU] Remove stale W4A8 weight after repacking ( #57511 )
...
Signed-off-by: Guancheng Fu <guancheng.fu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-10 04:27:35 +00:00
Matthew Bonanni and Claude Opus 5.5
d5f356dae0
[HiSparse] Own GPU residency instead of editing core block-table rows ( #60647 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 21:25:44 -07:00
SeongJun Lee and Misha Goin
9bd074a996
[Attention] Decode an NVFP4 KV cache with XQA on SM12x ( #60452 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-09 22:24:00 -06:00
Misha Goin and Claude Opus 5.5
29e4d2d42e
[Frontend][Build] Add vllm download-kernels for FlashInfer precompiled kernels ( #58765 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 21:22:35 -07:00
Miłosz Grunwald and Kunshang Ji
be0823357f
[TEST][XPU][CI] Fix ci uva kernel test instability 2 ( #60404 )
...
Signed-off-by: microslaw <milosz.grunwald@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-10 12:09:13 +08:00
Giulio De Pasquale
f1a3c60558
[Feature][Attention] Cover a whole spec-verify in one DiffKV split-KV program ( #59085 )
...
Signed-off-by: Giulio De Pasquale <git@depasquale.giugl.io >
2026-10-10 11:55:45 +08:00
2d69083c4a
[ROCm][MXFP4] Align the AITER W4A4 MoE intermediate size to 128 ( #55104 )
...
Signed-off-by: xiaofei.zheng <xiaofei.zheng@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
2026-10-10 11:14:01 +08:00
DimensionSTP and OpenAI Codex
10cc2f6ae2
[Bugfix] Bound GDN speculative decode query widths ( #55504 )
...
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-10 11:00:16 +08:00
02aa10539f
[Bugfix][ModelLoader] Make enable_weights_track check quantized layers ( #57108 )
...
Signed-off-by: siliangchen-amd <SiLiang.Chen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-10 10:44:06 +08:00
aoshen02 and Claude Opus 5.5
1dfe9fbdae
[Core] Release FlashInfer allreduce workspace with --enable-nccl-comm-suspend ( #59360 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 02:26:40 +00:00
Fangzhou Ai and Claude Opus 5.5
193922d6ff
[ROCm][Perf][DSv4.1] Mono decode layer for DeepSeek-V4.1-Flash on MI355X ( #60397 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 20:53:18 -05:00
3c802af3cf
[Feat][Frontend] structured decisions: DiffusionGemma read strategy ( #60466 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-10 09:33:22 +08:00
Nick Hill
7d1d8660d6
[MRV2] Miscellaneous code structure updates ( #60754 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-09 17:58:09 -07:00
9f1c783886
[Test] Allow one fp8 step on K bytes in the kpool decode fuzz test ( #60725 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-10 00:26:53 +00:00
f8e6e365ae
[Bugfix][Mooncake] Save the EAGLE attention proof with hybrid Mamba partial tails ( #60695 )
...
Signed-off-by: Dao Le <Dao007forever@gmail.com >
Signed-off-by: Dao007forever <daole@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-10 00:08:08 +00:00
Wei Zhao and Codex
03a7d3776f
[Perf] Support minimum KV splits configuration for TokenSpeed MLA decode ( #60345 )
...
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-09 23:33:47 +00:00
04b02fc8a5
[Core] shm tensor arena for efficient cpu->gpu worker broadcast ( #51207 )
...
Signed-off-by: Bolin Sun <bolins@nvidia.com >
Signed-off-by: <bolins@nvidia.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-09 16:13:24 -07:00
Rohan Potdar and Claude Opus 5.5
fa94ad4143
[Test] Allow one bf16 ulp in the fused mHC post/pre residual check ( #60724 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 17:54:06 -05:00
Matthew Bonanni and Claude Opus 5.5
4c43254d52
[HiSparse] Report host-pool block residency as HiSparse metrics ( #59485 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 16:45:34 -06:00
Kevin H. Luu and Claude Opus 5.5
783ac12071
[CI] Accept whitespace around /ci commands and reply instead of going silent ( #60925 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 15:43:37 -07:00
ed7ae705a1
[Bugfix][MRV2] Use int64 index mappings to fix row-offset overflow ( #60736 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-09 14:38:11 -07:00
51144dd83e
[Bugfix][Spec Decode] Trim token_ids/logprobs left past a stop string under speculative decoding ( #47616 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 14:22:31 -07:00
Bugen Zhao and Claude Opus 5.5
35e920d671
[Rust Frontend] Support --stream-interval and per-request stream_interval ( #60589 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 14:10:42 -07:00
Giancarlo Delfin
ffdde24f54
[Docs] Document adaptive verification without a confidence head ( #60913 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-10-09 13:54:40 -07:00
Kai Hoang and Flora Feng
e0cb3efce0
[Bugfix][Structured Output] Decode choice specs as JSON in outlines and LMFE ( #60748 )
...
Signed-off-by: hpdkhoa <hpdkhoa2311@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-09 20:52:08 +00:00
02e55d2af3
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots ( #57169 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-09 15:48:37 -04:00
Misha Goin and Claude
3766a67a32
[WideEP] Explain how to fix a too-old NCCL for DeepEP v2 ( #60449 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-09 12:34:58 -07:00
04462732a4
[Bugfix] Reject max_num_scheduled_tokens=0 at config time instead of hanging ( #60714 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Hui Li <40770106+untrall@users.noreply.github.com >
2026-10-09 19:15:10 +00:00
Willow Lopez and Artem Perevedentsev
ad8a6065f4
[Bugfix] Validate guidance tokenizer compatibility before EngineCore ( #52298 )
...
Signed-off-by: Oxygen56 <jiangth99@163.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-09 21:59:02 +03:00
Misha Goin and Claude
1dc60dd367
[CI] Pin pydantic in the mypy pre-commit hook ( #60894 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-09 11:42:41 -07:00
Ning Xie and Simon Mo
a0f28d73a8
[sharded state loader] support pp in sharded state loader ( #52110 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
Co-authored-by: Simon Mo <simon.mo@hey.com >
2026-10-09 11:08:31 -07:00
premsurawut and Claude Opus 5.5
c811943137
[Bugfix][Frontend] Reject beam search with echo and logprobs in /v1/completions ( #60876 )
...
Signed-off-by: surawut.j <surawut.jirasaktavee@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 18:06:26 +00:00
Bugen Zhao and Claude Opus 5.5
5dc9bcd8cd
[Rust Frontend] Pass reasoning controls through render -> generate ( #60546 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 11:04:49 -07:00
Bugen Zhao and Claude Opus 5.5
ce4603cad6
[Bugfix][Rust Frontend] Pass through raw characters in byte-level decode ( #59861 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 11:04:00 -07:00
Bugen Zhao and Claude Opus 5.5
9a585c62aa
[Rust Frontend] Box rarely-set fields of per-token output types ( #60588 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 11:01:16 -07:00
1a3a7a8f01
[Bugfix] Treat max_tokens as output upper bound, not input-space reservation ( #57035 )
...
Signed-off-by: YuvrajSinghBhadoria2 <245782664+YuvrajSinghBhadoria2@users.noreply.github.com >
Signed-off-by: YuvrajSinghBhadoria2 <yuvrajsinghbhado2030@gmail.com >
Signed-off-by: Yuvraj Singh Bhadoria <yuvrajsinghbhado2030@gmail.com >
Co-authored-by: YuvrajSinghBhadoria2 <245782664+YuvrajSinghBhadoria2@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-10-10 01:15:22 +08:00
Andrey Talman and Andrey Talman
89ac8d971b
[CI][Test] Allow 10% ngram acceptance drop under preemption on CUDA in test_async_scheduling_accuracy ( #60715 )
...
pre-commit / pre-commit (push) Canceled after 0s
pre-commit / pre-run-check (push) Canceled after 0s
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com >
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com >
2026-10-09 16:59:17 +00:00
stefankoncarevic
b74233860d
[CI][ROCm] Wait for VRAM to settle before building LLM in test_full_graph ( #60872 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-09 16:48:43 +00:00
Taewoong Kim and Cyrus Leung
75e3b17c05
[Frontend] Add cache salt support to generative scoring ( #60841 )
...
Signed-off-by: Taewoong Kim <ktw2172@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-09 16:07:50 +00:00
90ba2f34b5
[Build] Find compatible published wheels for precompiled installs ( #58871 )
...
Signed-off-by: greg pereira <grpereir@redhat.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 08:05:54 -07:00
Abhijit Roy
970dc63478
[Docs] Add Giga-Embeddings bidirectional Qwen3 usage to embed docs ( #60854 )
...
Signed-off-by: Abhijit <abroy@redhat.com >
2026-10-09 14:41:38 +00:00
Taneem Ibrahim
a6b39f0262
[Bugfix] Clean up async client sockets after event loop closure ( #57757 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-10-09 10:32:47 -04:00
Warren Deng and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
fc39317392
[Bugfix][Kernel] Remove invalid NC from Kimi-K3 KDA autotune keys ( #60446 )
...
Signed-off-by: Warren Deng <warrdeng@meta.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 14:25:41 +00:00
Juan Pérez de Algaba and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
3d52578e13
[Security] Do not retain Phi-4 MM audio attention matrices ( #60614 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 22:23:35 +08:00
Mikko Tukiainen and Cursor
81c993658e
[ROCm][Perf] Fuse main QK-norm/RoPE/gate and KV-cache write into the AMD QSA prepare launch ( #59732 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-09 21:14:49 +08:00
Li, Jiang
9adba10f3f
[Bugfix] Fix pre-commit check ( #60840 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-10-09 12:48:32 +00:00
Suraj Sharan and linitra24
1ee69dde9c
[Bugfix][LoRA] Reject PEFT adapter features vLLM does not implement ( #60225 )
...
Signed-off-by: suraj <surajsharan85@gmail.com >
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-09 16:39:58 +04:00
Yan Ma
c17fadcfdc
[XPU] fix fused_input_norm for MM ( #59865 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-10-09 20:16:35 +08:00
YiSheng5 and Kunshang Ji
0cc0460870
[XPU]Fix the accuracy issue when meet topk_ids=-1 on DP+EP scenarios ( #57787 )
...
Signed-off-by: yisheng <yi.sheng@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-09 20:01:54 +08:00
Louie Tsai
196483e25b
[Tools] Recipes sweep workflow ( #57307 )
...
Signed-off-by: louie-tsai <louie.tsai@intel.com >
2026-10-09 19:44:09 +08:00
Praneeth Reddy A and Li, Jiang <jiang1.li@intel.com>
1c47ebfc57
[Docs] Fix VLLM_CPU_KVCACHE_SPACE default and units ( #60053 )
...
Signed-off-by: Praneeth <praneethreddyarikatla@gmail.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-10-09 19:43:55 +08:00
Chen Cheng and Isotr0py
d095e61c0b
[Bugfix][Spec Decode] Fix GLM-5.3-Flash DFlash2/EAGLE3 aux hidden-state capture with the upstream config ( #60763 )
...
Signed-off-by: Chen Cheng <85005591+ischencheng@users.noreply.github.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-09 11:35:48 +00:00
wang.yuqi
7368fadebd
[CI Failure] Move MTEB tests to h200_35gb ( #60817 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-09 11:34:38 +00:00
Luca Motz and Claude Opus 5.5
83ed0c7c09
[Bugfix][NIXL] Fix receive post-process for different P/D block sizes ( #58860 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 19:30:11 +08:00
Shreyansh Jain and Artem Perevedentsev
28d76f3bad
[Bugfix][Structured Output] Reject unsupported regex inside groups ( #60359 )
...
Signed-off-by: Shreyansh Jain <shreyanshjain646@gmail.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-09 14:16:13 +03:00
Chaojun Zhang
b027ac8312
[XPU] Support register KV offload mmap region as pinned host memory on XPU ( #51956 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
2026-10-09 18:58:07 +08:00
5cadabfafd
[Docs] Add Qwen2.5-Coder-7B-Instruct to batch invariance tested models ( #46368 )
...
Signed-off-by: Yuval Luria <yluria@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-09 10:41:29 +00:00
Yan Ma
4e318ac563
[XPU] fix online fp8_per_channel quantization ( #56027 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-10-09 18:38:01 +08:00
Thang Nguyen and Claude Opus 5.5
efd141047d
[CI] Enroll a third batch of five steps in automatic sharding ( #60757 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 03:26:47 -07:00
Thang Nguyen and Claude Opus 5.5
b8bd335754
[CI] Enroll five more steps in automatic sharding ( #60662 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 03:26:27 -07:00
Warren Deng
59ffd73e85
[Bugfix][Kernel] Restore FP fusion for indexer Q quantization ( #60716 )
...
Signed-off-by: Warren Deng <warrdeng@meta.com >
2026-10-09 09:46:41 +00:00
Damien Laine and Cyrus Leung
2186b89a9a
[Bugfix][Model] Voxtral realtime: fix boot OOM and engine crash at max_model_len ( #45022 )
...
Signed-off-by: Damien Laine <damien.laine@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-09 17:40:29 +08:00
Harry Mellor and Claude Opus 5.5
2c99ee9333
[Chore] Remove vendored code made redundant by Transformers 5.16.1 ( #60623 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 09:29:03 +00:00
Chao Lei
bc713cf8cc
[Bugfix]MooncakeConnect add query timeout and retry ( #55923 )
2026-10-09 09:19:33 +00:00
Taewoong Kim and Chauncey
18c19eaf23
[Frontend] Add cache salt and tracing to structured decisions ( #60786 )
...
Signed-off-by: Taewoong Kim <ktw2172@gmail.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-09 08:53:50 +00:00
a6f43e57d6
[ROCm] Enable the cuMem CUDA-graph pool offload on ROCm ( #59523 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: indianspeedster <cspandey016@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: indianspeedster <cspandey016@gmail.com >
2026-10-09 08:29:55 +00:00
The Anh Nguyen
bacbbe1878
[Bugfix][NIXL] Guard record_transfer against None telemetry ( #57734 )
...
Signed-off-by: The Anh Nguyen <ntheanh201@gmail.com >
2026-10-09 08:22:18 +00:00
Sriram Kumar
560fae64cd
[ROCm][Bugfix] Pass per-sequence context lengths to the AITER paged MQA-logits kernels ( #59420 )
...
Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com >
2026-10-09 08:16:20 +00:00
Maroon Ayoub and Claude
8a03ccf7b3
[Frontend] Add return_mm_kwargs to render requests ( #60728 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-09 08:14:13 +00:00
f43e4c39e3
[Structured Output] Allow propertyNames with additionalProperties on xgrammar ( #57594 )
...
Signed-off-by: errmakov <ide404@gmail.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-09 10:49:47 +03:00
efaca9547d
[Perf][DSv4.1] Exchange packed FP8 Engram rows across DP and feed wkv MXFP8 ( #59923 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 15:30:18 +08:00
82e89e9944
[Perf][Qwen4Exp] Retune H200 M=4 merged QSA LL-GEMM plan ( #60776 )
...
Signed-off-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-10-09 07:18:50 +00:00
19c0bed584
[Fusion] Support 3D activation quantization fusion and enable it for Pixtral ( #59612 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-09 15:12:03 +08:00
dd30786fef
[CI] Enable Zen CPU image build on AMD CI ( #60016 )
...
Signed-off-by: Chinmay Kulkarni <Chinmay.Kulkarni@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
2026-10-09 01:40:21 -05:00
Paweł Mańczak and Kunshang Ji
f0a5f111f2
[XPU] Add tuned Mamba SSU configs for Intel Arc Pro B60 ( #56765 )
...
Signed-off-by: pmanczak <pawel.manczak@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-09 14:29:29 +08:00
ad73a4740d
[Perf][DSv4.1] Trim the last KV-source layer in decoder replay ( #59894 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Juntian Liu <Juntianl777@gmail.com >
2026-10-09 06:07:49 +00:00
8cbd5d0300
[Frontend] structured decisions: noul and score question types ( #60696 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Co-authored-by: DeepSeek <noreply@deepseek.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-09 04:48:16 +00:00
Rohan Potdar and Claude Opus 5.5
88a419649e
[ROCm][Kimi-K3] Use vLLM's FP8 max in the fused AttnRes quantization ( #60717 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 04:45:13 +00:00
fa312090cb
[Frontend] Accept each connection on the least loaded API worker ( #60723 )
...
Signed-off-by: hao-aaron <aaronhao@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 04:40:13 +00:00
Rajeev Warrier
81d9c45e4b
[Docs] [Multimodal] Note that per-request media_io_kwargs needs --trust-request-mm… ( #60598 )
...
Signed-off-by: Rajeev Warrier <warrier.rajeev.95@gmail.com >
2026-10-09 04:23:07 +00:00
e31b823bf5
[Bugfix][Tool Parser] Use LongCat tags in the LongCat structural tag ( #60747 )
...
Signed-off-by: hpdkhoa <hpdkhoa2311@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 21:20:54 -07:00
9824159d94
[Bugfix] Fix InternLM2 tool parser dropping characters from streamed arguments ( #60709 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 21:20:49 -07:00
Vineeth Sai Varikuntla and Flora Feng
112a282915
[Bugfix][Frontend] Keep streamed completion text_offset in step with tokens ( #60668 )
...
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 21:20:43 -07:00
51ef239768
[Rust Frontend] Carry reasoning controls over gRPC ( #59836 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 11:11:19 +08:00
Rémi Delhaye
a1cd0ec650
[Model] Add zerank-2 reranker ( #60415 )
...
Signed-off-by: rdlhio <contact@rdlh.io >
2026-10-09 03:01:51 +00:00
zzaebok and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
702ce313f4
[Perf][Multimodal] Sort multimodal encoder inputs by modality ( #60196 )
...
Signed-off-by: Jaebok Lee <jaebok9541@naver.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 10:55:36 +08:00
Tiezhen WANG and Kevin H. Luu
295c062db8
Remove sudo from dockerfile command ( #60488 )
...
Signed-off-by: Tiezhen WANG <38108242+xianbaoqian@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-08 19:24:56 -07:00
Mia Ojeda and Lucas Wilkinson
4611c2c3d2
[BugFix][DiffusionGemma][CPU] Copy async output snapshots instead of aliasing reused buffers ( #59829 )
...
Signed-off-by: ojeda-e <eojeda@league.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-08 19:16:51 -07:00
201e537d98
[Bugfix][LoRA] Validate incompatible MoE adapter formats ( #59924 )
...
Signed-off-by: Brian Weston <brianweston@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-09 09:49:13 +08:00
Nishant Revur and Claude
581692cc0f
[Bugfix][Spec Decode] Fix multimodal embedding merge in Llama 4 and Mistral Large 3 EAGLE drafters ( #57633 )
...
Signed-off-by: Nishant Revur <kumrfni@amazon.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-09 01:44:50 +00:00
DGlan
4274ae956c
[Bugfix][EC Connector] Release shared memory on NIXL setup failure ( #59875 )
...
Signed-off-by: DGlan <162426117+DGlan@users.noreply.github.com >
2026-10-08 17:38:16 -07:00
ab905a885d
[CI] Run GH200 test on the shared arm64 CI image ( #60253 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-08 15:22:20 -07:00
Kamran and Codex
8d10c38b73
[Perf] Reduce fallback decode CPU by 69% at 32K ( #60497 )
...
Signed-off-by: spa5k <79936503+spa5k@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
2026-10-08 21:42:56 +00:00
Yifan Qiao and Claude Opus 5.5
240785b82c
[Bugfix] Keep the GLM-5.3 kpool tail and Qwen4 QSA ring out of the null block ( #59528 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 20:44:04 +00:00
sungbin1015 and Flora Feng
2ddde19e8e
[Frontend] Support min_p in the Responses API ( #48084 )
...
Signed-off-by: sungbin1015 <sbin@solbox.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 20:38:17 +00:00
Divakar Verma
c5c1161382
[ROCm][Build] Fail fast when setup.py develop deps aren't pip-preinstalled ( #59447 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
2026-10-08 15:28:38 -05:00
ae9adec905
[Frontend] Expose /v1/systemone without an opt-in flag ( #60677 )
...
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-08 20:21:05 +00:00
72d59adc2c
[Bugfix][KV Connector][NIXL] Transfer the whole PLE short-conv page in disaggregated P/D ( #59997 )
...
Signed-off-by: Alberto Massidda <amassidda@nvidia.com >
Co-authored-by: Alberto Massidda <amassidda@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-08 23:58:13 +04:00
Matthew Bonanni and Claude Opus 5.5
1afb029839
[Bugfix][HiSparse] Route by swap-row capacity, not the index-group workspace ( #60682 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 19:45:44 +00:00
c23ca06b6d
[Bugfix][Structured Output] Feed the token that implicitly ends reasoning to the grammar ( #59627 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Yifan Zong <263738792+yzong-rh@users.noreply.github.com >
2026-10-08 19:38:33 +00:00
dingdangmao and Flora Feng
bab0ac61f8
[Bugfix][Frontend] Forward the reasoning wiring to the engine for batch chat completions ( #59263 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 19:32:46 +00:00
Misha Goin and Claude
e3a29aa9b3
[Bugfix][MoE] Size the permute scratch from its inputs ( #60447 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-08 15:32:07 -04:00
SeongJun Lee and Misha Goin
fff10d2072
[Attention] Allow an NVFP4 KV cache on SM90 with fa2 ( #60620 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 12:16:22 -07:00
Kevin Li and Misha Goin
fc494822ea
[Test][Quantization] Gate CUTLASS W4A8 kernel tests to SM90 ( #58878 )
...
Signed-off-by: Kevin Li <kevinbean99@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 19:14:35 +00:00
Divakar Verma and Andreas Karatzas
18751bc6a7
[ROCm][CI] Add coverage for AiterExperts token padding: inf/nan garbage rows ( #59334 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-08 19:11:45 +00:00
Aarushi Jain and Andreas Karatzas
9bbc581560
[Bugfix] Close MessageQueue zmq resources via finalizer to avoid exit ( #60472 )
...
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-08 13:53:00 -05:00
Yeonwoo Sung and Flora Feng
56e6da2c47
[Bugfix][Frontend] Honor stop_token_ids on the Responses API ( #56592 )
...
Signed-off-by: YeonwooSung <neos960518@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 18:47:30 +00:00
Ning Xie and Simon Mo
c4a5ff5c18
[Frontend] shutdown engine core and its sub processes asap when initializing ( #52299 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
Co-authored-by: Simon Mo <simon.mo@hey.com >
2026-10-08 11:46:01 -07:00
Shijin Zhang
2274e86842
[Spec Decode][Model] Support DFlash2 draft models with GLM-5.3-Flash ( #56983 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
2026-10-08 14:41:10 -04:00
Misha Goin and Claude
30c7405890
[MoE] MXFP4 oracle: add CUTLASS W4A4 backend and BF16 activation fallback ( #60079 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-08 11:39:09 -07:00
d8fa463e2b
[Bugfix][Prefix caching] Fix inconsistent hybrid Mamba prefix cache boundaries ( #60533 )
...
Signed-off-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 18:33:48 +00:00
1664c4f444
[Bugfix][MiniMax-M3] Guard sparse decode sentinel indices ( #54597 )
...
Signed-off-by: Kyle Liang <kylliang@nvidia.com >
Signed-off-by: Kyle Liang <kyleliang-nv@users.noreply.github.com >
Co-authored-by: Kyle Liang <kyleliang-nv@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 18:27:57 +00:00
Tialo and Misha Goin
972195f4f5
[CI][Build] build macos wheels using abi3 ( #60312 )
...
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 14:19:20 -04:00
Francisco Javier Arceo and Misha Goin
705e124c05
[Bugfix] Followup to #59299 : preserve Unicode in structured decision state prompts ( #60651 )
...
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 11:00:03 -07:00
Doug Smith
242e4213fc
[Doc] Require GCC 13 for CUDA source builds ( #60259 )
...
Signed-off-by: Doug Smith <dosmith@redhat.com >
2026-10-08 09:43:39 -07:00
Elvir Crnčević and Misha Goin
8674a794c7
[Perf] Enable Kimi K3 shared expert sharding with DeepEP v2 ( #56875 )
...
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 16:42:15 +00:00
stefankoncarevic
f4dde3132c
[CI][ROCm] Reuse AITER's prebuilt MoE module in the modular-kernel harness ( #60633 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
2026-10-08 11:18:05 -05:00
9f3d70e081
[Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) ( #55686 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: fxmarty-amd <felmarty@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 12:04:46 -04:00
Mikko Tukiainen and Cursor
e2414ec1db
[ROCm][Qwen4Exp] Use fused PLE Triton kernels on the AMD backend ( #60021 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-08 15:47:56 +00:00
Justin Barlow and Harry Mellor
0fc0cefb4e
[Bugfix] Don't int-sort attention-type names in --kv-cache-dtype-skip-layers for packed KV dtypes ( #60364 )
...
Signed-off-by: lobo235 <7@netlobo.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-08 16:34:55 +01:00
e41d5d5543
[Bugfix] Don't warn about missing watermark config for default requests ( #60412 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-08 15:29:20 +00:00
Haokai Ma and Claude
0751f5272f
[Bugfix] Scope deprecated CLI argument warnings to the parser that defines them ( #59917 )
...
Signed-off-by: Tyler Ma <158780265+TylerMa-debugg@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-08 15:29:04 +00:00
Woosuk Kwon
194bd72be7
[Compile] Remove torch.compile-based sequence parallelism and AsyncTP ( #60517 )
2026-10-08 08:28:23 -07:00
wangxiyuan and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
072d49bb6e
[Platform] Support platform-specific MLA prefill backend selection ( #56959 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 15:25:21 +00:00
fd928e5f3c
[Bugfix] Check batch-invariance support for every attention backend ( #60271 )
...
Signed-off-by: yifanFengg <evanfengff@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-08 11:19:57 -04:00
JartX
994b8b77be
[Bugfix][ROCm] RDNA3 W4A16 MoE: handle the N-first CT weight layout ( #60388 )
...
Signed-off-by: JartX <sagformas@epdcenter.es >
2026-10-08 15:06:39 +00:00
kliuae
d80ddaac73
[ROCm][Perf] Kimi-K3 Store only the current rank's shards in latent MoE up-proj ( #59591 )
...
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com >
2026-10-08 14:55:20 +00:00
d33872dfeb
[Feat] Structured decisions endpoint (/v1/systemone) ( #59299 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-08 14:48:12 +00:00
yhcheong (vllmellm)
273e099693
[Bugfix][ROCm] Select page-aligned kernel blocks for pooled indexers so the block table addresses storage pages (GLM-5.3-Flash) ( #59412 )
...
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com >
2026-10-08 14:44:58 +00:00
zzaebok
2f7f05c256
[Bugfix][MoRIIO] Prevent remote KV reload after decoder preemption ( #53250 )
...
Signed-off-by: Jaebok Lee <jaebok9541@naver.com >
2026-10-08 22:28:16 +08:00
daa9085143
[KV Connector][NIXL] Register HiSparse host pool under single TP Rank ( #60161 )
...
Signed-off-by: Pravein Govindan Kannan <pravein.govindan.kannan@ibm.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 07:25:02 -07:00
Matthew Bonanni and Claude Opus 5.5
4ed4c4433b
[Bugfix][HiSparse] Drop the unaccounted, write-only prefill mirror staging buffer ( #60429 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 14:00:49 +00:00
Vineeth Sai Varikuntla and Harry Mellor
194da61de1
[Bugfix] Reject negative --device-ids indices instead of selecting the wrong device ( #49053 )
...
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-08 13:01:25 +00:00
458ba2edf8
[ROCm][Kimi-K3] Fuse AttnRes output with per-token FP8 input quantization ( #59069 )
...
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
2026-10-08 12:01:47 +00:00
73c742b09c
[Bugfix] Make disable_any_whitespace actually disable whitespace on xgrammar ( #58067 )
...
Signed-off-by: Stu Cao <stu@primeintellect.ai >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-08 14:41:14 +03:00
devshah-cohere and Bugen Zhao
b49c921a27
[Bugfix][Rust Frontend] Always include token_ids in completion stream choices ( #60286 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: devshah-cohere <dev.shah@cohere.com >
2026-10-08 19:11:55 +08:00
Andy Lo
ba77c4c130
[Bugfix][Attention] Preserve small FP8 softmax weights in Triton attention ( #60156 )
...
Signed-off-by: Andy Lo <andy@mistral.ai >
2026-10-08 12:35:22 +02:00
Nicolò Lucchesi
e161a0798c
[Perf][Core] Reduce free KV cache queue overhead ( #60269 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-08 10:17:11 +00:00
wang.yuqi
459764104d
[Frontend] Move offline_utils.py to entrypoints/common. ( #58052 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-08 09:56:05 +00:00
Muhammad Usman Mateen and Misha Goin
e73895a5df
[Bugfix] Make suppress_stdout redirect fd 1 instead of sys.stdout.fileno() ( #59336 )
...
Signed-off-by: usmanmateen <usmanmateen255@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 09:52:50 +00:00
Chaojun Zhang
dfaabf91ac
[XPU] [CI] Fix stale ignore path for weight_transfer_metrics tests on Intel CI ( #60558 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
2026-10-08 17:48:47 +08:00
Kevin H. Luu and Claude Opus 5.5
24be8c0bca
[CI] Remove dead in-tree TPU (torch_xla) CI scripts ( #60591 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:42:07 -07:00
Jiangyun Zhu and Claude Opus 5.5
e265b77982
[CI][Mamba] Use well-conditioned prompts for FlashInfer ReplaySSM parity tests ( #60515 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 17:39:56 +08:00
7816afdb25
[Bugfix][Core] Drain async KV loads in pause(mode="wait"), and pause the drain test only once a request is in flight ( #60572 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 09:33:41 +00:00
Kevin H. Luu and Claude Opus 5.5
c4cd88d9fe
[CI] Make (H100) Distributed DP + EP optional ( #60540 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:31:13 -07:00
Harry Mellor and Claude Opus 5.5
21d9758459
Remove Python 3.10 support now that it is EOL ( #60402 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:30:52 -07:00
liuzhenwei and Kunshang Ji
0004536dd0
[XPU][Graph] Replace boolean-mask assignment with torch.where ( #60529 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-08 17:13:48 +08:00
Nicolò Lucchesi
70bdce6133
[Model][GLM-5.3-Flash] Warm up the DCP top-k merge kernels for the kpool indexer ( #60032 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-08 11:05:52 +02:00
7d47ac2ddb
[CI] Enroll five steps in automatic sharding ( #60492 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: khluu <khluu000@gmail.com >
2026-10-08 01:52:59 -07:00
Paweł Mańczak
4cf1307e72
[XPU] Add tuned Mamba SSU configs for 2 more B70 shapes ( #57565 )
...
Signed-off-by: pmanczak <pawel.manczak@intel.com >
2026-10-08 16:36:05 +08:00
Ting SUN
0f112d1800
[Bugfix][Pooling] Preserve cache_salt for chunked embeddings ( #47696 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-10-08 07:33:28 +00:00
Alec
29ae81fe83
[Rust Frontend] Preserve explicit zero sampling values in gRPC requests ( #60115 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
2026-10-08 07:27:47 +00:00
Rehan Khan and Bob
fd0216d3b5
[Bugfix][CPU] Fix fused Gumbel-max wrapping-window bias ( #60221 )
...
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com >
Co-authored-by: Bob <bob@ibm.com >
2026-10-08 15:15:53 +08:00
ARAVINDHAN T and linitra24
e58f8dde7f
[Bugfix] Use FusedMoEWithLoRA for shared MoE LoRAs on 3D models ( #60098 )
...
Signed-off-by: ARAVINDHAN T <arvindhant01@gmail.com >
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-08 15:04:32 +08:00
xiangdong
032d39e9b4
[XPU][CI] Split Intel jobs to improve B50 utilization ( #58817 )
...
Signed-off-by: zengxian <xiangdong.zeng@intel.com >
Signed-off-by: xiangdong <40376367+zxd1997066@users.noreply.github.com >
2026-10-08 14:53:37 +08:00
stefankoncarevic and Andreas Karatzas
814301b268
[Bugfix][Quant] Read safetensors headers of checkpoints with other file names ( #60411 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-08 01:38:41 -05:00
Vadim Gimpelson and Claude
ee406adf0c
[Bugfix] Fix ZeroDivisionError in Qwen3-VL video sampling when the source fps is unknown ( #60501 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-08 06:28:44 +00:00
f226af0e11
[Bugfix] Eliminate frontend ZMQ port TOCTOU with inherited listeners ( #54113 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 06:25:27 +00:00
fb2ac82458
[ROCm][DSv4.1][Perf] Replace the Torch prefill combine with the shared combine_topk_swa_indices kernel ( #56638 )
...
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Ren Yuzhou <54501155+yuzhouo7@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-08 14:15:18 +08:00
Shanshan Shen
95307ac6d4
[Doc] Update ViT CUDA graph design doc ( #60519 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-10-08 14:00:58 +08:00
Jiahan Chang (Cyrus)
816173ca9d
[Kernel] Remove WAR for LL CuTeDSL BF16 GEMM ( #60434 )
...
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com >
2026-10-08 05:38:07 +00:00
81198e97ba
[Refactor] Abstract DeepGemm MegaMOE backend ( #58053 )
...
Signed-off-by: Hui Shi <kalinshi@tencent.com >
Co-authored-by: LuneExplorer <liyihan23@mails.tsinghua.edu.cn >
Co-authored-by: ArmageddonKnight <bojian@cs.toronto.edu >
2026-10-08 05:05:39 +00:00
4ac0179f8a
[ROCm][Perf] Use native merge_attn_states kernel on gfx942/gfx950 ( #60159 )
...
Signed-off-by: mghanimi <mghanimi@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-08 04:49:49 +00:00
775792f2d5
[Bugfix] Reland #53020 : tie lm_head.weight for Nemotron Parse when checkpoint omits it ( #60272 )
...
Signed-off-by: Anish Kumar <aniskumar@nvidia.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-08 04:40:55 +00:00
4187ade7b8
[Frontend] Consolidate rl entrypoints. ( #57849 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 21:37:11 -07:00
1fbbfa1657
[Bugfix][Kimi-K3] Declare max_tp_shards on the DSpark MLA KV cache spec ( #59733 )
...
Signed-off-by: hyukjlee <hyukjlee@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
2026-10-08 00:33:32 -04:00
Matt and Claude Opus 5.5
5dd4a628dd
[ROCm][Bugfix] Map sleep-mode cuMem memory for the host ( #60367 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-08 04:26:12 +00:00
Isotr0py and Shanshan Shen
8352b2427f
[Multimodal] Optimize encoder cudagraph padding logic ( #58925 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-08 03:38:09 +00:00
Simon Danielsson and Douglas Lehr
87d9996abe
[ROCm][Perf][GLM-5.3-Flash] BF16 splitk kernel for sparse MLA decode ( #58584 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-07 21:37:51 -05:00
bnkaiteam and Flora Feng
3722645539
[Bugfix][Frontend] Accept successful 2xx responses for batch output uploads ( #60507 )
...
Signed-off-by: bnkaiteam <bnkaiteam@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 02:32:35 +00:00
caff2b44cb
[BugFix] Pick the DP world-group port at bind time via the coordination store ( #51018 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-08 02:30:35 +00:00
wangxiyuan and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a4b14e57b6
[Platform] Allow BackendEnum dynamic register ( #41091 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 09:56:19 +08:00
Matt and Claude Opus 5.5
9dba339d07
[Bugfix][ROCm] Keep MLA derived weights in place on weight reload ( #60389 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-07 20:38:06 -05:00
Nicolò Lucchesi
8733a21d55
[Core] Bound execute_dummy_batch RPC waits by the execute-model timeout ( #60398 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-08 01:22:04 +00:00
Junyi Yao and sfeng33
fdfc171a74
[Bugfix][ToolParser] Stream every tool call when one delta completes several ( #60354 )
...
Signed-off-by: Junyi Yao <j.yao@wustl.edu >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-08 01:07:35 +00:00
2dd887426e
[Bugfix][Frontend] Do not modify caller params in LLM.score() ( #59959 )
...
Signed-off-by: 叮当猫 <2756823972@qq.com >
Signed-off-by: apex-mochen <2756823972@qq.com >
Co-authored-by: DeepSeek Harness <noreply@deepseek.com >
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-08 09:03:08 +08:00
9e3e37cb3c
[TEST][XPU][CI] disable tests of unsupported quantization ( #60383 )
...
Signed-off-by: microslaw <you@example.com >
Co-authored-by: microslaw <you@example.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-08 08:56:28 +08:00
Kang Nahoon and Flora Feng
1995c0fcd0
[Bugfix][Frontend] Apply output_config and thinking in Anthropic count_tokens ( #59180 )
...
Signed-off-by: Kang Nahoon <ejrtks102020@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 00:24:20 +00:00
Kevin H. Luu and Sherlock
1283362e76
[CI] Add headroom and selection coverage for H200 fast lanes ( #56775 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: Sherlock <sherlock@raft.ai >
2026-10-07 17:17:56 -07:00
Nicolò Lucchesi
a58bdd0d4e
[Perf] JIT monitor: hook TileLang JITImpl.compile instead of __call__ ( #58685 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-08 00:11:59 +00:00
8faac3dd2a
[Bugfix] [Spec Decode] DeepSeek V4/V4.1 NVFP4: keep ignored MTP/DSpark experts on MXFP4 ( #59655 )
...
Signed-off-by: Shobhit Verma <shobhitv@nvidia.com >
Signed-off-by: v-shobhit <161510941+v-shobhit@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
2026-10-08 00:10:32 +00:00
YueshenZ and sfeng33
ed825aafc0
[Bugfix][Frontend] Fix Responses namespace aliases in named tool calling ( #59723 )
...
Signed-off-by: Shen <312058264+YueshenZ@users.noreply.github.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-08 00:07:52 +00:00
www6v and sfeng33
38dc8ee500
[Bugfix] Validate tool-parser/tokenizer compatibility at startup ( #59749 )
...
Signed-off-by: www6v <www6v@126.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-07 23:46:41 +00:00
Dhairyashil R G and sfeng33
d6fe5dca68
[Bugfix][Parser] Emit buffered post-reasoning text when no tool parser is configured ( #58911 )
...
Signed-off-by: Dhairyashil R G <18738735+dhairyashilRG@users.noreply.github.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-07 23:21:51 +00:00
Artem Perevedentsev
4ac5a36a8e
[CI/Build][NVIDIA] Build NIXL with NIXL EP from source in Rubin image ( #60090 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-08 02:44:17 +04:00
ef63c23d35
[Bugfix][DSv4.1] Skip SWA bounded replay for KV loads that carry the window ( #59197 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 15:28:42 -07:00
Nicolò Lucchesi and Mistral Vibe
d169c5a903
[Docs] Add disagg PD flow-example guidance to /pr-checklist skill ( #60111 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Mistral Vibe <noreply@mistral.ai >
2026-10-07 22:16:05 +00:00
f38c6778ce
[quantization] Add per-token NVFP4 quantization MoE support for ReLU2 ( #56740 )
...
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-07 22:01:09 +00:00
b803edbc9f
[Model] Streamline EmbeddingGemma 2 config resolution and harmonize Triton prefill attention ( #60289 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-07 14:56:40 -07:00
Simon Danielsson
947dd62c07
[CI] Fix failing test_deepseek_v4_rocm_dspark* tests ( #60462 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-10-07 14:18:19 -07:00
Thang Nguyen and Kimi
b47ec440bc
[CI] Stamp provenance labels on the torch-nightly image so Initialized Snapshot E2E can run ( #59487 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-10-07 14:15:39 -07:00
Misha Goin and Claude Opus 5.5
a982c81e3e
[Bugfix] Limit GlmMoeDsaForCausalLM fp8 KV cache default to SM100 ( #60281 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 13:37:13 -07:00
Rui
08567505b0
[Bugfix][KV Offload] Restore KVCR construction through the secondary-tier factory ( #58088 )
2026-10-07 21:36:57 +02:00
e572e02d21
[UX] Skip checkpoint shards an MTP head does not load ( #60152 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 15:16:19 -04:00
Moein Khazraee
17c70c8953
[KV Offload] Update KVCR adapter to pool-and-index API ( #59899 )
2026-10-07 21:14:07 +02:00
b9767299db
[Bugfix] Respect Inductor deterministic mode in combo-kernel defaults ( #59074 )
...
Signed-off-by: cheesecake <farzanaman99@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 20:08:11 +01:00
Jinzhen Lin and Kevin H. Luu
32fbfa15e8
[Bugfix][Quantization] Accept folded NVFP4 scales for Humming A16 ( #60337 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-07 11:04:43 -07:00
Giancarlo Delfin
6ee9be6b3c
[Spec Decode] Rename AR speculators to StandaloneAR / TargetDependentAR ( #60335 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-10-07 14:01:59 -04:00
fba462e008
[Bugfix]Fix FlashInfer warmup crash: autotune(tuning_buckets=...) must round M up, not down (Invalid MXFP8 split-K tactic with spec-decode) ( #58165 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 14:01:21 -04:00
554340f3d3
[Attention] Support NVFP4 KV cache on SM8x and SM12x with FlashInfer ( #46963 )
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-07 13:24:21 -04:00
3d6853b32c
[Bugfix] Seed hybrid mamba state index with mamba_block_size on prefix-cache hits ( #55601 )
...
Signed-off-by: pondzikk <pondzik@gmail.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: ptorsten <1652174+ptorsten@users.noreply.github.com >
2026-10-07 17:09:50 +00:00
Harry Mellor and Claude Opus 5.5
a8260cbc5c
[CI] Bump Transformers version to 5.19.0 ( #60381 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 10:01:41 -07:00
d64a18832b
[ROCm][Triton] Migrating MiniMax-M3 kernels from make_block_ptr to tensor descriptors ( #59285 )
...
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Co-authored-by: jpvillam <juan.villamizar@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Rohan Potdar <rohan.potdar@amd.com >
Co-authored-by: Daniel <danichan@amd.com >
2026-10-07 11:49:59 -05:00
MohitAMD and Cursor
240ee8de9e
[Bugfix][KV Connector][ROCm] Fix MoRI-IO KV block-offset for speculative decoding (MTP) ( #55053 )
...
Signed-off-by: Mohit Deopujari <Mohit.Deopujari@amd.com >
Signed-off-by: MohitAMD <mohit.deopujari@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-07 11:33:30 -05:00
Andy Lo
c741bfca70
[Docs] Add review areas for andylolu2 ( #60416 )
...
Signed-off-by: Andy Lo <andy@mistral.ai >
2026-10-07 16:05:29 +00:00
pmanczak
f7f62d1916
[XPU][MoE] Tune Triton fused MoE for Intel XPU ( #53065 )
2026-10-08 00:02:13 +08:00
xaguilar-amd and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
9367d8b969
[Bugfix][MLA] Scope DSpark non-causal capability to builder layers ( #57992 )
...
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-07 10:22:32 -05:00
stefankoncarevic
71f9a74920
[CI][ROCm] Re-enable the DSv4.1 decoder replay graph test on ROCm ( #60392 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-07 09:57:52 -05:00
Alec
14e3902d51
[Frontend][Rust] Add gRPC forbidden token sequences and cache usage ( #59837 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
2026-10-07 07:46:44 -07:00
Mikko Tukiainen and Cursor
548597367e
[ROCm][Perf] Add AITER FlyDSL GDN prefill backend ( #57560 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-07 09:44:22 -05:00
Micah Williamson and Fangzhou Ai
f6e3944e31
[ROCm][DSv4] AITER MegaMoEV2 Integration For DeepSeek V4 ( #59685 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
2026-10-07 09:35:53 -05:00
c83935b300
[Bugfix][Model Loader] Support reload_weights with runai_streamer load format ( #58124 )
...
Signed-off-by: Viktor Koukouliev <vkoukouliev@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-07 14:04:35 +00:00
00CC
5281e49908
[ROCm][Perf] Enable medium-skinny dispatch in RDNAHybridW4A16LinearKernel ( #52619 )
...
Signed-off-by: tangzzycc <3081129260@qq.com >
2026-10-07 13:21:43 +00:00
Divakar Verma and Andreas Karatzas
efb8d85f24
[ROCm][CI] Add FP32 rounding slack to accuracy test near-tie upper bounds ( #60292 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-07 08:19:22 -05:00
2a54f6b625
[ROCm][Perf][GLM-5.3-Flash] Enable AITER fused shared experts on gfx950 ( #59221 )
...
Signed-off-by: Jin Tao <jin.tao@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
2026-10-07 20:53:40 +08:00
d9503dc503
[Bugfix][Qwen4Exp] Derive the QSA head_dim from the unsharded head count ( #59945 )
...
Signed-off-by: lesj0610 <lesj0610@godoiksan.org >
Co-authored-by: lesj0610 <lesj0610@godoiksan.org >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-07 12:24:40 +00:00
stefankoncarevic
3ca00a8261
[CI] Skip EmbeddingGemma2 pooling tests needing transformers >= 5.19.0 ( #60371 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
2026-10-07 10:22:31 +00:00
SeongJun Lee
6b75dfb8a1
[Misc] Report a missing DeepSelect extension only when it is requested ( #60242 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
2026-10-07 09:53:17 +00:00
Kevin H. Luu and Claude Opus 5.5
02d8a94e61
[CI][ROCm] Skip DSv4.1 decoder replay graph test on ROCm ( #60378 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 02:38:31 -07:00
df417f780a
[Bugfix][MoRIIO] Exclude synchronous READ destinations from KV zeroing ( #59164 )
...
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com >
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
Signed-off-by: Hexiang Wang <56632993+whx-sjtu@users.noreply.github.com >
Co-authored-by: Yichao Zhu <Yichao.Zhu@amd.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-07 16:35:56 +08:00
42ecc8eebe
[CI][RL] Consolidate RL entrypoint tests under tests/entrypoints/rl ( #59948 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 08:29:09 +00:00
Bugen Zhao and Claude Opus 5.5
bc8dd4ad27
[Rust Frontend] Share SchemaRoot between argument coercion and grammars ( #59408 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 08:28:27 +00:00
stefankoncarevic
90f4936d9e
[CI/Build] Widen the ragged prefill scoring tolerance, drop batch invariance ( #60192 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-07 16:21:47 +08:00
Bugen Zhao and Claude Opus 5.5
7436a7f119
[Rust Frontend] Backtrack from safe text at a complete marker ( #59563 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 07:45:15 +00:00
e6fc81bc78
[Bugfix][Scheduler] Let pooling chunked prefill use full context ( #48039 )
...
Signed-off-by: Tommy <tran.tommy@hotmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-07 07:32:00 +00:00
cbee82324c
[Docs][RL] Move pause/resume and batch invariance examples to examples/rl ( #59949 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 06:22:10 +00:00
Sriraj and Claude
43b4aaea3e
[Bugfix][Frontend] Return 4xx for client errors in /cohere/v2/chat ( #60309 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-07 04:55:34 +00:00
Turner Jabbour and Claude Sonnet 5
aecb717f10
[CI] Allowlist-shrink batch 2: wire 9 tests/models/ files into Buildkite, fix 2 stale bugs found ( #59050 )
...
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-10-06 21:16:53 -07:00
Mohammad Hijjawi and Cyrus Leung
ef014bdf06
[Bugfix][Frontend] Return logprobs in Cohere Chat v2 responses ( #59072 )
...
Signed-off-by: Mohammad Hijjawi <mohammad.hijjawi1997@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-07 04:13:52 +00:00
Hyukjoon Lee and Claude Opus 5.5
2342343746
[ROCm][DCP] Default MLA DCP verify to the round-robin asm decode where it applies ( #59965 )
...
Signed-off-by: hyukjlee <hyukjlee@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 03:10:54 +00:00
8a26869bca
[Bugfix] Index Mamba align state copies by request slot in MRV2 ( #60210 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Karl0007 <44745209+Karl0007@users.noreply.github.com >
Co-authored-by: Yongqin Wang <yongqinwang@roblox.com >
2026-10-07 02:32:46 +00:00
956960f85f
[Perf][MRV2] Avoid stream syncs in NonUvaBuffer host-to-device copies ( #60215 )
...
Signed-off-by: cheesecake <farzanaman99@gmail.com >
Co-authored-by: William Joyner <133391209+joynerwk03@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 01:51:12 +00:00
Robert Shaw and Claude Code
283d76d56a
[Bugfix] Widen ep_gather output offset to int64 ( #58373 )
...
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-10-07 01:45:04 +00:00
fe51194e2d
[Bugfix][MiniMax-M3] Fence decode indexer shared-memory stage reuse ( #60194 )
...
Signed-off-by: Vladimir Anisimov <vanisimov@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-07 01:35:12 +00:00
Kevin H. Luu and Claude Opus 5.5
4b4eb6277a
[CI] Remove AMD mirror of CPU-only Rust frontend cargo steps ( #60323 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 18:30:52 -07:00
Micah Williamson and Kevin H. Luu
13f8c453ee
[CI][Releases] Trigger perf-eval Through Release Pipeline ( #52213 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-06 18:27:27 -07:00
Miłosz Grunwald
703c29179c
[TEST][XPU][CI] Generalize model runner prefill tail classification tests to all platforms ( #60201 )
...
Signed-off-by: microslaw <milosz.grunwald@intel.com >
2026-10-07 08:59:20 +08:00
Sriraj and Flora Feng
db6e3cd8c4
[Bugfix][Frontend] Log the full request body when max_log_len is unset ( #60310 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-07 00:42:17 +00:00
jxj and OpenAI Codex
6c2925fe5f
[Bugfix][Core] Preserve speculative metrics when coalescing outputs ( #55708 )
...
Signed-off-by: git-jxj <65210887+git-jxj@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-07 08:35:14 +08:00
Sriraj and Flora Feng
d77cf81156
[Bugfix][Frontend] Honor default penalties in batch chat completions ( #60307 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-07 00:21:00 +00:00
5ab4e4445b
[Perf][MoE] Skip padded work in block-FP8 DeepGEMM experts ( #59128 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-06 18:49:57 -05:00
Cheese Cake and Claude Opus 5.5
b6d8e8afd9
[Bugfix][MRV2][Spec Decode] Accept num_speculative_tokens in NgramGPUSpeculator.propose ( #60300 )
...
Signed-off-by: cheesecake <farzanaman99@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 15:02:04 -07:00
ae19af883a
[CI] Require transformers 5.19 for the EmbeddingGemma2 registry test ( #60295 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-06 14:50:10 -07:00
aditi-amd
385d86a6cf
[Attention] Add UltraQuant 4-bit KV cache backend (FlyDSL D=256) ( #57057 )
...
Signed-off-by: Aditi Rana <aditi.rana@amd.com >
2026-10-06 20:42:52 +00:00
819852df6b
[ModelRunner V2] Speculative Decoding NGram GPU Implementations ( #40704 )
...
Signed-off-by: PatchouliTaisa <patchychen@tencent.com >
Signed-off-by: PatchouliTaisa <pyramkar@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Signed-off-by: Xuanan Chen <xuananchenc@nvidia.com >
Co-authored-by: PatchouliTaisa <patchychen@tencent.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Xuanan Chen <xuananchenc@nvidia.com >
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com >
2026-10-06 13:32:59 -07:00
Benjamin Chislett and Claude Opus 5.5
dac35c5b66
[Misc] Add chat-completion options to gsm8k_eval.py CLI ( #60114 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 20:13:44 +00:00
PatrykSaffer and Misha Goin
e43db1f5e2
Pad to 64 not 128 Q heads in flashMLA sparse ( #60029 )
...
Signed-off-by: Patryk Saffer <patryk.saffer@mistral.ai >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-06 16:03:05 -04:00
d1f3d8b870
[Quantization] Prefer Humming before Marlin backends on SM90 ( #56997 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 12:31:35 -07:00
Roger Wang and Kimi Code
bb87d227d4
[Bugfix][Model] Fix mypy safe-super error in EmbeddingGemma2Model ( #60280 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-10-06 15:10:00 -04:00
e37e51dd24
[Metrics] Expose cached prompt tokens by cache tier ( #56318 )
...
Signed-off-by: Cam Quilici <cjquilici@gmail.com >
Signed-off-by: Cam Quilici <cameron@semianalysis.com >
Co-authored-by: Cam Quilici <cameron@semianalysis.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-06 14:52:05 -04:00
Nishant Revur and Claude
ad0f67a7ec
[Bugfix][Spec Decode] Keep heterogeneous-vocab draft models on Model Runner V1 ( #59541 )
...
Signed-off-by: Nishant Revur <kumrfni@amazon.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-06 18:03:44 +00:00
3403e0f176
[Bugfix] Persist FlashInfer autotune cache per rank to fix the rank-0-only cache-hit deadlock ( #57635 )
...
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com >
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-06 17:51:01 +00:00
Luciano Martins and Luciano Martins
02b83919aa
[Model] Support EmbeddingGemma2 multimodal pooling architecture ( #60254 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
2026-10-06 13:43:38 -04:00
7b665f7a58
[Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) ( #58181 )
...
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-06 17:22:47 +00:00
Matthew Bonanni and Claude Opus 5.5
e82b80099a
[Perf][HiSparse] Cache per-request residency instead of rescanning every page each step ( #60083 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 13:04:55 -04:00
Wentao Ye
fba31f31a9
[GLM5.3 Perf] Switch to fp8 kv cache by default for GLM, 2.3%~5.5% E2E Throughput Improvement ( #60140 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-06 12:54:20 -04:00
f8d92cae41
[Quantization] Layer re-quantization for linear layers through online quantization API (MXFP8 -> FP8 PTPC showcase) ( #55684 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com >
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com >
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-06 11:38:18 -05:00
eky-amd and Claude Fable 5
29f955cce0
[ROCm] Fuse MLA dual RMSNorm + FP8 group quant for DeepSeek-R1 ( #54857 )
...
Signed-off-by: Ethan Ky <ethan.ky@amd.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-10-06 11:26:38 -05:00
stefankoncarevic
4516896626
[CI/Build] Warm the DP engines before measuring load balance in test_load ( #60241 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-06 11:23:37 -05:00
Giancarlo Delfin
fcf53d240c
[Spec Decode] Remove eager metadata rebuild during MTP fused multi-step decode ( #58463 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-10-06 12:17:30 -04:00
SeongJun Lee
4ea0c28bc4
[Bugfix] Warn only on per-request do_normalize/do_rescale overrides ( #60240 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
2026-10-06 15:55:27 +00:00
Jeff-Tseng-dev
aad4579bbe
Upgrade tpu-inference to v0.31.0 ( #60190 )
2026-10-06 15:44:56 +00:00
Chris Leonard and Shengqi Chen
cc73cca9c8
[Build] Migrate vendored DeepGEMM from pybind to TORCH_LIBRARY (abi3) ( #48962 )
...
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-06 08:39:04 -07:00
ovidiusm
cbe1f9740d
[NIXL] Bump NIXL version to 1.5.0 ( #56907 )
...
Signed-off-by: Ovidiu Mara <ovidium@nvidia.com >
2026-10-06 16:55:03 +02:00
Bugen Zhao and Claude Opus 5.5
0719975cee
[CI][Rust Frontend] Pass --locked to cargo binstall ( #60182 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 22:53:17 +08:00
8bd7737a03
[Docs] Clarify that vLLM does not isolate tenants sharing a server ( #59811 )
...
Signed-off-by: SunnyR <sunny.rangnani@chainguard.dev >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Juan Pérez de Algaba <124347725+jperezdealgaba@users.noreply.github.com >
2026-10-06 22:39:59 +08:00
Samuel Nordmann and Misha Goin
4a99ca40d6
[Perf][MoE] Eliminate staging from NCCL symmetric reduce-scatter ( #49194 )
...
Signed-off-by: samnordmann <snordmann@nvidia.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-06 07:37:01 -07:00
Jiangyun Zhu and Claude Opus 5.5
bf049ddb8f
[Docs] Run pr-checklist before opening a PR ( #60235 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 14:12:18 +00:00
Matthias Gehre
b267fe4881
[ROCm][Perf] W4A16: pad gfx11 weight and activation strides ( #56301 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
2026-10-06 08:22:33 -05:00
Artem Perevedentsev and Shashank Varma
049507aa76
[Bugfix][Structured Output] Flag multi-branch allOf as unsupported for xgrammar ( #59061 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Shashank Varma <324153016+shashankvarma499@users.noreply.github.com >
2026-10-06 17:01:04 +04:00
Roy Wang
1a86733c7b
[Docs] Add model recipes skill ( #57786 )
2026-10-06 20:40:10 +08:00
Rodrigo Garcia
3faf214051
[vllm-bench, feature] Added support for Mooncake-style, timed-traces replay. ( #55937 )
2026-10-06 20:39:35 +08:00
Sherif Waly
25385102fe
[Bugfix][KV Offload] Honor speculative cacheability in SimpleCPU ( #60071 )
...
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai >
2026-10-06 14:25:08 +02:00
Wentao Ye
3e182185aa
[Deprecation] Deprecate sonet dataset as scheduled ( #60129 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-06 08:15:12 -04:00
Mia Ojeda
3313825983
[Bugfix][DiffusionGemma] Read the causal mask as int32 and compile the sample step ( #59992 )
...
Signed-off-by: ojeda-e <eojeda@league.com >
2026-10-06 19:25:51 +08:00
09e0ce3d82
[Bugfix][Frontend] Apply model-default reasoning parser in GPU-less render server ( #54835 )
...
Signed-off-by: Shimi Bandiel <shimib@google.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-06 18:46:24 +08:00
zhangchengzhucufe-dev
68088ed392
Fix grammar in BlockPool.reset_prefix_cache docstring: "to invalid" → "to invalidate" ( #59886 )
...
Signed-off-by: zhangchengzhucufe-dev <zhangchengzhucufe@gmail.com >
2026-10-06 10:19:54 +00:00
Louie Tsai
888074b42d
[CPU][Recipes] Detect model head constraints for automatic tensor parallel selection in vLLM Recipes Tool ( #60170 )
...
Signed-off-by: louie-tsai <louie.tsai@intel.com >
2026-10-06 18:15:40 +08:00
Bugen Zhao and Claude Opus 5.5
6bbad6acdf
[Rust Frontend] Pass tool defer_loading through to chat templates ( #60203 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-10-06 18:00:15 +08:00
aoshen02 and Claude Opus 5.5
31e2443c90
[Model] LongCat-Flash: scale MLA norms while loading and drop the post-load sweep ( #60080 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 08:47:02 +00:00
roikoren755 and zjy0516
e11962fc1c
[Feat][Mamba2] Enable internal prefill checkpoints ( #57329 )
...
Signed-off-by: Roi Koren <roik@nvidia.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-06 07:53:51 +00:00
edfccc9469
[ROCM][Perf] Add ROCM_SEGMENTED_ATTN attention-backend works across RDNA3/3.5/4 ( #59132 )
...
Signed-off-by: root <jeffaw99@hotmail.com >
Signed-off-by: big-yellow-duck <jeffaw99@hotmail.com >
Co-authored-by: Tan Pin Siang <1716735+tanpinsiang@users.noreply.github.com >
Co-authored-by: vllmellm <190700713+vllmellm@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-06 07:53:47 +00:00
15c1cc12ac
[Bugfix][EPLB] Use group-local ranks for torch P2P transfers ( #55804 )
...
Signed-off-by: LostFox11 <wangziyue17@huawei.com >
Co-authored-by: LostFox11 <wangziyue17@huawei.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: YangShuai52 <178856220+YangShuai52@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-06 07:44:57 +00:00
Tarun Kumar
6ad2e06c4a
fix: exclude flashinfer-jit-cache from wheel install_requires ( #60052 )
...
Signed-off-by: Tarun Kumar <takumar@redhat.com >
2026-10-06 06:31:13 +00:00
sungbin1015 and Claude
1e5d0ea888
[CI/Build] Fix incorrect CLI arg validation tests in test_cli_args.py ( #46779 )
...
Signed-off-by: sungbin1015 <sbin@solbox.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-06 05:58:47 +00:00
40c86ae1ce
[Kernel] Enable OAI Triton MXFP4 MoE on SM12x ( #58877 )
...
Signed-off-by: lazymio <mio@lazym.io >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-06 01:49:13 -04:00
Bugen Zhao and Claude Opus 5.5
b3082d3044
[Rust Frontend] Remove the PyO3 tool-parser bridge ( #59744 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 05:46:27 +00:00
Bugen Zhao and Claude Opus 5.5
5145e35e50
[Frontend] Port the MiniMax M3 tool parser to the parser engine ( #59743 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 05:46:27 +00:00
czhu-cohere and aoshen02
6fddc9b21e
[Profiler] Add per-session profiling controls to Python and Rust frontends ( #57875 )
...
Signed-off-by: czhu <conway.zhu@cohere.com >
Signed-off-by: Conway Zhu <conway.zhu@cohere.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-06 05:21:19 +00:00
aoshen02 and Claude Opus 5.5
9a1f6fd4cb
[Model][DeepSeek-V4.1] Permute mega-attention wq_b / wo_a while loading ( #60064 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 05:20:09 +00:00
Chinmay-Kulkarni-AMD and Cursor Agent
533d81b4b0
[ZenCPU][Test] Add zentorch/torch version pin consistency test ( #52888 )
...
Signed-off-by: Chinmay Kulkarni <Chinmay.Kulkarni@amd.com >
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2026-10-06 13:12:19 +08:00
eb2901b735
[Perf][DSv4.1] Run decoder replay layers in CUDA graphs and trim in PIECEWISE graphs ( #59532 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 22:11:45 -07:00
21d93d0d8c
[Attention][Model] FP8 KV cache for Triton DiffKV and MiMo-V2.6-Flash ( #58128 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-05 21:18:29 -07:00
Tianyu Guo
30ab428235
[Bugfix][Attention] Restrict FlashInfer sparse MLA FULL graphs to decode ( #59751 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-10-06 04:13:23 +00:00
lzhan011 and Flora Feng
a3e0243b1c
[Bugfix][Parser] Mistral pre-v11: don't fail requests on unexpected tool call JSON ( #54844 )
...
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-06 03:37:30 +00:00
Tianyu Guo and OpenAI Codex
8c417a2dae
[Bugfix][Model] Fix Qwen3-Omni interleaved M-RoPE boundary positions ( #59842 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-06 03:35:53 +00:00
Tianyu Guo
48eb44b719
[Bugfix][KVConnector][NIXL] Fix TP handshake test after per-region FA desc flags ( #60157 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-10-06 03:19:41 +00:00
6fa67afed9
[Bugfix][Core] Retire stale Mamba spec blocks in prefill checkpoint steps ( #59759 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-06 11:01:19 +08:00
2b32d9ac6a
[Bugfix][Core] Drop stale block hashes when a session truncates tokens ( #59103 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Co-authored-by: Brian Woods <me@brian.biz >
Co-authored-by: Change72 <changg@nvidia.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-06 02:51:03 +00:00
b0b23a48d4
[Bugfix] Fix Qwen4Exp FP8 PLE pinned lookup kernel failing to compile below SM89 ( #60142 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-06 01:56:44 +00:00
9465e1c783
[Bugfix][NIXL] Recover locally invalidated pull peer metadata ( #55471 )
...
Signed-off-by: xijiade.aihemaiti <3146335281@qq.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 18:46:53 -07:00
lzhan011 and Flora Feng
9bf4e51030
[Bugfix][Frontend] Malformed $defs in a tool schema should be a 400, not a 500 ( #54850 )
...
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-06 01:37:20 +00:00
amd-xavierwang and TJian
efd001756b
[ROCm] Disable AITER attention query quantization on RDNA3 ( #57750 )
...
Signed-off-by: xawang <xawang@amd.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-10-06 01:37:18 +00:00
2537629eb2
[Bugfix][KVConnector][NIXL] Size per-region replicate flags by each region's block count ( #60107 )
...
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-06 01:27:50 +00:00
0e8f8ecba8
[Bugfix][KVConnector][NIXL] Return NumPy descriptors from mixed-memory local registration ( #60108 )
...
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-06 01:23:20 +00:00
4e198cef83
[TEST][XPU][CI] Fix ci uva kernel test instability ( #60041 )
...
Signed-off-by: Your Name <you@example.com >
Co-authored-by: Your Name <you@example.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-06 09:19:01 +08:00
97bd8c74eb
[ROCm][DSv4.1][Perf] One-pass dequantize and a gather-sized grid for the compressed K cache ( #56720 )
...
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 17:57:46 -07:00
b2863c0de9
[Kimi-K3] Use the fused K/V pack kernel on the decode-context-parallel prefill path ( #56881 )
...
Signed-off-by: Brandon Zhang <31413216+brandonfzhang@users.noreply.github.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-06 00:13:55 +00:00
Mai Hoàng Đức and Flora Feng
5cd8a2626b
[Bugfix][Structured Output] Cap JSON schema nesting to prevent 500s and API-server stalls ( #60036 )
...
Signed-off-by: Mazukiri <maihoangduc04@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-06 00:06:59 +00:00
Aarushi Jain and Andreas Karatzas
fa423b614c
[CI][ROCm] Wait for engine teardown between LM Eval models ( #60100 )
...
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-05 23:39:59 +00:00
Mason and Yongye Zhu
6dd813e0b7
[Bugfix][MoE] Allow MiniMax2 routing in TRTLLM BF16 monolithic backend ( #59571 )
...
Signed-off-by: Mason Pitt <bradley.b.pitt@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-10-05 16:24:14 -07:00
0eac152707
[Bugfix] Route every speculation-capable row through the speculative path so recurrent state stays consistent ( #56531 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-06 03:03:44 +04:00
Ting SUN and Flora Feng
4a705c7b54
[Bugfix][Frontend] Preserve logprobs when top_logprobs is null ( #47838 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 22:43:31 +00:00
Kevin H. Luu and Claude Opus 5.5
89a00995c9
[CI] Read each profiler round's trace as it stops in test_gpu_profiler ( #59769 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 15:25:11 -07:00
RomeJay and Flora Feng
2e06395fd3
[Bugfix][Frontend] Build tool-call grammars from the tools the prompt renders ( #59879 )
...
Signed-off-by: Rongjie Jiang <j1870831863@outlook.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 22:01:03 +00:00
Martin Hickey and Flora Feng
3fb1cd796b
[Frontend] Extract parse and assembly out of batch chat derender ( #60073 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 21:57:45 +00:00
2e3154aa18
[HiSparse] Log steady-state max concurrency and expose host-tier utilization gauges ( #58949 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-05 14:20:32 -07:00
Matthew Bonanni and Claude Opus 5.5
8542c3eff5
[CI] Fix test_mixed_warmup_gate after #57053 ( #60116 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 21:17:00 +00:00
Yida Weng and Flora Feng
18c8a65edd
[BugFix][Frontend] Pass reasoning_ended through render → generate ( #60059 ) ( #60062 )
...
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 20:39:25 +00:00
Wentao Ye
74c5cbcd7b
[Bugfix][MRV2][Spec Decode] Honor dynamic K in autoregressive speculators, 1.2~1.3x kernel perf improvement ( #57053 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-05 14:54:11 -04:00
Raphaël Rialland and Simon Veitner
c32495d58f
[watermark] compatibility validation ( #56801 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
2026-10-05 14:37:13 -04:00
Giancarlo Delfin
88f3c601b3
[Docs] Add reviewer area of interest for TheEpicDolphin ( #60092 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-10-05 18:35:19 +00:00
aoshen02 and Claude Opus 5.5
d09ff77637
[Bugfix] Give each data-parallel engine its own global RNG streams ( #59788 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 17:38:18 +00:00
877ddcc456
[Bugfix][Spec Decode] Reserve the bonus KV slot for fill-in DSpark ( #59105 )
...
Signed-off-by: Evgeny Savinov <notime.sea@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
2026-10-05 17:35:51 +00:00
akii96
182247257f
[ROCm][Perf] Use a stacked hipBLASLt GEMM for the bf16x3 router ( #52668 )
...
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com >
2026-10-05 17:28:07 +00:00
Matthew Bonanni and Claude Opus 5.5
54d93af9fb
[Bugfix][HiSparse] Reject cudagraph_mode=FULL at startup ( #59688 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 12:42:27 -04:00
Sohaib Ahmed
64cb683842
[Model] Add Nemotron 3.5 ASR transcription support ( #59827 )
...
Signed-off-by: Sohaib-Ahmed21 <sohaibahmed1919@gmail.com >
2026-10-05 16:28:09 +00:00
stefankoncarevic and Andreas Karatzas
87954c5b04
[CI/Build] Declare the video modality on the CohereCompass reference model ( #60063 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-05 16:25:42 +00:00
Mikko Tukiainen and Cursor
d23af12cb1
[ROCm][Perf] Reach the fused QSA pre-indexer from the AMD path ( #57947 )
...
Signed-off-by: mjkvaak-amd <mjkvaak-amd@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-05 16:22:29 +00:00
bzsuni and cjackal
528772a4bf
[Bugfix][Core] Retain encoder cache references for repeated multimodal inputs ( #59942 )
...
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io >
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-10-05 15:52:46 +00:00
charlesli640
edde9d2ce4
Remove redundant dependency requirement for TPU ( #59977 )
...
Signed-off-by: Charles Li <licharles@google.com >
2026-10-05 08:45:37 -07:00
stefankoncarevic
f2d8fbf459
[CI/Build] Run the prefill token scoring test under batch invariance ( #60056 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-05 10:32:33 -05:00
Matthew Bonanni and Claude Opus 5.5
9e931a0c6c
[CI] Add CODEOWNERS for HiSparse ( #60060 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 15:18:29 +00:00
60932a6401
[Bugfix][Structured Output] Preserve literal values in Guidance disable_additional_properties ( #58709 )
...
Signed-off-by: Abhay Joshi <105213625+abhayjoshi201@users.noreply.github.com >
Co-authored-by: Antigravity <noreply@google.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-05 18:12:40 +03:00
Nicolò Lucchesi
30d4032363
[Model][GLM-5.3-Flash] Support DCP for the kpool sparse indexer ( #59211 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-05 16:44:14 +02:00
Jee Jee Li
f9c9e8ac24
[LoRA] Remove tensorizer ( #60024 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-05 07:01:56 -07:00
Abhay Shukla and Claude Opus 5.5
d3547f9d03
[Test] Re-enable Voxtral HF reference test on Transformers v5 ( #59771 )
...
Signed-off-by: AbhayShuklaIIT <abhay.shukla@live.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 13:43:49 +00:00
cjackal
0e468adb43
[Model] Extend device-side mm normalization to Kimi K2.5 / K3 ( #59278 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-10-05 21:28:13 +08:00
Nicolò Lucchesi and Leoyzen
ff53f32409
[Attention][MLA] Support fp8_ds_mla KV cache for NoPE-512 models on SM90 ( #59246 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Leoyzen <leoyzen@gmail.com >
2026-10-05 13:12:00 +00:00
Vadim Gimpelson and Claude
d0d6e5f3a2
[Bugfix] Fix Mamba page size AssertionError with spec decoding on GraniteMoeHybrid, FalconH1 and Zamba2 ( #59975 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-05 12:49:47 +00:00
roikoren755 and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
51eeb0c58f
[Bugfix] Load stacked expert weights for non-gated MoE ( #59031 )
...
Signed-off-by: Roi Koren <roik@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 15:26:55 +03:00
Stefano Castagnetta and Thien Tran
4ff028d77e
[Bugfix][Qwen4Exp] Honor --kv-cache-dtype-skip-layers in QSA attention ( #60023 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-05 12:13:30 +00:00
Stefano Castagnetta and Thien Tran
55b80221ed
[Perf][Qwen4Exp] Keep the HC up projection on the skinny GEMM path ( #60027 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-05 20:09:36 +08:00
Juan Pérez de Algaba
710ac56e69
[Security] Restrict Pillow image formats on untrusted media paths ( #60022 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
2026-10-05 10:40:15 +00:00
edca360f13
[Bugfix][Qwen4Exp] Load PLE tables unquantized under Quark checkpoints ( #59443 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-10-05 10:40:02 +00:00
c04c79e50d
[Performance] Add SM121 TP=2 skinny-GEMM plans ( #59632 )
...
Signed-off-by: sudhanshu shukla <sudhanshu112233shukla@users.noreply.github.com >
Co-authored-by: sudhanshu shukla <sudhanshu112233shukla@users.noreply.github.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-05 18:27:51 +08:00
Maroon Ayoub
b1401e0aa7
[Bugfix][NIXL] Count heartbeats as remote engine activity ( #59873 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
2026-10-05 09:36:13 +00:00
ae53b06898
[Bugfix][GLM-5.3-Flash] Keep batch x heads out of gridDim.z in the GLM-5.3-Flash fused recurrent KDA kernel ( #56974 )
...
Signed-off-by: NolenLiang <nliang@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com >
2026-10-05 09:31:45 +00:00
Ian Eaves and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
b1f229fb75
[Model] Declare SupportsEagle3 on Qwen3ASRForConditionalGeneration ( #52824 )
...
Signed-off-by: Ian Eaves <ian.k.eaves@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 09:23:47 +00:00
Jee Jee Li
e3af5bf3ce
[LoRA] Code cleanup ( #60017 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-10-05 09:15:53 +00:00
aoshen02 and Claude Opus 5.5
4a30c4cad0
[Model][DeepSeek-V4] Make MegaMoE shared-expert finalize independent of linear post-load order ( #59927 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 08:39:49 +00:00