 MOVZX
|
49d708e158
|
Merge branch 'main' into patches
Signed-off-by: MOVZX <movzx@yahoo.com>
|
2026-10-18 18:09:11 +00:00 |
|
 alpandmottopanikeiku
|
ff01a6b14f
|
[Bugfix][Frontend] Stop beam search on generation config EOS tokens (#62072)
pre-commit / pre-run-check (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
|
2026-10-18 16:49:42 +00:00 |
|
|
|
e80b6c642f
|
[ROCm][DSv4] Overlap the unfused shared expert with AITER MegaMoE (#61819)
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-18 22:39:38 +08:00 |
|
|
Harry HuangandThien Tran
|
494663940b
|
[Qwen4Exp] Clean up HC and PLE implementations for Qwen3.8-Flash-Next (#61454)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-10-18 19:38:21 +08:00 |
|
|
Rémi Delhaye
|
3c13653738
|
[Frontend] Score with the chat template declared by Sentence Transformers CrossEncoders (#60877)
Signed-off-by: rdlhio <contact@rdlh.io>
|
2026-10-18 11:30:12 +00:00 |
|
|
Isotr0py
|
3f1b04e045
|
[Bugfix] Fix Opencv video backend frames count missing for WebM video inputs (#61759)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-10-18 10:37:39 +00:00 |
|
|
MOVZX
|
c5ab42bc0f
|
Merge branch 'main' into patches
|
2026-10-18 16:04:36 +07:00 |
|
|
|
7c9042c05a
|
[Bugfix][MRV2] Fix speculative warmup deadlock with pipeline parallelism (#62129)
Signed-off-by: chenjr97 <chenjr97@mail2.sysu.edu.cn>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-17 23:51:32 -07:00 |
|
|
|
da9134bc89
|
[ROCm][CI] Give the MoE parallel-launch watchdog room for AITER JIT builds (#62161)
Signed-off-by: Andreas Karatzas <AndreasKaratzas@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <AndreasKaratzas@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-17 21:49:27 -05:00 |
|
|
|
b2456c622b
|
[ROCm][CI] Make ROCm 10.0 the default image; keep ROCm 7.2 as -rocm72 (#58761)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-10-17 21:43:59 -05:00 |
|
|
|
03ee2e134d
|
[CI][ROCm] Fix order-dependent test_rocm_aiter_sampler_defers_import (CPU tensors hit Triton) (#62213)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-10-18 01:10:08 +00:00 |
|
|
|
7879f3687b
|
[NIXL] Drop unreachable producer-side notif counting in push mode (#62210)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-17 21:24:35 +00:00 |
|
|
Afonso RibeiroandFlora Feng
|
5c1c903501
|
[Perf][Parser] Stream Qwen3 tool argument values incrementally (12 KB call: 2.1 s to 11 ms) (#62027)
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-17 18:49:06 +00:00 |
|
|
Nick HillandClaude Opus 5.5
|
0c5dd18e7a
|
[MRV2] Collect KV transfer results before the drafter runs (#62041)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-17 10:27:57 -07:00 |
|
|
|
c0b6f74bee
|
[Benchmark] Add per-case kernel profiles to vllm bench forward (#61692)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-17 08:27:11 -07:00 |
|
|
|
837e87e1c5
|
[Feature][LoRA][MoE] Expert LoRA for FlashInfer TRTLLM MXFP4 experts (#61979)
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-17 20:17:56 +08:00 |
|
|
Vadim GimpelsonandClaude
|
ec8b20fbf0
|
[Bugfix] Fix Gemma 4 encoder CUDA graph capture OOM at startup (#61365)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-17 11:04:49 +00:00 |
|
|
|
2b85d9cb65
|
[Perf][Bugfix] Prefer MNNVL for single-node FlashInfer allreduce again (#61989)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-10-17 02:55:28 -07:00 |
|
|
|
026f6ab268
|
[Model] DeepSeek-V4/V4.1 VL: stream language-model weights instead of sorting the checkpoint (#58857)
Signed-off-by: git bisector <gitbisector@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-17 09:37:56 +00:00 |
|
|
|
f98ae6a9fd
|
[Bugfix][Core] Keep resumable sessions that stop while preempted (#58778)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Zong <yzong@redhat.com>
|
2026-10-17 09:08:14 +00:00 |
|
|
|
2067297b2f
|
[ROCm][Perf] Use AITER tuned configs to select a4w4 gemm kernel instead of env var (#61278)
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Signed-off-by: afriedri <afriedri@amd.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-10-17 04:00:28 -05:00 |
|
|
Jee Jee Li
|
e56b1603af
|
[Kernel]Add norm fused kernel for Step-5 Preview and Muse Spark Open (#62132)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-10-17 01:53:00 -07:00 |
|
|
Ting SUN
|
eb8de8d1a1
|
[Bugfix][OpenAI] Respect allowed_token_ids in beam search (#47630)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-10-17 08:50:40 +00:00 |
|
|
dskwe
|
b05ede5616
|
[Bugfix][Frontend] Preserve token IDs in completion derender responses (#62114)
Signed-off-by: Yuan Li <dskwelmcy@163.com>
|
2026-10-17 08:27:48 +00:00 |
|
|
|
61bb590877
|
[Bugfix][Spec Decode] Clear stale shared block-table rows (#61174)
Signed-off-by: Cheng Rui <286040359@qq.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
|
2026-10-17 01:25:02 -07:00 |
|
|
|
c3de0b37e2
|
[Core][Spec Decode] Replace EAGLE group heuristics with explicit draft-layer tags (#61455)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-17 08:22:33 +00:00 |
|
|
MOVZX
|
e0fe5df731
|
Merge branch 'main' into patches
|
2026-10-17 08:06:56 +00:00 |
|
|
znzrybandWoosuk Kwon
|
ad2688ab30
|
[Bugfix] Validate draft-token capacity in forward benchmark (#61809)
Signed-off-by: znzryb <znzryb34@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-10-17 00:51:36 -07:00 |
|
|
JovanandOpenAI Codex
|
5be4266f5c
|
[Bugfix][Frontend] Preserve Cohere and Anthropic streaming errors (#61947)
Signed-off-by: Kevin <jia__kuan@163.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-10-17 06:10:16 +00:00 |
|
|
Wei Zhao
|
90d6d871d6
|
[Attention] Add CuTe DSL backend selection for MLA context prefill (#62069)
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
|
2026-10-17 01:21:55 -04:00 |
|
|
Cy ChengandCy
|
d74f4180cd
|
[Bugfix][Frontend] Detokenize echoed prompts after truncation (#61876)
Signed-off-by: Cy <15118312+cycancode@users.noreply.github.com>
Co-authored-by: Cy <15118312+cycancode@users.noreply.github.com>
|
2026-10-17 05:20:38 +00:00 |
|
|
Jee Jee Li
|
39983a5e7a
|
[Kernel] Add FP32 router GEMM support for Step-5 Preview and Muse Spark Open (#62126)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-10-16 22:17:55 -07:00 |
|
|
|
41b8c15809
|
[Feature][MoRIIO] Support hybrid WRITE transfers with DSpark (#60544)
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-17 04:57:58 +00:00 |
|
|
|
7b6310c109
|
[Perf][Sampler] Speed up the small-batch split top-p pipeline (#61996)
Signed-off-by: Bowen Wang <bowen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-16 21:22:11 -07:00 |
|
|
Fadi Arafehandvllm-agent
|
556ac15eb9
|
[Bugfix] Remove extra args from CPU's compute slot mapping kernel (#61992)
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-17 12:11:33 +08:00 |
|
|
Gabriel WuandClaude Opus 5.5
|
dcd07967ca
|
[CPU] Keep the hybrid block size across the engine core's config rerun (#61805)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-17 12:06:03 +08:00 |
|
|
|
ce49c47cbb
|
[Bugfix][Sampler] Carry the boundary tie count across blocks in the split top-p mask kernel (#62046)
Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: Bowen Wang <bowen@inferact.ai>
|
2026-10-17 02:26:57 +00:00 |
|
|
Sherif Walyandvllm-agent
|
ce8853d27a
|
[Bugfix][KV Offload] Allow explicit SimpleCPUOffload block counts (#61674)
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-17 02:11:12 +00:00 |
|
|
|
71c20f73f2
|
[FlashInfer] Migrate autotune warmup to flashinfer.autotune_v2 managed cache (#55671)
Signed-off-by: Yang Xu <yanxu@nvidia.com>
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Signed-off-by: YangXu <yanxu@nvidia.com>
Co-authored-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-17 02:08:28 +00:00 |
|
|
Shuolei Wangandaoshen02
|
b3a7e2ff6e
|
[RL][Frontend] Return weight-version spans on OpenAI completions and chat (#61347)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-17 02:08:07 +00:00 |
|
|
|
6b6a6c5c98
|
[RL] Support per-parameter source layouts for NCCL M2N (#60934)
Signed-off-by: pkousha <pkousha@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-16 21:41:17 -04:00 |
|
|
|
f019e3e105
|
[ROCm] attn res + sigmoid_mul + conv fusions (#52968)
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-17 00:54:54 +00:00 |
|
|
Vadim GimpelsonandClaude
|
783380af39
|
[Bugfix] Fix NaN output of the Mamba2 chunk scan kernel on sm90 (#61718)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-16 17:24:51 -07:00 |
|
|
Wei ZhaoandCodex
|
be25977532
|
[CI][Tests] Account for FP8 KV-cache rounding in MLA reference (#62067)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-16 17:10:08 -07:00 |
|
|
|
a94c315711
|
[DP] Engines own wave wake-ups; drop the front-end FIRST_REQ (#59559)
Signed-off-by: fh3 <a@b>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-16 19:41:54 -04:00 |
|
|
|
b544587408
|
[Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning (#53558)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-16 23:16:47 +00:00 |
|
|
alpandmottopanikeiku
|
2268d83bd9
|
[Bugfix][Multimodal] Honor RGB PNG transparency when loading images (#61986)
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
|
2026-10-16 23:09:12 +00:00 |
|
|
|
28e92eeff2
|
[ROCm][Perf][Kimi-K3] Use AITER Gluon chunk_kda for KDA prefill (#59552)
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-16 22:58:56 +00:00 |
|
|
Afonso RibeiroandFlora Feng
|
b088eba944
|
[Perf][Parser] Stream MiniMax M2 tool argument values incrementally (16 KB call: 5.9 s to 26 ms) (#62035)
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-16 22:46:02 +00:00 |
|
|
|
596e530675
|
[Kernel] Add router GEMM kernels for (5120, 256) and BF16 output to ll_bf16_gemm (#62042)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Lain <fusiyuan2000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-16 15:39:46 -07:00 |
|