iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,240 Commits 3 Branches 0 Tags
ce49c47cbb241b59a32ef110079a5261dbd08e49
Commit Graph
9275 Commits
Author SHA1 Message Date
Akshat AnandcipheraxatBowen Wang
 		ce49c47cbb [Bugfix][Sampler] Carry the boundary tie count across blocks in the split top-p mask kernel (#62046) 
Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: Bowen Wang <bowen@inferact.ai>
 2026-10-17 02:26:57 +00:00
Sherif Walyandvllm-agent ce8853d27a [Bugfix][KV Offload] Allow explicit SimpleCPUOffload block counts (#61674) 
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-17 02:11:12 +00:00
Yang XuZihua WuKimi CodeCodexMisha Goinvllm-agent
 		71c20f73f2 [FlashInfer] Migrate autotune warmup to flashinfer.autotune_v2 managed cache (#55671) 
Signed-off-by: Yang Xu <yanxu@nvidia.com>
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Signed-off-by: YangXu <yanxu@nvidia.com>
Co-authored-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-17 02:08:28 +00:00
Shuolei Wangandaoshen02 b3a7e2ff6e [RL][Frontend] Return weight-version spans on OpenAI completions and chat (#61347) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-17 02:08:07 +00:00
pkoushaOpenAI Codexaoshen02
 		6b6a6c5c98 [RL] Support per-parameter source layouts for NCCL M2N (#60934) 
Signed-off-by: pkousha <pkousha@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-16 21:41:17 -04:00
Omar MuhammadNicholas SusantoDouglas Lehrmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f019e3e105 [ROCm] attn res + sigmoid_mul + conv fusions (#52968) 
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-17 00:54:54 +00:00
Vadim GimpelsonandClaude 783380af39 [Bugfix] Fix NaN output of the Mamba2 chunk scan kernel on sm90 (#61718) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-16 17:24:51 -07:00
Wei ZhaoandCodex be25977532 [CI][Tests] Account for FP8 KV-cache rounding in MLA reference (#62067) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-16 17:10:08 -07:00
aoshen02fh3Claude Opus 5.5
 		a94c315711 [DP] Engines own wave wake-ups; drop the front-end FIRST_REQ (#59559) 
Signed-off-by: fh3 <a@b>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 19:41:54 -04:00
wangxiyuanClaude Opus 5.5Yifan Qiao
 		b544587408 [Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning (#53558) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-10-16 23:16:47 +00:00
alpandmottopanikeiku 2268d83bd9 [Bugfix][Multimodal] Honor RGB PNG transparency when loading images (#61986) 
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
 2026-10-16 23:09:12 +00:00
Omar MuhammadNicholas SusantoDouglas LehrKevin H. Luu
 		28e92eeff2 [ROCm][Perf][Kimi-K3] Use AITER Gluon chunk_kda for KDA prefill (#59552) 
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-16 22:58:56 +00:00
Afonso RibeiroandFlora Feng b088eba944 [Perf][Parser] Stream MiniMax M2 tool argument values incrementally (16 KB call: 5.9 s to 26 ms) (#62035) 
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-16 22:46:02 +00:00
Woosuk KwonLainCodexClaude Opus 5.5vllm-agent
 		596e530675 [Kernel] Add router GEMM kernels for (5120, 256) and BF16 output to ll_bf16_gemm (#62042) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Lain <fusiyuan2000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-16 15:39:46 -07:00
Woosuk KwonBugen ZhaoYoni GozlanneilkgCursorClaude Opus 5.5
 		153a17166e [Frontend] Add the Beam reasoning and tool parsers (#62052) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
Co-authored-by: neilkg <33635204+neilkg@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 14:08:29 -07:00
Dan BlanaruandClaude Opus 5.5 9731f30a03 [Bugfix] Keep VLLM_RAY_BUNDLE_INDICES out of the compile cache key (#61211) 
Signed-off-by: DanBlanaru <48605845+DanBlanaru@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 20:59:48 +00:00
Tyler Michael SmithCodexRobert Shaw
 		68ebea789a [Perf][MoE] Skip padded work in SM100 FP4 DeepGEMM experts (#59044) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-16 16:20:52 -04:00
afriedrimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Claude Opus 5.5
 		cc2e4fecf2 [ROCm] [fusion] support ROCm aiter fusion of per tensor quant + rmsnorm (#52831) 
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 15:19:07 -05:00
cjackal 663e535640 [Perf][Spec Decode] Skip bonus-logit reduction for short drafts in rejection sampling (#61451) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-10-16 12:42:48 -07:00
Oxana KorzhandClaude Opus 5.5 5bf3d9f665 [ROCm][Kimi-K3] Support DSpark adaptive verification on MI355X (#59824) 
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-16 13:53:49 -05:00
djramic 307c5844ad [ROCm][CI] Enable batch invariance for the ngram scheduling accuracy test (#62048) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
 2026-10-16 18:36:14 +00:00
Guan-Ming Chiuandvllm-agent 3dc6def360 [Attention][Spec Decode] Enable varlen Mamba2 decode for adaptive verification (#53036) 
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-16 11:23:59 -07:00
alberto ae8f41238c [Bugfix][Parser] Honor a prompt-closed reasoning block in non-streaming parse and reasoning token counts (#61857) 
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
 2026-10-16 12:54:42 -04:00
Misha GoinClaude Opus 5.5Kevin H. Luu
 		cffc0cfa31 [CI] Fix B200 FusedMoE Layer Kernels hang on NVSHMEM IB setup (#61695) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-16 12:00:12 -04:00
Andreas KaratzasandClaude 706791d516 [CI][AMD] Move 17 test groups from MI300 to MI355 and test large DeepSeek/Nemotron evals on AITER (#61955) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-16 10:58:59 -05:00
Wentao Ye 8e25bbca19 [Refactor] Remove dead code multiple places (#61843) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-16 15:01:07 +00:00
Ayushman Singh de7eeaa87b [Perf] Share Triton JIT warmup compilation pool across registered kernels (#61422) 2026-10-16 08:59:42 -04:00
Martin Hickey 64175d3827 [Bugfix][Frontend] Keep the parser's tool call IDs in batch derender (#61852) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-16 10:27:17 +00:00
alpandmottopanikeiku 4ab99ff136 [Bugfix][Frontend] Honor the constructor chat template in offline scoring (#61990) 
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
 2026-10-16 10:20:46 +00:00
Shimi BandielClaudewang.yuqiKevin H. Luu
 		024c5188bf [Frontend] Resolve logprobs on the parsed streaming derender path (#60437) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-16 18:04:15 +08:00
Hexiang Wang 9a16d43ef0 [Bugfix][Mooncake] Fix MultiConnector stats and proxy SSE responses (#61218) 
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
 2026-10-16 17:54:48 +08:00
Yicong WangOpenAI CodexLucas WilkinsonClaude Opus 5.5Kevin H. Luu
 		fa75d3456d [Feature][PCP] Add DeepSeek-V4.1-Flash support (#59857) 
Signed-off-by: wangyicong <wangyicong@bytedance.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-15 23:17:51 -07:00
dskwe 49cb2bbf71 [Bugfix] Honor code_revision in config-based tokenizer loading (#61611) 
Signed-off-by: Yuan Li <dskwelmcy@163.com>
 2026-10-16 05:21:12 +00:00
Maroon AyoubClaudeRobert Shaw
 		4ac1f8d2ac [Bugfix][Structured Output] Mask grammar-invalid implicit reasoning ends (#61574) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-16 12:53:41 +08:00
maithilijoshi20andCyrus Leung 92dc3dd393 [Bugfix] Honor OPT token embedding width for prompt embeddings (#61736) 
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-16 04:41:50 +00:00
Andreas KaratzasandClaude b466fbacd2 [ROCm][Perf] Any-shape low-latency MoE router GEMM on MI355 (#61895) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-15 23:22:31 -05:00
fangyuchulqh-hhTyler Michael SmithOpenAI Codex2017403603Julien MancusoItay AlroyCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		dd9e07a25f [Feature] Add External LB Mode Support for Elastic EP Scaling (#43202) 
Signed-off-by: lqh-hh <liuqihang6@huawei.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: lqh-hh <liuqihang6@huawei.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: 2017403603 <705328646@qq.com>
Co-authored-by: Julien Mancuso <jmancuso@nvidia.com>
Co-authored-by: Itay Alroy <75032521+itayalroy@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-15 21:29:02 -04:00
Andreas KaratzasandClaude 29345f42a6 [ROCm][Qwen4Exp] Support HC and MoE sequence parallelism on ROCm (#61471) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-15 20:11:17 -05:00
RonaldAo ShenOpenAI CodexAo Shen
 		a8be1500ba [Bugfix][MoE] Restore TP-sharded routed-expert capture for DP=1 (#61388) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: Ao Shen <281812371+aoshen02@users.noreply.github.com>
Signed-off-by: Ao Shen <aoshen@inferact.ai>
Co-authored-by: Ao Shen <281812371+aoshen02@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Ao Shen <aoshen@inferact.ai>
 2026-10-16 00:21:24 +00:00
Jack HuandDouglas Lehr ef38e8d006 [Model][Hardware][AMD][Kernel] Part 2/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_ATTN (#42755) 
Signed-off-by: Jack Hu <Jack.Hu@amd.com>
Signed-off-by: Jack Hu <jingxujaster@hotmail.com>
Signed-off-by: Jack Hu <jack.hu@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-10-15 17:10:14 -07:00
Juntian LiuandClaude Opus 5.5 19b0efce30 [Perf][Core] Skip the null head of sliding-window block tables (#61724) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 16:14:51 -07:00
Woosuk KwonandClaude Opus 5.5 4a89316efa [Core] Drop the Ray driver-node GPU pin left over from the legacy executor (#61713) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 16:00:18 -07:00
Omar MuhammadNicholas SusantoSimon Danielsson
 		3f1e57225a [ROCm][Perf][Kimi-K3] Use AITER Gluon KDA kernels for decode and spec decode (#57392) 
Signed-off-by: omuhamma <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
 2026-10-15 17:54:24 -05:00
Nick HillhjjqClaude Opus 5.5
 		de7d70676d [Perf] Low-latency CPU all-reduce for DP metadata syncs (#61767) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: hjjq <50634613+hjjq@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-15 15:29:13 -07:00
Stefano CastagnettaandKevin H. Luu 784d1fff0c [MRV2][Spec Decode] Quantized draft lm_head for drafters that share the target head (#59973) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-15 22:06:40 +00:00
Shern Shiou TanClaude Opus 5.5Simon VeitnerRaphael RiallandLucas Wilkinson
 		d92a4f25f2 [watermarking] Add DFlash support to dual-key gumbel-max watermarking (#59554) 
Signed-off-by: Shern Shiou Tan <shernshiou@gmail.com>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-15 22:06:06 +00:00
Juan VillamizarJaden MathiasjpvillamOmar MuhammadDanielClaude Opus 5Daniel Chan
 		e711723352 [ROCm] gfx1250 optimizations and model enablement (#58800) 
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: Juan Villamizar <jpvillam@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
 2026-10-15 14:51:57 -07:00
DimensionSTPandKevin H. Luu 9edcc7fa7e [Bugfix][Spec Decode] Avoid reusing random draws across block verification rounds (#61161) 
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-15 14:46:38 -07:00
Mikko Tukiainen 0c25849292 [ROCm][Perf][Bugfix] Probe the preshuffled FP4 GEMM config by logical K (#59406) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
 2026-10-15 21:11:33 +00:00
Juntian LiuClaude Opus 5.5Kimi CodeYifan Qiao
 		f8460ef236 [Kernel][DSv4.1] Bump FlashMLA to the 2026.09.30 release and its interleaved V4.1 KV cache (#61371) 
Signed-off-by: juntianl <juntianl@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-10-15 13:58:25 -07:00