iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,240 Commits 3 Branches 0 Tags
ce49c47cbb241b59a32ef110079a5261dbd08e49
Commit Graph
23240 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Akshat AnandcipheraxatBowen Wang
 		ce49c47cbb [Bugfix][Sampler] Carry the boundary tie count across blocks in the split top-p mask kernel (#62046) 
Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: Bowen Wang <bowen@inferact.ai>
 2026-10-17 02:26:57 +00:00
Sherif Walyandvllm-agent ce8853d27a [Bugfix][KV Offload] Allow explicit SimpleCPUOffload block counts (#61674) 
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-17 02:11:12 +00:00
Yang XuZihua WuKimi CodeCodexMisha Goinvllm-agent
 		71c20f73f2 [FlashInfer] Migrate autotune warmup to flashinfer.autotune_v2 managed cache (#55671) 
Signed-off-by: Yang Xu <yanxu@nvidia.com>
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Signed-off-by: YangXu <yanxu@nvidia.com>
Co-authored-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-17 02:08:28 +00:00
Shuolei Wangandaoshen02 b3a7e2ff6e [RL][Frontend] Return weight-version spans on OpenAI completions and chat (#61347) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-17 02:08:07 +00:00
pkoushaOpenAI Codexaoshen02
 		6b6a6c5c98 [RL] Support per-parameter source layouts for NCCL M2N (#60934) 
Signed-off-by: pkousha <pkousha@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-16 21:41:17 -04:00
Harry MellorandClaude Opus 5.5 f8112704d5 [Chore] Use upstream Transformers InklingProcessor (#61883) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-17 01:04:54 +00:00
Omar MuhammadNicholas SusantoDouglas Lehrmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f019e3e105 [ROCm] attn res + sigmoid_mul + conv fusions (#52968) 
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-17 00:54:54 +00:00
Vadim GimpelsonandClaude 783380af39 [Bugfix] Fix NaN output of the Mamba2 chunk scan kernel on sm90 (#61718) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-16 17:24:51 -07:00
Wei ZhaoandCodex be25977532 [CI][Tests] Account for FP8 KV-cache rounding in MLA reference (#62067) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-16 17:10:08 -07:00
Beichen HuangClaude Opus 5.5aoshen02
 		523d74e2ac [Bugfix][LoRA] Fix garbage/NaN lora_expand output on SM90 with fp32 i… (#61906) 
Signed-off-by: BeichenHuang <polarishuang0.0@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-16 19:42:30 -04:00
aoshen02fh3Claude Opus 5.5
 		a94c315711 [DP] Engines own wave wake-ups; drop the front-end FIRST_REQ (#59559) 
Signed-off-by: fh3 <a@b>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 19:41:54 -04:00
wangxiyuanClaude Opus 5.5Yifan Qiao
 		b544587408 [Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning (#53558) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-10-16 23:16:47 +00:00
alpandmottopanikeiku 2268d83bd9 [Bugfix][Multimodal] Honor RGB PNG transparency when loading images (#61986) 
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
 2026-10-16 23:09:12 +00:00
Omar MuhammadNicholas SusantoDouglas LehrKevin H. Luu
 		28e92eeff2 [ROCm][Perf][Kimi-K3] Use AITER Gluon chunk_kda for KDA prefill (#59552) 
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-16 22:58:56 +00:00
Afonso RibeiroandFlora Feng b088eba944 [Perf][Parser] Stream MiniMax M2 tool argument values incrementally (16 KB call: 5.9 s to 26 ms) (#62035) 
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-16 22:46:02 +00:00
Woosuk KwonLainCodexClaude Opus 5.5vllm-agent
 		596e530675 [Kernel] Add router GEMM kernels for (5120, 256) and BF16 output to ll_bf16_gemm (#62042) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Lain <fusiyuan2000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-16 15:39:46 -07:00
Woosuk KwonBugen ZhaoYoni GozlanneilkgCursorClaude Opus 5.5
 		153a17166e [Frontend] Add the Beam reasoning and tool parsers (#62052) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
Co-authored-by: neilkg <33635204+neilkg@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 14:08:29 -07:00
Dan BlanaruandClaude Opus 5.5 9731f30a03 [Bugfix] Keep VLLM_RAY_BUNDLE_INDICES out of the compile cache key (#61211) 
Signed-off-by: DanBlanaru <48605845+DanBlanaru@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 20:59:48 +00:00
Tyler Michael SmithCodexRobert Shaw
 		68ebea789a [Perf][MoE] Skip padded work in SM100 FP4 DeepGEMM experts (#59044) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-16 16:20:52 -04:00
afriedrimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Claude Opus 5.5
 		cc2e4fecf2 [ROCm] [fusion] support ROCm aiter fusion of per tensor quant + rmsnorm (#52831) 
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 15:19:07 -05:00
Tyler Michael SmithCodexMisha GoinKevin H. Luu
 		9eb7fb9157 [Build] Remove unused tools/install_deepgemm.sh (#61265) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-16 15:57:49 -04:00
cjackal 663e535640 [Perf][Spec Decode] Skip bonus-logit reduction for short drafts in rejection sampling (#61451) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-10-16 12:42:48 -07:00
Oxana KorzhandClaude Opus 5.5 5bf3d9f665 [ROCm][Kimi-K3] Support DSpark adaptive verification on MI355X (#59824) 
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-16 13:53:49 -05:00
djramic 307c5844ad [ROCm][CI] Enable batch invariance for the ngram scheduling accuracy test (#62048) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
 2026-10-16 18:36:14 +00:00
Guan-Ming Chiuandvllm-agent 3dc6def360 [Attention][Spec Decode] Enable varlen Mamba2 decode for adaptive verification (#53036) 
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-16 11:23:59 -07:00
djramic e7a5bdb454 [ROCm][CI] Move Spec Decode Speculators + MTP to MI355 (#62053) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
 2026-10-16 18:22:38 +00:00
alberto ae8f41238c [Bugfix][Parser] Honor a prompt-closed reasoning block in non-streaming parse and reasoning token counts (#61857) 
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
 2026-10-16 12:54:42 -04:00
Misha GoinClaude Opus 5.5Kevin H. Luu
 		cffc0cfa31 [CI] Fix B200 FusedMoE Layer Kernels hang on NVSHMEM IB setup (#61695) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-16 12:00:12 -04:00
Andreas KaratzasandClaude 706791d516 [CI][AMD] Move 17 test groups from MI300 to MI355 and test large DeepSeek/Nemotron evals on AITER (#61955) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-16 10:58:59 -05:00
Sawyer BowermanandCodex 76081b9ee4 [Bugfix][Model] Honor quantization config in Inkling MTP (#61921) 
Signed-off-by: Sawyer Bowerman <sbowerma@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-16 15:49:26 +00:00
bnellnm 1a5b187066 Add @bnellnm to CODEOWNERS (#62025) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-10-16 11:16:03 -04:00
Wentao Ye 8e25bbca19 [Refactor] Remove dead code multiple places (#61843) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-16 15:01:07 +00:00
Avinash Paulavininjamay8Kevin H. Luuvllm-agent
 		8a6c0449b3 [Bugfix][ROCm][GLM-5.2] Stop forcing skip_final_all_reduce on AMD DSA model path (#58589) 
Signed-off-by: avininjamay8 <Avinash.Paul@amd.com>
Signed-off-by: avininjamay8 <avpaul@amd.com>
Co-authored-by: avininjamay8 <avpaul@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-16 21:38:55 +08:00
Ayushman Singh de7eeaa87b [Perf] Share Triton JIT warmup compilation pool across registered kernels (#61422) 2026-10-16 08:59:42 -04:00
Martin Hickey 64175d3827 [Bugfix][Frontend] Keep the parser's tool call IDs in batch derender (#61852) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-16 10:27:17 +00:00
alpandmottopanikeiku 4ab99ff136 [Bugfix][Frontend] Honor the constructor chat template in offline scoring (#61990) 
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
 2026-10-16 10:20:46 +00:00
Shimi BandielClaudewang.yuqiKevin H. Luu
 		024c5188bf [Frontend] Resolve logprobs on the parsed streaming derender path (#60437) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-16 18:04:15 +08:00
Hexiang Wang 9a16d43ef0 [Bugfix][Mooncake] Fix MultiConnector stats and proxy SSE responses (#61218) 
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
 2026-10-16 17:54:48 +08:00
zofia 07f3857fb9 [XPU][Linear][MXFP4] enable torch as mxfp4 linear backend on xpu (#51965) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Signed-off-by: zofia <110436990+zufangzhu@users.noreply.github.com>
 2026-10-16 17:07:05 +08:00
Yicong WangOpenAI CodexLucas WilkinsonClaude Opus 5.5Kevin H. Luu
 		fa75d3456d [Feature][PCP] Add DeepSeek-V4.1-Flash support (#59857) 
Signed-off-by: wangyicong <wangyicong@bytedance.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-15 23:17:51 -07:00
Chaojun Zhangandvllm-agent 074c83a09a [CI][Intel] Skip ROCm-only test_topk_gating_enabled_by_cdna on XPU (#61932) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-16 13:44:32 +08:00
Wei Zhao e365792997 [Dependency] Upgrade TokenSpeed MLA to 2.0.3 (#61689) 
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-10-15 22:28:59 -07:00
dskwe 49cb2bbf71 [Bugfix] Honor code_revision in config-based tokenizer loading (#61611) 
Signed-off-by: Yuan Li <dskwelmcy@163.com>
 2026-10-16 05:21:12 +00:00
Maroon AyoubClaudeRobert Shaw
 		4ac1f8d2ac [Bugfix][Structured Output] Mask grammar-invalid implicit reasoning ends (#61574) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-16 12:53:41 +08:00
maithilijoshi20andCyrus Leung 92dc3dd393 [Bugfix] Honor OPT token embedding width for prompt embeddings (#61736) 
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-16 04:41:50 +00:00
Andreas KaratzasandClaude b466fbacd2 [ROCm][Perf] Any-shape low-latency MoE router GEMM on MI355 (#61895) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-15 23:22:31 -05:00
Benjamin ChislettClaude Opus 5.5vllm-agent
 		642c18d64c [Perf] Kimi K3 Specialized Decode: CuTeDSL Optimized GB300 (1.5-2x) (#61867) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-15 22:50:41 -04:00
Sandeep MaddipatlaClaude Opus 5.5Kevin H. Luu
 		3ddc6702a8 [XPU][Docker] Bump nixl to 1.5.0 (#60356) 
Signed-off-by: Sandeep Maddipatla <sandeep.maddipatla@intel.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-16 10:35:29 +08:00
Shuolei Wang 05ad0a8ace [Perf][DSv4.1] Split fused_wqa_wkv by token across TP ranks on SM90 (#58872) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
 2026-10-16 02:31:34 +00:00
fangyuchulqh-hhTyler Michael SmithOpenAI Codex2017403603Julien MancusoItay AlroyCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		dd9e07a25f [Feature] Add External LB Mode Support for Elastic EP Scaling (#43202) 
Signed-off-by: lqh-hh <liuqihang6@huawei.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: lqh-hh <liuqihang6@huawei.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: 2017403603 <705328646@qq.com>
Co-authored-by: Julien Mancuso <jmancuso@nvidia.com>
Co-authored-by: Itay Alroy <75032521+itayalroy@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-15 21:29:02 -04:00