iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,341 Commits 3 Branches 0 Tags
patches
Commit Graph
23341 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Kevin H. LuuClaude Opus 5.5Jee Jee Livllm-agent
 		08a65e287e [Bugfix][Kernel] hadacore: don't load past the input on a partial last warp (#61944) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-17 00:52:17 -07:00
znzrybandWoosuk Kwon ad2688ab30 [Bugfix] Validate draft-token capacity in forward benchmark (#61809) 
Signed-off-by: znzryb <znzryb34@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-10-17 00:51:36 -07:00
JovanandOpenAI Codex 5be4266f5c [Bugfix][Frontend] Preserve Cohere and Anthropic streaming errors (#61947) 
Signed-off-by: Kevin <jia__kuan@163.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-17 06:10:16 +00:00
Wei Zhao 90d6d871d6 [Attention] Add CuTe DSL backend selection for MLA context prefill (#62069) 
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-10-17 01:21:55 -04:00
Cy ChengandCy d74f4180cd [Bugfix][Frontend] Detokenize echoed prompts after truncation (#61876) 
Signed-off-by: Cy <15118312+cycancode@users.noreply.github.com>
Co-authored-by: Cy <15118312+cycancode@users.noreply.github.com>
 2026-10-17 05:20:38 +00:00
Jee Jee Li 39983a5e7a [Kernel] Add FP32 router GEMM support for Step-5 Preview and Muse Spark Open (#62126) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-10-16 22:17:55 -07:00
YukioZzzOpenAI CodextjtanaaClaude Opus 5.5
 		41b8c15809 [Feature][MoRIIO] Support hybrid WRITE transfers with DSpark (#60544) 
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-17 04:57:58 +00:00
Micah WilliamsonandCursor a0e7137c4b [CI][ROCm][Bugfix] DeepSeek V4.1: don't pass expert intermediate padding to AITER (#61960) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-10-17 04:25:39 +00:00
Bowen WangClaude Opus 5.5Nick Hillvllm-agent
 		7b6310c109 [Perf][Sampler] Speed up the small-batch split top-p pipeline (#61996) 
Signed-off-by: Bowen Wang <bowen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-16 21:22:11 -07:00
Fadi Arafehandvllm-agent 556ac15eb9 [Bugfix] Remove extra args from CPU's compute slot mapping kernel (#61992) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-17 12:11:33 +08:00
Gabriel WuandClaude Opus 5.5 dcd07967ca [CPU] Keep the hybrid block size across the engine core's config rerun (#61805) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-17 12:06:03 +08:00
Akshat AnandcipheraxatBowen Wang
 		ce49c47cbb [Bugfix][Sampler] Carry the boundary tie count across blocks in the split top-p mask kernel (#62046) 
Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: Bowen Wang <bowen@inferact.ai>
 2026-10-17 02:26:57 +00:00
Sherif Walyandvllm-agent ce8853d27a [Bugfix][KV Offload] Allow explicit SimpleCPUOffload block counts (#61674) 
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-17 02:11:12 +00:00
Yang XuZihua WuKimi CodeCodexMisha Goinvllm-agent
 		71c20f73f2 [FlashInfer] Migrate autotune warmup to flashinfer.autotune_v2 managed cache (#55671) 
Signed-off-by: Yang Xu <yanxu@nvidia.com>
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Signed-off-by: YangXu <yanxu@nvidia.com>
Co-authored-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-17 02:08:28 +00:00
Shuolei Wangandaoshen02 b3a7e2ff6e [RL][Frontend] Return weight-version spans on OpenAI completions and chat (#61347) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-17 02:08:07 +00:00
pkoushaOpenAI Codexaoshen02
 		6b6a6c5c98 [RL] Support per-parameter source layouts for NCCL M2N (#60934) 
Signed-off-by: pkousha <pkousha@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-16 21:41:17 -04:00
Harry MellorandClaude Opus 5.5 f8112704d5 [Chore] Use upstream Transformers InklingProcessor (#61883) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-17 01:04:54 +00:00
Omar MuhammadNicholas SusantoDouglas Lehrmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f019e3e105 [ROCm] attn res + sigmoid_mul + conv fusions (#52968) 
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-17 00:54:54 +00:00
Vadim GimpelsonandClaude 783380af39 [Bugfix] Fix NaN output of the Mamba2 chunk scan kernel on sm90 (#61718) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-16 17:24:51 -07:00
Wei ZhaoandCodex be25977532 [CI][Tests] Account for FP8 KV-cache rounding in MLA reference (#62067) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-16 17:10:08 -07:00
Beichen HuangClaude Opus 5.5aoshen02
 		523d74e2ac [Bugfix][LoRA] Fix garbage/NaN lora_expand output on SM90 with fp32 i… (#61906) 
Signed-off-by: BeichenHuang <polarishuang0.0@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-16 19:42:30 -04:00
aoshen02fh3Claude Opus 5.5
 		a94c315711 [DP] Engines own wave wake-ups; drop the front-end FIRST_REQ (#59559) 
Signed-off-by: fh3 <a@b>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 19:41:54 -04:00
wangxiyuanClaude Opus 5.5Yifan Qiao
 		b544587408 [Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning (#53558) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-10-16 23:16:47 +00:00
alpandmottopanikeiku 2268d83bd9 [Bugfix][Multimodal] Honor RGB PNG transparency when loading images (#61986) 
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
 2026-10-16 23:09:12 +00:00
Omar MuhammadNicholas SusantoDouglas LehrKevin H. Luu
 		28e92eeff2 [ROCm][Perf][Kimi-K3] Use AITER Gluon chunk_kda for KDA prefill (#59552) 
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-16 22:58:56 +00:00
Afonso RibeiroandFlora Feng b088eba944 [Perf][Parser] Stream MiniMax M2 tool argument values incrementally (16 KB call: 5.9 s to 26 ms) (#62035) 
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-16 22:46:02 +00:00
Woosuk KwonLainCodexClaude Opus 5.5vllm-agent
 		596e530675 [Kernel] Add router GEMM kernels for (5120, 256) and BF16 output to ll_bf16_gemm (#62042) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Lain <fusiyuan2000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-16 15:39:46 -07:00
Woosuk KwonBugen ZhaoYoni GozlanneilkgCursorClaude Opus 5.5
 		153a17166e [Frontend] Add the Beam reasoning and tool parsers (#62052) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
Co-authored-by: neilkg <33635204+neilkg@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 14:08:29 -07:00
Dan BlanaruandClaude Opus 5.5 9731f30a03 [Bugfix] Keep VLLM_RAY_BUNDLE_INDICES out of the compile cache key (#61211) 
Signed-off-by: DanBlanaru <48605845+DanBlanaru@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 20:59:48 +00:00
Tyler Michael SmithCodexRobert Shaw
 		68ebea789a [Perf][MoE] Skip padded work in SM100 FP4 DeepGEMM experts (#59044) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-16 16:20:52 -04:00
afriedrimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Claude Opus 5.5
 		cc2e4fecf2 [ROCm] [fusion] support ROCm aiter fusion of per tensor quant + rmsnorm (#52831) 
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 15:19:07 -05:00
Tyler Michael SmithCodexMisha GoinKevin H. Luu
 		9eb7fb9157 [Build] Remove unused tools/install_deepgemm.sh (#61265) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-16 15:57:49 -04:00
cjackal 663e535640 [Perf][Spec Decode] Skip bonus-logit reduction for short drafts in rejection sampling (#61451) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-10-16 12:42:48 -07:00
Oxana KorzhandClaude Opus 5.5 5bf3d9f665 [ROCm][Kimi-K3] Support DSpark adaptive verification on MI355X (#59824) 
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-16 13:53:49 -05:00
djramic 307c5844ad [ROCm][CI] Enable batch invariance for the ngram scheduling accuracy test (#62048) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
 2026-10-16 18:36:14 +00:00
Guan-Ming Chiuandvllm-agent 3dc6def360 [Attention][Spec Decode] Enable varlen Mamba2 decode for adaptive verification (#53036) 
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-16 11:23:59 -07:00
djramic e7a5bdb454 [ROCm][CI] Move Spec Decode Speculators + MTP to MI355 (#62053) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
 2026-10-16 18:22:38 +00:00
alberto ae8f41238c [Bugfix][Parser] Honor a prompt-closed reasoning block in non-streaming parse and reasoning token counts (#61857) 
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
 2026-10-16 12:54:42 -04:00
Misha GoinClaude Opus 5.5Kevin H. Luu
 		cffc0cfa31 [CI] Fix B200 FusedMoE Layer Kernels hang on NVSHMEM IB setup (#61695) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-16 12:00:12 -04:00
Andreas KaratzasandClaude 706791d516 [CI][AMD] Move 17 test groups from MI300 to MI355 and test large DeepSeek/Nemotron evals on AITER (#61955) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-16 10:58:59 -05:00
Sawyer BowermanandCodex 76081b9ee4 [Bugfix][Model] Honor quantization config in Inkling MTP (#61921) 
Signed-off-by: Sawyer Bowerman <sbowerma@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-16 15:49:26 +00:00
bnellnm 1a5b187066 Add @bnellnm to CODEOWNERS (#62025) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-10-16 11:16:03 -04:00
Wentao Ye 8e25bbca19 [Refactor] Remove dead code multiple places (#61843) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-16 15:01:07 +00:00
Avinash Paulavininjamay8Kevin H. Luuvllm-agent
 		8a6c0449b3 [Bugfix][ROCm][GLM-5.2] Stop forcing skip_final_all_reduce on AMD DSA model path (#58589) 
Signed-off-by: avininjamay8 <Avinash.Paul@amd.com>
Signed-off-by: avininjamay8 <avpaul@amd.com>
Co-authored-by: avininjamay8 <avpaul@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-16 21:38:55 +08:00
Ayushman Singh de7eeaa87b [Perf] Share Triton JIT warmup compilation pool across registered kernels (#61422) 2026-10-16 08:59:42 -04:00
MOVZX be95edb7f6 Merge branch 'main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-16 17:35:42 +07:00
Martin Hickey 64175d3827 [Bugfix][Frontend] Keep the parser's tool call IDs in batch derender (#61852) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-16 10:27:17 +00:00
alpandmottopanikeiku 4ab99ff136 [Bugfix][Frontend] Honor the constructor chat template in offline scoring (#61990) 
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
 2026-10-16 10:20:46 +00:00
Shimi BandielClaudewang.yuqiKevin H. Luu
 		024c5188bf [Frontend] Resolve logprobs on the parsed streaming derender path (#60437) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-16 18:04:15 +08:00
Hexiang Wang 9a16d43ef0 [Bugfix][Mooncake] Fix MultiConnector stats and proxy SSE responses (#61218) 
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
 2026-10-16 17:54:48 +08:00