iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
22,313 Commits 3 Branches 0 Tags
c4fc3c7d322bbcd1b25307ccbd9770c8eebc5c09
Commit Graph
22313 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jiangyun ZhuchanhClaude Opus 5.5
 		c4fc3c7d32 [Bugfix][Spec Decode] Per-module LM heads for multi-layer MTP on Model Runner V2 (#58921) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: chanh <channguyen@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 01:09:14 -07:00
Robert ShawRobert ShawClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick Hill
 		e5e38ba9b7 [Model Runner V2] Support randomized dummy inputs (#58411) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-10-01 00:52:26 -07:00
Ganesh RCursorDouglas Lehr
 		5addde2a0a [CPU][Zen] Pass f32 weight scales to the zentorch INT8 MoE (#59434) 
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-10-01 15:51:53 +08:00
jiangyunfan1andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> e12b735109 [Test] Make Anthropic messages test compatible with SDK 1.x via extra_body (#57780) 
Signed-off-by: jiangyunfan1 <jiangyunfan1@h-partners.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 07:38:29 +00:00
Ganesh RCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e12291d733 [CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers (#54024) 
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 15:18:25 +08:00
Wentao Ye 4d6874d137 [GLM 5.3 Perf] Skip qlnorm calculation for MHA, 4.4~7.7% E2E TTFT Improvement (#58845) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-01 15:13:57 +08:00
Mehdi GhanimifardCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Cyrus Leung
 		aee8fe202a [Bugfix][ROCm] Fix race on AITER MLA FP8 prefill scheduling metadata under async scheduling (#58887) 
Signed-off-by: Mehdi Ghanimifard <mehdi.ghanimifard@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-01 15:02:52 +08:00
Luca MotzClaude Opus 5.5Vadim Gimpelson
 		87a4bf664f [Perf][GDN] Slice pure spec-decode rows instead of a host-mask gather (#58763) 
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-10-01 06:53:06 +00:00
aoshen02Codexfh3Claude Fable 5.1
 		4a9d6f9572 [Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs (#55935) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 14:43:38 +08:00
Shuolei WangandThien c9578f10ab [Qwen3.8-Flash-Next] Fuse main QK-norm/RoPE/gate and KV-cache write into the QSA pre-indexer launch (#57097) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Thien <thien@inferact.ai>
Co-authored-by: Thien <thien@inferact.ai>
 2026-10-01 14:21:06 +08:00
vx120Cuiping Chengaoshen02Claude Opus 5.5
 		70be6aae19 [Feature][Spec Decode] Support sampling mask replay for MRV2 MTP (#59359) 
Signed-off-by: Cuiping Cheng <chengcuipingswu@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Cuiping Cheng <chengcuipingswu@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 13:51:39 +08:00
Shijie Lyu 37cecc8141 [Bugfix][Tokenizer] Fix off-by-one max_token_id from vocab_size (#59491) 
Signed-off-by: Shijie Lyu <lshjhf@gmail.com>
 2026-10-01 05:43:30 +00:00
Bugen ZhaoandClaude Opus 5.5 b2d3a3b9ad [Rust Frontend] Split argument grammars into schema resolution and per-model rendering (#59395) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 05:43:18 +00:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ac9126e58a [Rust Frontend] Snapshot structural-tag grammars as readable outlines (#59393) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 05:09:43 +00:00
Bugen ZhaoandClaude Opus 5.5 36229e1337 [Rust Frontend] Token-aware marker parsing for Kimi K3 (#58358) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 04:21:50 +00:00
Bugen ZhaoandClaude Opus 5.5 ec5ef92c87 [Rust Frontend] Anchor tokens after pending UTF-8 bytes at their own first byte (#58357) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <bugen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 04:21:50 +00:00
Juntian LiuandClaude Opus 5.5 4c2d277643 [Perf][DSv4.1] Faster Engram host lookups: sorted rows, inline big lookups (#59327) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-01 11:54:44 +08:00
namgyu-younandThien Tran 4ed438b79c [Perf][Qwen4Exp] Add SM100 low-latency decode GEMM plans (#59214) 
Signed-off-by: namgyu-youn <namgyu.dev@gmail.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-01 03:40:55 +00:00
Yoni Gozlan 2eaa3bc5ac [Frontend] Enrich chat_parsing streaming events (#58603) 
Give serving adapters the delimiter spans and opener captures they need,
and keep streaming after a region that fails to parse.

Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Fix mypy index types in the event-stream test.
 2026-10-01 11:13:13 +08:00
Ning Xie a37c402f32 [weight loader] add dtype equality validation between parameter and weight (#51792) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-10-01 03:07:31 +00:00
Ning Xie bfbcde26d5 [Observability] add model initializing duration log (#52141) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-10-01 02:54:33 +00:00
Nils Matteson 9e6550bd05 [Bugfix] Register IR providers before hashing configuration (#58756) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
 2026-10-01 02:34:28 +00:00
Hanjie QiuCodexShang Wang
 		bc505fc823 [Bugfix] Support CuTe DSL 4.8.0 block-scale API (#59480) 
Signed-off-by: hjjq <50634613+hjjq@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Shang Wang <samshang.wang@mail.utoronto.ca>
 2026-10-01 02:30:06 +00:00
Duncan MossRoger Wangmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Thien Tran
 		766b4ae54c [Kimi-K3] Add FlashInfer speculative KDA backend (#54255) 
Signed-off-by: Duncan Moss <dmoss@nvidia.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-01 02:28:50 +00:00
VenkyClaude Opus 5.5vadiklyutiy
 		edaf3e7b61 [Attention][SM120] Occupancy-adaptive Triton split-K segment count (#58482) 
Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: vadiklyutiy <156319763+vadiklyutiy@users.noreply.github.com>
 2026-10-01 02:20:33 +00:00
Nils MattesonSimon Momergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		76d3929916 [Docker] Compile Python bytecode at image build time (#55422) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 02:20:17 +00:00
LostFox11LostFox11Nick Hill
 		7314360c9e [Perf][PP] Skip sampled-token broadcasts whose requests leave the engine (#58542) 
Signed-off-by: LostFox11 <wangziyue17@huawei.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: LostFox11 <wangziyue17@huawei.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-30 19:18:55 -07:00
AusarGPT-5.6Claude Opus 5Wei ZhaoWentao Ye
 		34c458d27f [Bugfix] Don't let a structured-output request sample from an unmasked row (#54442) 
Signed-off-by: ArcheyChen <45146080+ArcheyChen@users.noreply.github.com>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: GPT-5.6 <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-01 02:11:44 +00:00
ZhenchengLinClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Roy Wang
 		31ec1c3aa1 [Bugfix][Rust Frontend] Stop vllm-bench chat latency at the last token (#59251) 
Signed-off-by: zhenchenglin <zhl132@ucsd.edu>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Roy Wang <jasonailu87@gmail.com>
 2026-10-01 02:04:19 +00:00
Andreas KaratzasOpenAI CodexOpenAI Codex
 		df8fd42116 [CI][Spec Decode] Add async scheduling accuracy tests (#55840) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-10-01 01:06:24 +00:00
Guy StoneandClaude Fable 5 37d6174096 [Metrics] Add --custom-histogram-buckets to override histogram bucket families (#48867) 
The core engine histograms ship with default bucket boundaries tuned for
typical serving workloads. The `--custom-histogram-buckets` option replaces
the boundaries of one or more bucket families. Use it, for example, to track
sub-300ms latency SLOs with the request-phase histograms, whose smallest
default boundary is 0.3s.

Signed-off-by: Guy Stone <guys@spotify.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-10-01 01:03:39 +00:00
vx120aoshen02Claude Fable 5.1
 		a47e4a5568 [Doc] Score centering via top-k processed logprobs (#59361) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 08:57:00 +08:00
Nick HillandClaude Opus 5.5 c055c1e075 [Core] Include the LoRA path in prefix-cache block hashes (#59335) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 08:37:09 +08:00
VenkyandClaude Opus 5.5 d7c55116ac [CI] Fix MiniMax-M3 PP aux-state test mock after #58648 (#59508) 
Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-30 17:34:52 -07:00
Kevin H. LuuandClaude Opus 5.5 8bc31bd210 [CI] Deflake the multi-API-server metrics test and Mooncake PD ports (#59507) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 17:22:58 -07:00
jackLei 21c08ac458 [CI] Deflake shutdown wait-timeout test by synchronizing on request admission (#58125) 
Signed-off-by: jackLei0901 <42642542+jackLei0901@users.noreply.github.com>
 2026-10-01 00:10:03 +00:00
Aarushi JainandAndreas Karatzas 26ce58b48c [ROCm][CI] Sync two AMD test groups between test-amd.yaml and test_areas (#59499) 
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-30 18:58:07 -05:00
Benjamin ChislettandCopilot 8fb16ea5a6 [Docs] Add PR checklist skill for coding agents (#57084) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-09-30 14:47:28 -07:00
Andrii SkliarandAndrii Skliar deb148e730 [Mamba] Add FlashInfer ReplaySSM support for MTP (#52928) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
 2026-09-30 21:41:17 +00:00
kliuaeandNick Hill 7e583e615c [Bugfix][Core] Fix mamba prefill checkpoint block reservation and prompt-end eviction in align mode (#59175) 
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-30 21:26:54 +00:00
VenkyandClaude Opus 5.5 42f0c17ea7 [Bugfix][PP][Spec Decode] MiniMax-M3 EAGLE3 aux-state relay at PP > 1 and per-stage FlashInfer autotune (#57197) 
Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-01 01:14:24 +04:00
Yueh-Ting (eop) ChenMatthew BonanniClaude Opus 5.5
 		73c7cae4d7 [Bugfix][HiSparse] Stop the host pool feeding device KV cache residency metrics (#58725) 
Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 20:50:01 +00:00
Nils Matteson 51048b13ce [Docs] Clarify snapshot runtime image support (#59127) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
 2026-09-30 20:31:21 +00:00
Matthew BonanniClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		3eb6cec22a [Bugfix][HiSparse] Adopt GPU prefix copies after the hit's allocation (#59282) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-30 15:43:09 -04:00
VenkyClaude Opus 5.5Zijing LiuKyle Liang
 		84f738f187 [Perf][MiniMax-M3] Triton indexer: decode grid retune + SM12.0 split-K (#56151) 
Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: Kyle Liang <199154931+kyleliang-nv@users.noreply.github.com>
 2026-09-30 23:33:23 +04:00
djramicandAndreas Karatzas b56b54bc54 [ROCm][CI] Relax simple-nemotron-h-8b GSM8K threshold on ROCm (#59477) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-30 19:27:21 +00:00
Matthew BonanniandClaude Opus 5.5 a21a3f2ec7 [CI] Add supports_multimodal_inputs to test_executor_replace's mock config (#59476) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 19:04:47 +00:00
Chaojun ZhangandKunshang Ji aba01ef1f7 [XPU][CI] Skip test_core_engine_actor_manager.py on Intel CI (#59338) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-30 13:23:02 -05:00
Aarushi JainandAndreas Karatzas ed3f6d1a56 [ROCm][CI] Remove duplicate MI355 DPX jobs (#59256) 
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-30 17:51:20 +00:00
Aarushi JainandAndreas Karatzas b33e1cb172 [ROCm]Transpose compressed-tensors MoE weights on device (#59253) 
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-30 12:44:40 -05:00