iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
22,490 Commits 3 Branches 0 Tags
4a3ebc6b5bfcfa8cef4873f7ff213bf6bec4fef0
Commit Graph
294 Commits
Author SHA1 Message Date
MOVZX 85c31d47ad Merge remote-tracking branch 'upstream/main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-03 22:06:10 +07:00
MOVZX 2877121b2d Merge remote-tracking branch 'upstream/main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-03 21:59:03 +07:00
C 44198f577f [Bugfix][Model] Fix M-RoPE offset double-count in Qwen3-Omni (#58890) 
Signed-off-by: C1ves <3121985758@qq.com>
 2026-10-03 09:00:51 +08:00
Eric Tangaoshen02Claude Opus 5.5
 		4c4003bb5f [Bugfix] Bind routed-experts capture to the MoE layer, not the kernel (#59455) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 15:50:38 +00:00
Qiu ChunshuoandLucas Wilkinson 8486f1ebfe [Perf][PCP] Shard decode requests across PCP ranks (#52162) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-01 23:50:13 -04:00
Wentao Ye 4d6874d137 [GLM 5.3 Perf] Skip qlnorm calculation for MHA, 4.4~7.7% E2E TTFT Improvement (#58845) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-01 15:13:57 +08:00
Andrii SkliarandAndrii Skliar deb148e730 [Mamba] Add FlashInfer ReplaySSM support for MTP (#52928) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
 2026-09-30 21:41:17 +00:00
VenkyandClaude Opus 5.5 42f0c17ea7 [Bugfix][PP][Spec Decode] MiniMax-M3 EAGLE3 aux-state relay at PP > 1 and per-stage FlashInfer autotune (#57197) 
Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-01 01:14:24 +04:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
almayneandLi, Jiang <jiang1.li@intel.com> 93dd80f636 [CPU] Conv1d optimised kernel for aarch64 (#54093) 
Signed-off-by: Anna Mayne <anna.mayne@arm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-30 20:23:42 +08:00
Jakub ZakrzewskiandCodex 5a4351ab09 [MM] Keep device input normalization fused when encoder compilation is enabled (#59195) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-30 11:08:24 +00:00
Alexandra Sidorova b81984879a [K2 Horizon] fold partial-RoPE permutation into q/k (and norm) weights (#55335) 
Signed-off-by: Alexandra Sidorova <asidorov@amd.com>
 2026-09-30 16:03:37 +08:00
shiyuan680shiyuan680Ronald1995DeepSeek Harnessaoshen02Claude Opus 5.5aoshen02
 		42d9f8ccdb support rl feature : weight checker (#51350) 
Signed-off-by: shiyuan680 <yangcheng104@huawei.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: shiyuan680 <yangcheng104@huawei.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: DeepSeek Harness <noreply@deepseek.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 03:13:43 +00:00
Martin HickeyHarry MellorClaude Opus 5.5
 		5dd693122c [MyPy][2/N] Fix mypy errors in small tests/ dirs (#51043) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 21:27:45 +01:00
siyuIsotr0pyXun Sun
 		f5ed3d9c03 [Fast Start] Charge daemon-held weights against gpu_memory_utilization (#57298) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Xun Sun <UNIDY2002@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
 2026-09-29 17:58:35 +00:00
siyuIsotr0pyXun Sun
 		af5b4857e1 [Fast Start] Support PP (#55477) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
 2026-09-29 09:37:43 +00:00
MOVZXandHinata cb275fdf20 [Qwen4Exp] Fix MTP buffer placement, config parsing, Triton warmup (PR #56742) 
Intake of open upstream PR vllm-project/vllm#56742. Every rig here runs
MTP on both Qwen3.8 models, so these three bugs sit on our hot path.

The MTP hidden buffer was allocated with torch.empty and no device, so
it followed the ambient default instead of device_config.device; on a
CPU-default process the draft buffer lands on the wrong device. Both
NVIDIA and AMD backends now call make_mtp_hidden_buffer() in
vllm/models/qwen4_exp/common/mtp.py, which pins it to the configured
device.

Config parsing now accepts configs carrying M-RoPE keys and the
Transformers spelling of layer_types; both previously failed at load.

Triton warmup: Qwen4Exp was missing from the Qwen warmup dispatch, so
its Triton kernels compiled during the first served requests instead of
at startup.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-09-29 13:36:05 +07:00
Xun SunandIsotr0py e7900156e1 [Fast Start] Add /health endpoint for the weight cache daemon (#58552) 
Signed-off-by: Xun Sun <UNIDY2002@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-27 17:48:54 +00:00
lesj0610 521235d6df Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes 
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
 2026-09-27 18:25:21 +09:00
Isotr0py 24928a4d62 [UX][Frontend] Introduce vllm preload cli for fast restart (#56680) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 06:56:39 +00:00
wang.yuqiIsotr0pyIsotr0py
 		8a269919f4 [MM] Add Triton kernel for mm_input_normal. (#56798) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 04:31:20 +00:00
wangxiyuan 44257c2e35 [Feature] Triton kernel dispatcher (#43048) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-09-25 04:07:43 +00:00
Samuel Kimandmgoin f55419c43c [Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxfp8 shape limits (#54223) 
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-25 02:54:11 +00:00
Juntian LiuKimi CodeClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		6936e77ba4 [Perf][Engram] Serialize offloaded lookups and pack host tables into huge pages (#56926) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 10:52:02 +08:00
Thien Tran ade1056ee5 [Kimi-K3][Perf] Dispatch GEMM for vision patch embedder (#58527) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-09-24 18:30:11 -07:00
Misha GoinandCodex 71891bdb94 [Perf] Parallelize registered CUDA Triton kernel warmup at startup (#58582) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-24 19:52:00 -04:00
fxmarty-amdmgoinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		90a9515006 [Cleanup] Remove online quantization support in fp8.py in favor of online shorthands (#53585) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 12:48:14 -07:00
Xun Sun e773247f8a [Fast Start] Wait for weight cache daemon readiness (#58370) 2026-09-24 19:18:04 +00:00
Harry MellorandClaude Opus 5.5 d881812500 [Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 14:14:10 +00:00
Thien TranandOpenAI Codex 00b7847c80 [Perf] Use Conv3dLayer for MiniMax M3 patch embedding (#58512) 
Signed-off-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-24 00:41:36 -07:00
GuorongYegrYe99Cursor Agent
 		1291bdb85e [EPD][Model Loader] Skip language-model checkpoint shards for --mm-encoder-only (#58086) 
Signed-off-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-09-24 00:41:15 -07:00
Farzad AbdolhosseiniClaudeLi, Jiang <jiang1.li@intel.com>
 		8b06a343ec [Bugfix][CPU][MoE] Fix out-of-bounds write and segfault when router weights are fp32 (#56168) 
Signed-off-by: Farzad Abdolhosseini <farzad@elastix.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-24 13:15:17 +08:00
Juntian LiuandClaude Opus 5.5 9ef37771be [Bugfix][DSA] Bound DeepSelect sentinel columns in the sparse top-k remap (#58215) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 00:54:27 +00:00
00CC eb2e91d3ba [ROCm][Perf] Use wvSplitK for single-output GEMMs (#53283) 
Signed-off-by: tangzzycc <3081129260@qq.com>
 2026-09-23 06:38:17 +00:00
Tianyu GuoandCyrus Leung 3ee2906f47 [EPD] Support metadata-only audio inputs (#57887) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-23 09:03:50 +08:00
siyuandIsotr0py c024cfdb02 [Fast Start] Support data parallelism in the weight cache daemon (#57386) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-22 18:19:40 +00:00
lesj0610 045467c102 Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes 
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
 2026-09-23 00:40:48 +09:00
siyuandIsotr0py c723a831a8 [Fast Start] Cache the MTP draft model in a separate daemon group (#57312) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-22 03:13:38 +00:00
Hongxin Xuaoshen02OpenAI CodexRoger WangNick HillCodex
 		0b7f11a1ee [AuxOutput] Add block-keyed storage for routed-expert outputs (#45635) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-21 10:25:05 +00:00
lesj0610 51d5b3ac8a Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes 
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
 2026-09-20 22:55:38 +09:00
Misha GoinandErinYin e5fce7b56b [Core][Kernel] Share persistent workspaces for Marlin and Humming (#57421) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: ErinYin <liang.yin@daocloud.io>
 2026-09-20 17:28:33 +08:00
lesj0610 5780c0125a Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes 
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
 2026-09-20 09:27:57 +09:00
Woosuk Kwon 38537331d3 [Bugfix][DSA] Avoid runtime JIT for offset candidate end buffers (#57667) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-09-19 00:27:11 -07:00
Andreas KaratzasandOpenAI Codex 2c3fb4c545 [CI][ROCm] Deprecate DinD for MI250 test groups (#56162) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-18 14:13:35 -07:00
Woosuk KwonandCodex 64563d0ec4 [Bugfix][DSv4.1] Preserve NaN-scored candidate block indices (#57454) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-17 19:00:10 -07:00
vllm-agentkimi-no-na-waMisha Goin
 		b2f2bd71ba [CI][Bugfix] Add tp_shard_with_padding to padded MoE reload test mock (#57402) 
Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com>
Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-17 13:02:24 -04:00
Yuchen WangDakai AnOpenAI Codexaoshen02
 		78fdf4efb8 [Bugfix][MoE] Convert FlashInfer BF16 weights in place (#54699) 
Signed-off-by: Yuchen Wang <93700456+yuchenwang3@users.noreply.github.com>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-17 22:00:04 +08:00
lesj0610 6c5552776b Merge remote-tracking branch 'origin/main' into lesj/qwen4exp-bugfixes 
Both sides edited the same import block in the Qwen4Exp NVIDIA and AMD
models and nothing else collided. This branch adds
`from ..common.mtp import make_mtp_hidden_buffer`, the helper the MTP
buffer extraction calls; upstream ec4a3a5370 widened the `..config`
import to `ATTENTION_LAYER_TYPES, QSA_LAYER_TYPE, Qwen4ExpConfig` for the
layer-type constants it introduced. The two changes are independent, so
the resolution keeps both lines. Every symbol still has a use outside the
import in both files.

Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
 2026-09-17 08:16:59 +09:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00