iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
22,825 Commits 3 Branches 0 Tags
f2376cdc4da852ec48ab5bcb747fdfa65a4dfd4a
Commit Graph
1757 Commits
Author SHA1 Message Date
Mook bf16c9c139 [Bugfix][Model] K-EXAONE: restore loading of pre-2.0 configs and apply RoPE to the MTP layer (#60187) 
Signed-off-by: godmook <cmoh4135@naver.com>
Signed-off-by: Mook <68294499+Godmook@users.noreply.github.com>
 2026-10-10 15:05:55 +00:00
wang.yuqi c1692c85f1 [Misc] mv chat_utils.py to renderers. (#60522) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-10-10 06:50:34 +00:00
Fangzhou AiandClaude Opus 5.5 193922d6ff [ROCm][Perf][DSv4.1] Mono decode layer for DeepSeek-V4.1-Flash on MI355X (#60397) 
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 20:53:18 -05:00
Dao007foreverWei ZhaoClaude Opus 5.5zjy0516
 		f8e6e365ae [Bugfix][Mooncake] Save the EAGLE attention proof with hybrid Mamba partial tails (#60695) 
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Signed-off-by: Dao007forever <daole@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-10-10 00:08:08 +00:00
Yuvraj Singh BhadoriaYuvrajSinghBhadoria2Cyrus LeungClaude Opus 4.8
 		1a3a7a8f01 [Bugfix] Treat max_tokens as output upper bound, not input-space reservation (#57035) 
Signed-off-by: YuvrajSinghBhadoria2 <245782664+YuvrajSinghBhadoria2@users.noreply.github.com>
Signed-off-by: YuvrajSinghBhadoria2 <yuvrajsinghbhado2030@gmail.com>
Signed-off-by: Yuvraj Singh Bhadoria <yuvrajsinghbhado2030@gmail.com>
Co-authored-by: YuvrajSinghBhadoria2 <245782664+YuvrajSinghBhadoria2@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-10-10 01:15:22 +08:00
Mikko TukiainenandCursor 81c993658e [ROCm][Perf] Fuse main QK-norm/RoPE/gate and KV-cache write into the AMD QSA prepare launch (#59732) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-10-09 21:14:49 +08:00
Damien LaineandCyrus Leung 2186b89a9a [Bugfix][Model] Voxtral realtime: fix boot OOM and engine crash at max_model_len (#45022) 
Signed-off-by: Damien Laine <damien.laine@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-09 17:40:29 +08:00
Harry MellorandClaude Opus 5.5 2c99ee9333 [Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 09:29:03 +00:00
Canlin GuoClaude Opus 5.5Juntian Liu
 		ad73a4740d [Perf][DSv4.1] Trim the last KV-source layer in decoder replay (#59894) 
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Juntian Liu <Juntianl777@gmail.com>
 2026-10-09 06:07:49 +00:00
Rohan PotdarandClaude Opus 5.5 88a419649e [ROCm][Kimi-K3] Use vLLM's FP8 max in the fused AttnRes quantization (#60717) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 04:45:13 +00:00
Rémi Delhaye a1cd0ec650 [Model] Add zerank-2 reranker (#60415) 
Signed-off-by: rdlhio <contact@rdlh.io>
 2026-10-09 03:01:51 +00:00
Yifan QiaoandClaude Opus 5.5 240785b82c [Bugfix] Keep the GLM-5.3 kpool tail and Qwen4 QSA ring out of the null block (#59528) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 20:44:04 +00:00
Wei ZhaoWei ZhaoCodexzjy0516Claude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		d8fa463e2b [Bugfix][Prefix caching] Fix inconsistent hybrid Mamba prefix cache boundaries (#60533) 
Signed-off-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-08 18:33:48 +00:00
Elvir CrnčevićandMisha Goin 8674a794c7 [Perf] Enable Kimi K3 shared expert sharding with DeepEP v2 (#56875) 
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 16:42:15 +00:00
fxmarty-amdOpenAI CodexMisha Goin
 		9f3d70e081 [Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) (#55686) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 12:04:46 -04:00
Mikko TukiainenandCursor e2414ec1db [ROCm][Qwen4Exp] Use fused PLE Triton kernels on the AMD backend (#60021) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-10-08 15:47:56 +00:00
kliuae d80ddaac73 [ROCm][Perf] Kimi-K3 Store only the current rank's shards in latent MoE up-proj (#59591) 
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
 2026-10-08 14:55:20 +00:00
Hexiang WangOpenAI CodexFangzhou Ai
 		458ba2edf8 [ROCm][Kimi-K3] Fuse AttnRes output with per-token FP8 input quantization (#59069) 
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
 2026-10-08 12:01:47 +00:00
Hui ShiLuneExplorerArmageddonKnight
 		81198e97ba [Refactor] Abstract DeepGemm MegaMOE backend (#58053) 
Signed-off-by: Hui Shi <kalinshi@tencent.com>
Co-authored-by: LuneExplorer <liyihan23@mails.tsinghua.edu.cn>
Co-authored-by: ArmageddonKnight <bojian@cs.toronto.edu>
 2026-10-08 05:05:39 +00:00
aniskumar-nvCyrus LeungClaude Fable 5.1
 		775792f2d5 [Bugfix] Reland #53020: tie lm_head.weight for Nemotron Parse when checkpoint omits it (#60272) 
Signed-off-by: Anish Kumar <aniskumar@nvidia.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-08 04:40:55 +00:00
Hyukjoon LeeClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Summer Yang
 		1fbbfa1657 [Bugfix][Kimi-K3] Declare max_tp_shards on the DSpark MLA KV cache spec (#59733) 
Signed-off-by: hyukjlee <hyukjlee@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
 2026-10-08 00:33:32 -04:00
Luciano MartinsLuciano MartinsMatthew Bonanni
 		b803edbc9f [Model] Streamline EmbeddingGemma 2 config resolution and harmonize Triton prefill attention (#60289) 
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-07 14:56:40 -07:00
Simon Danielsson 947dd62c07 [CI] Fix failing test_deepseek_v4_rocm_dspark* tests (#60462) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
 2026-10-07 14:18:19 -07:00
Harry MellorandClaude Opus 5.5 a8260cbc5c [CI] Bump Transformers version to 5.19.0 (#60381) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 10:01:41 -07:00
stefankoncarevic 71f9a74920 [CI][ROCm] Re-enable the DSv4.1 decoder replay graph test on ROCm (#60392) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-10-07 09:57:52 -05:00
Micah WilliamsonandFangzhou Ai f6e3944e31 [ROCm][DSv4] AITER MegaMoEV2 Integration For DeepSeek V4 (#59685) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
 2026-10-07 09:35:53 -05:00
stefankoncarevic 3ca00a8261 [CI] Skip EmbeddingGemma2 pooling tests needing transformers >= 5.19.0 (#60371) 
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
 2026-10-07 10:22:31 +00:00
Kevin H. LuuandClaude Opus 5.5 02d8a94e61 [CI][ROCm] Skip DSv4.1 decoder replay graph test on ROCm (#60378) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 02:38:31 -07:00
Turner JabbourandClaude Sonnet 5 aecb717f10 [CI] Allowlist-shrink batch 2: wire 9 tests/models/ files into Buildkite, fix 2 stale bugs found (#59050) 
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-10-06 21:16:53 -07:00
Kevin H. LuuClaude Opus 5.5Andreas Karatzas
 		ae19af883a [CI] Require transformers 5.19 for the EmbeddingGemma2 registry test (#60295) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-06 14:50:10 -07:00
Luciano MartinsandLuciano Martins 02b83919aa [Model] Support EmbeddingGemma2 multimodal pooling architecture (#60254) 
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
 2026-10-06 13:43:38 -04:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		eb2901b735 [Perf][DSv4.1] Run decoder replay layers in CUDA graphs and trim in PIECEWISE graphs (#59532) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-05 22:11:45 -07:00
Brandon ZhangClaude Fable 5.1Wei Zhao
 		b2863c0de9 [Kimi-K3] Use the fused K/V pack kernel on the decode-context-parallel prefill path (#56881) 
Signed-off-by: Brandon Zhang <31413216+brandonfzhang@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-10-06 00:13:55 +00:00
Vadim GimpelsonCursor Agentmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0eac152707 [Bugfix] Route every speculation-capable row through the speculative path so recurrent state stays consistent (#56531) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-06 03:03:44 +04:00
stefankoncarevicandAndreas Karatzas 87954c5b04 [CI/Build] Declare the video modality on the CohereCompass reference model (#60063) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-05 16:25:42 +00:00
Mikko TukiainenandCursor d23af12cb1 [ROCm][Perf] Reach the fused QSA pre-indexer from the AMD path (#57947) 
Signed-off-by: mjkvaak-amd <mjkvaak-amd@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-10-05 16:22:29 +00:00
Abhay ShuklaandClaude Opus 5.5 d3547f9d03 [Test] Re-enable Voxtral HF reference test on Transformers v5 (#59771) 
Signed-off-by: AbhayShuklaIIT <abhay.shukla@live.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 13:43:49 +00:00
cjackal 0e468adb43 [Model] Extend device-side mm normalization to Kimi K2.5 / K3 (#59278) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-10-05 21:28:13 +08:00
Stefano CastagnettaandThien Tran 4ff028d77e [Bugfix][Qwen4Exp] Honor --kv-cache-dtype-skip-layers in QSA attention (#60023) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-05 12:13:30 +00:00
Mikko TukiainenCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>TJian
 		edca360f13 [Bugfix][Qwen4Exp] Load PLE tables unquantized under Quark checkpoints (#59443) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-10-05 10:40:02 +00:00
bcsdhjewmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Codex
 		ae53b06898 [Bugfix][GLM-5.3-Flash] Keep batch x heads out of gridDim.z in the GLM-5.3-Flash fused recurrent KDA kernel (#56974) 
Signed-off-by: NolenLiang <nliang@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-05 09:31:45 +00:00
aoshen02andClaude Opus 5.5 4a30c4cad0 [Model][DeepSeek-V4] Make MegaMoE shared-expert finalize independent of linear post-load order (#59927) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 08:39:49 +00:00
Harshal JanjaniHarry MellorClaude Opus 5
 		34051ad714 feat: Add video support for the Transformers backend (#57441)
pre-commit / pre-run-check (push) Canceled after 0s
Details
pre-commit / pre-commit (push) Canceled after 0s
Details 
Signed-off-by: Harshal Janjani <harshaljanjani@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-10-05 00:51:15 -07:00
Thien TranCodexnamgyu-youn
 		042ab0305c [Perf][Qwen4Exp] Merge QSA QKVG and indexer QK projections (#59533) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: namgyu-youn <namgyu.dev@gmail.com>
 2026-10-05 07:03:29 +00:00
Vadim GimpelsonandClaude 4f52fa35ef [Bugfix] Fix Qwen4Exp PLE embedding rejecting INC (AutoRound) checkpoints (#59990) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-05 01:52:06 +00:00
Harry Mellor 9e0b6fcf77 [CI] Bump Transformers version to 5.18.0 (#59621) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-10-04 00:42:35 +00:00
Markus Hartikainen b0e21b3083 [ROCm][Kimi-K3] Add opt-in gfx942 MXFP4-to-int4 conversion (#51274) 
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Signed-off-by: Hartikainen <mahartik@amd.com>
 2026-10-03 14:28:02 +08:00
Julien Debache 111f71a6db [feat] FlashInfer CuteDSL MegaMoE integration (#54049) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-10-02 15:43:33 -04:00
Thang NguyenandClaude Opus 5.5 7cfd233538 Revert "[Bugfix] Tie lm_head.weight for Nemotron Parse when checkpoint omits it" (#53020) (#59805) 
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 10:19:25 -07:00
Jiangyun ZhuandClaude Opus 5.5 d648847fe4 [CI][Kimi-K3] Test prefix cache reuse with KV offload, P/D and DCP (#59229) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 20:58:35 +08:00