iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,722 Commits 3 Branches 0 Tags
295c062db8a48ea73bbd05d1174cb37d24bf86f9
Commit Graph
1746 Commits
Author SHA1 Message Date
Yifan QiaoandClaude Opus 5.5 240785b82c [Bugfix] Keep the GLM-5.3 kpool tail and Qwen4 QSA ring out of the null block (#59528) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 20:44:04 +00:00
Wei ZhaoWei ZhaoCodexzjy0516Claude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		d8fa463e2b [Bugfix][Prefix caching] Fix inconsistent hybrid Mamba prefix cache boundaries (#60533) 
Signed-off-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-08 18:33:48 +00:00
Elvir CrnčevićandMisha Goin 8674a794c7 [Perf] Enable Kimi K3 shared expert sharding with DeepEP v2 (#56875) 
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 16:42:15 +00:00
fxmarty-amdOpenAI CodexMisha Goin
 		9f3d70e081 [Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) (#55686) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 12:04:46 -04:00
Mikko TukiainenandCursor e2414ec1db [ROCm][Qwen4Exp] Use fused PLE Triton kernels on the AMD backend (#60021) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-10-08 15:47:56 +00:00
kliuae d80ddaac73 [ROCm][Perf] Kimi-K3 Store only the current rank's shards in latent MoE up-proj (#59591) 
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
 2026-10-08 14:55:20 +00:00
Hexiang WangOpenAI CodexFangzhou Ai
 		458ba2edf8 [ROCm][Kimi-K3] Fuse AttnRes output with per-token FP8 input quantization (#59069) 
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
 2026-10-08 12:01:47 +00:00
Hui ShiLuneExplorerArmageddonKnight
 		81198e97ba [Refactor] Abstract DeepGemm MegaMOE backend (#58053) 
Signed-off-by: Hui Shi <kalinshi@tencent.com>
Co-authored-by: LuneExplorer <liyihan23@mails.tsinghua.edu.cn>
Co-authored-by: ArmageddonKnight <bojian@cs.toronto.edu>
 2026-10-08 05:05:39 +00:00
aniskumar-nvCyrus LeungClaude Fable 5.1
 		775792f2d5 [Bugfix] Reland #53020: tie lm_head.weight for Nemotron Parse when checkpoint omits it (#60272) 
Signed-off-by: Anish Kumar <aniskumar@nvidia.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-08 04:40:55 +00:00
Hyukjoon LeeClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Summer Yang
 		1fbbfa1657 [Bugfix][Kimi-K3] Declare max_tp_shards on the DSpark MLA KV cache spec (#59733) 
Signed-off-by: hyukjlee <hyukjlee@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
 2026-10-08 00:33:32 -04:00
Luciano MartinsLuciano MartinsMatthew Bonanni
 		b803edbc9f [Model] Streamline EmbeddingGemma 2 config resolution and harmonize Triton prefill attention (#60289) 
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-07 14:56:40 -07:00
Simon Danielsson 947dd62c07 [CI] Fix failing test_deepseek_v4_rocm_dspark* tests (#60462) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
 2026-10-07 14:18:19 -07:00
Harry MellorandClaude Opus 5.5 a8260cbc5c [CI] Bump Transformers version to 5.19.0 (#60381) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 10:01:41 -07:00
stefankoncarevic 71f9a74920 [CI][ROCm] Re-enable the DSv4.1 decoder replay graph test on ROCm (#60392) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-10-07 09:57:52 -05:00
Micah WilliamsonandFangzhou Ai f6e3944e31 [ROCm][DSv4] AITER MegaMoEV2 Integration For DeepSeek V4 (#59685) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
 2026-10-07 09:35:53 -05:00
stefankoncarevic 3ca00a8261 [CI] Skip EmbeddingGemma2 pooling tests needing transformers >= 5.19.0 (#60371) 
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
 2026-10-07 10:22:31 +00:00
Kevin H. LuuandClaude Opus 5.5 02d8a94e61 [CI][ROCm] Skip DSv4.1 decoder replay graph test on ROCm (#60378) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 02:38:31 -07:00
Turner JabbourandClaude Sonnet 5 aecb717f10 [CI] Allowlist-shrink batch 2: wire 9 tests/models/ files into Buildkite, fix 2 stale bugs found (#59050) 
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-10-06 21:16:53 -07:00
Kevin H. LuuClaude Opus 5.5Andreas Karatzas
 		ae19af883a [CI] Require transformers 5.19 for the EmbeddingGemma2 registry test (#60295) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-06 14:50:10 -07:00
Luciano MartinsandLuciano Martins 02b83919aa [Model] Support EmbeddingGemma2 multimodal pooling architecture (#60254) 
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
 2026-10-06 13:43:38 -04:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		eb2901b735 [Perf][DSv4.1] Run decoder replay layers in CUDA graphs and trim in PIECEWISE graphs (#59532) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-05 22:11:45 -07:00
Brandon ZhangClaude Fable 5.1Wei Zhao
 		b2863c0de9 [Kimi-K3] Use the fused K/V pack kernel on the decode-context-parallel prefill path (#56881) 
Signed-off-by: Brandon Zhang <31413216+brandonfzhang@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-10-06 00:13:55 +00:00
Vadim GimpelsonCursor Agentmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0eac152707 [Bugfix] Route every speculation-capable row through the speculative path so recurrent state stays consistent (#56531) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-06 03:03:44 +04:00
stefankoncarevicandAndreas Karatzas 87954c5b04 [CI/Build] Declare the video modality on the CohereCompass reference model (#60063) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-05 16:25:42 +00:00
Mikko TukiainenandCursor d23af12cb1 [ROCm][Perf] Reach the fused QSA pre-indexer from the AMD path (#57947) 
Signed-off-by: mjkvaak-amd <mjkvaak-amd@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-10-05 16:22:29 +00:00
Abhay ShuklaandClaude Opus 5.5 d3547f9d03 [Test] Re-enable Voxtral HF reference test on Transformers v5 (#59771) 
Signed-off-by: AbhayShuklaIIT <abhay.shukla@live.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 13:43:49 +00:00
cjackal 0e468adb43 [Model] Extend device-side mm normalization to Kimi K2.5 / K3 (#59278) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-10-05 21:28:13 +08:00
Stefano CastagnettaandThien Tran 4ff028d77e [Bugfix][Qwen4Exp] Honor --kv-cache-dtype-skip-layers in QSA attention (#60023) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-05 12:13:30 +00:00
Mikko TukiainenCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>TJian
 		edca360f13 [Bugfix][Qwen4Exp] Load PLE tables unquantized under Quark checkpoints (#59443) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-10-05 10:40:02 +00:00
bcsdhjewmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Codex
 		ae53b06898 [Bugfix][GLM-5.3-Flash] Keep batch x heads out of gridDim.z in the GLM-5.3-Flash fused recurrent KDA kernel (#56974) 
Signed-off-by: NolenLiang <nliang@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-05 09:31:45 +00:00
aoshen02andClaude Opus 5.5 4a30c4cad0 [Model][DeepSeek-V4] Make MegaMoE shared-expert finalize independent of linear post-load order (#59927) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 08:39:49 +00:00
Harshal JanjaniHarry MellorClaude Opus 5
 		34051ad714 feat: Add video support for the Transformers backend (#57441)
pre-commit / pre-run-check (push) Canceled after 0s
Details
pre-commit / pre-commit (push) Canceled after 0s
Details 
Signed-off-by: Harshal Janjani <harshaljanjani@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-10-05 00:51:15 -07:00
Thien TranCodexnamgyu-youn
 		042ab0305c [Perf][Qwen4Exp] Merge QSA QKVG and indexer QK projections (#59533) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: namgyu-youn <namgyu.dev@gmail.com>
 2026-10-05 07:03:29 +00:00
Vadim GimpelsonandClaude 4f52fa35ef [Bugfix] Fix Qwen4Exp PLE embedding rejecting INC (AutoRound) checkpoints (#59990) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-05 01:52:06 +00:00
Harry Mellor 9e0b6fcf77 [CI] Bump Transformers version to 5.18.0 (#59621) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-10-04 00:42:35 +00:00
Markus Hartikainen b0e21b3083 [ROCm][Kimi-K3] Add opt-in gfx942 MXFP4-to-int4 conversion (#51274) 
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Signed-off-by: Hartikainen <mahartik@amd.com>
 2026-10-03 14:28:02 +08:00
Julien Debache 111f71a6db [feat] FlashInfer CuteDSL MegaMoE integration (#54049) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-10-02 15:43:33 -04:00
Thang NguyenandClaude Opus 5.5 7cfd233538 Revert "[Bugfix] Tie lm_head.weight for Nemotron Parse when checkpoint omits it" (#53020) (#59805) 
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 10:19:25 -07:00
Jiangyun ZhuandClaude Opus 5.5 d648847fe4 [CI][Kimi-K3] Test prefix cache reuse with KV offload, P/D and DCP (#59229) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 20:58:35 +08:00
Abhay ShuklaandClaude Opus 5.5 fa212b8dc5 [Test] Re-enable Ovis2.5 and Ovis2.6-MoE vLLM tests on Transformers v5 (#59763) 
Signed-off-by: AbhayShuklaIIT <abhay.shukla@live.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 11:59:40 +00:00
Harry MellorandClaude Opus 5.5 5d20979fc9 [Misc] Remove code paths for Transformers < 5.16.1 (#59762) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 11:17:24 +00:00
Simon Danielssonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 98b29aa99a [ROCm][Perf][GLM-5.3-Flash] Add AITER topk backend for decodes (#58167) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-02 11:47:41 +01:00
Yufeng HeKimi CodeIsotr0py
 		5688a4dd4a [Bugfix][GLM-5.3] Size the image encoder cache from the exact token ceiling (#59565) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Kimi Code <kimi-code@moonshot.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-02 10:43:29 +00:00
Juan Pérez de Algaba c15672cfb4 [Security] Accept zero-sum DeepSeek-OCR pixel tensors (#59417) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
 2026-10-02 09:17:12 +00:00
kliuae 0154979656 [ROCm][Perf] Kimi-K3 enable prefill checkpoints on ROCm (#58344) 
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
 2026-10-02 06:58:04 +00:00
Harry MellorandClaude Opus 5 e3cae8d2ac [Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor (#57387) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-10-02 02:25:18 +00:00
aniskumar-nvandCyrus Leung 83cadd65d9 [Bugfix] Tie lm_head.weight for Nemotron Parse when checkpoint omits it (#53020) 
Signed-off-by: Anish Kumar <aniskumar@nvidia.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-01 09:26:33 -07:00
Shuolei WangandThien c9578f10ab [Qwen3.8-Flash-Next] Fuse main QK-norm/RoPE/gate and KV-cache write into the QSA pre-indexer launch (#57097) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Thien <thien@inferact.ai>
Co-authored-by: Thien <thien@inferact.ai>
 2026-10-01 14:21:06 +08:00
Duncan MossRoger Wangmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Thien Tran
 		766b4ae54c [Kimi-K3] Add FlashInfer speculative KDA backend (#54255) 
Signed-off-by: Duncan Moss <dmoss@nvidia.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-01 02:28:50 +00:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00