iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
22,112 Commits 3 Branches 0 Tags
4e59d53ab534fbb4aaa751eb37ecfbcf438855b5
Commit Graph
1681 Commits
Author SHA1 Message Date
Ashraf BhuiyanandWentao Ye 2619b6b72b [Mypy] Fix mypy typing for Voxtral and vision models (#58251) 
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-28 15:52:50 -04:00
xaguilar-amdandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> a7ff44355c [ROCm][Perf] Kimi-K3 Enable sharded latent MoE up-projection under EP (#54956) 
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-28 10:31:29 -04:00
Jiangyun ZhuandOpenAI Codex 20b52e9f5b [Perf][Qwen3.8] Reduce PLE metadata construction overhead (#58114) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-28 20:46:22 +08:00
Thien Tran d6cce94fd4 [Perf][Qwen4Exp] Fuse HC down projection and SiLU on NVIDIA (#58957) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-09-28 18:51:38 +08:00
Gabriel WuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		a2be4d3cc7 [Bugfix][Qwen4Exp] Release the profiling KV cache held by QSA key views (#58961) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-28 08:11:46 +00:00
Kevin H. LuuandClaude Opus 5.5 0da126676e [Bugfix] Fix the two multimodal root tests that fail on main (OpenPangu-VL embed merge, MiMo sink test fixture) (#58900) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-28 04:52:51 +00:00
Luca MotzXin YangClaude Opus 5.5CodexLucas Wilkinson
 		dd3bc9c72f [Perf][MRV2] Reuse Mamba/GDN metadata across KV cache groups (#58762) 
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-09-28 04:36:58 +00:00
Thien TranandCodex 55de40a2fc [Minimax-M3] Add Encoder CUDA graph support (#58673) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <codex@openai.com>
 2026-09-28 04:17:36 +00:00
Wentao Ye bf93e1f92c [Refactor] Remove dead tests utils (#58803) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-26 09:16:53 -04:00
qizixiOpenAI CodexClaude Opus 5.5Nick Hill
 		a4eb3f25d6 [Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer (#57263) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-26 17:01:17 +08:00
Aarushi JainandAndreas Karatzas 3b4566c5cf [CI] Only isolate the registry tests that need a fresh process (#58764) 
Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-26 14:03:45 +08:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		e3dd5b5a75 [ROCm][CI] Test AMD DeepSeek V4 MoE routing against a PyTorch reference (#58740) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 18:05:54 +00:00
jiacao-amdOpenAI CodexAndreas Karatzas
 		1417022c3d [ROCm][Kimi-K3] Optimize low-concurrency speculative KDA (#58045) 
Signed-off-by: jiacao-amd <jiahui.cao@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 12:33:30 -05:00
Andrii SkliarAndrii SkliarCodexBenjamin Chislett
 		73a78e6f1f [SpecDecode] Add LiLiCorr drafter (#57934) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2026-09-25 10:08:28 -04:00
Yongye ZhuandClaude Opus 5.5 378504a544 [MoE] Defer the TRTLLM-Gen top-k finalize on the modular path (#58635) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-25 15:38:29 +08:00
wang.yuqiIsotr0pyIsotr0py
 		8a269919f4 [MM] Add Triton kernel for mm_input_normal. (#56798) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 04:31:20 +00:00
Mathew Oddenopencode+deepseek-v4-flash+vllmAndreas KaratzasOpenAI Codex
 		267eee54bf [Qwen4Exp][ROCm] PLE n-gram table CPU offload (#57497) 
Signed-off-by: Mathew Odden <modden@redhat.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: opencode+deepseek-v4-flash+vllm <opencode+deepseek-v4-flash+vllm@example.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-24 23:28:20 -05:00
mkunreddMahesh KunreddiClaudeAndreas KaratzasCodexAndreas Karatzas
 		4013f3ad6c [ROCm] Fix CI runtime and tests for MI355 DPX (#58244) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Mahesh Kunreddi <mahesh.kunreddi@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 23:15:11 -05:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		48d8880d09 [Bugfix][Qwen4Exp] Keep pinned PLE prefetch ids out of the CUDA graph pool (#58489) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 10:51:50 +08:00
fxmarty-amdmgoinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		90a9515006 [Cleanup] Remove online quantization support in fp8.py in favor of online shorthands (#53585) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 12:48:14 -07:00
nightcitybladenightcitybladeCyrus Leung
 		5b3f280a6d [Bugfix] Count unsplit Idefics3 image patches (#48760) 
Signed-off-by: nightcityblade <nightcityblade@gmail.com>
Co-authored-by: nightcityblade <nightcityblade@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-24 11:36:26 -07:00
Thomas OrtnerandHarry Mellor 8b84e15066 [transformer] RMSNorm matching for alternative rsqrt (#54461) 
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-24 16:33:06 +00:00
Harry MellorandClaude Opus 5.5 d881812500 [Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 14:14:10 +00:00
Linze Shi 484c211fe8 [Bugfix][Pooling] Fix JinaVL label configuration and restore multimodal tests (#57347) 
Signed-off-by: Linze-Shi <linzeshi0@gmail.com>
 2026-09-24 13:51:25 +00:00
Aarushi JainandAndreas Karatzas e0f03eb4f4 [CI] Share BF16 baselines across quantization comparison tests (#58469) 
Signed-off-by: Aarushi Jain <Aarushi.Jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-23 19:26:09 -05:00
Oxana KorzhClaude Opus 5Andreas Karatzas
 		c4cfd7007b [CI][ROCm] Add an MI355 Kimi-K3 unit test group (#58012) 
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-23 11:25:51 -05:00
Tianyu GuoandCyrus Leung 3ee2906f47 [EPD] Support metadata-only audio inputs (#57887) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-23 09:03:50 +08:00
Jared WenClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c9b34fdb2d [Bugfix][GLM-5.3-Flash] Run the dense MLP layers on the sequence-parallel shard (#58061) 
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-22 21:58:21 +08:00
Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> a33b3bac5d [Bugfix][Qwen2.5-VL] Honor video fps for temporal M-RoPE (#47736) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-22 05:56:12 +00:00
Shanshan Shen 42a85a4976 [ROCm][Bugfix] Explicitly reject FSE=1 with DPA+ETP deployment for DeepSeek-V4 (#57919) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-09-22 04:45:53 +00:00
Nils MattesonTaneem IbrahimelehayymAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		13058faa49 [Frontend] Add reusable TP1 initialized-engine snapshots (#51360) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-21 23:18:49 +00:00
yinfengLiuLiuyinfeng01Andreas Karatzas
 		382970ee6c [Kimi-K3][AMD] Return KDA and MLA projection outputs directly (#50592) 
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
Co-authored-by: Liuyinfeng01 <199041580+LiuYinfeng01@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-21 15:44:41 -05:00
djramicandAndreas Karatzas 4f14516790 [ROCm][CI] Use ROCm backend for DeepSeek V4.1 ViT test (#57931) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-21 16:39:31 +00:00
Cyrus Leung 7853700d4c [MM] Move get_dummy_processor_inputs into MM processor (#57967) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-21 09:39:20 -07:00
Sahil PatelandCyrus Leung 4e61ad8ee1 [Bugfix] unskip InternViT test for transformers v5 compatibility (#55767) 
Signed-off-by: sahil <sahil@example.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-21 09:38:25 -07:00
Tuukka Sarviandlarryli2-amd e34685dfc0 [ROCm][DSv4] Enable DSpark adaptive verification (#52362) 
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Signed-off-by: larryli2-amd <larryli2@amd.com>
Co-authored-by: larryli2-amd <larryli2@amd.com>
 2026-09-21 10:03:41 -05:00
Cyrus Leung c8824a1e42 [MM] Move supports_multimodal_inputs and cache out of registry (#57913) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-21 07:43:07 -07:00
Andreas KaratzasandOpenAI Codex 8902dbb177 [CI][ROCm] Mirror remaining portable suites without duplicate coverage (#57877) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-21 04:16:26 -05:00
akii96 0ff0477ff9 [Perf][ROCm] Add a ROCm path for Hy4 and compile the backbone (#57526) 
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
 2026-09-21 08:20:43 +00:00
Oliver Holworthyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 9598487bc1 [Bugfix][Model] Mistral3: align placeholder grid with public processor (#53758) 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-21 06:36:59 +00:00
Tianyao Wu 59f1527280 [Bugfix] Fix Whisper engine crash on audio clips longer than 30s (#57769) 
Signed-off-by: Tianyao Wu <rayroy31@gmail.com>
 2026-09-21 05:44:40 +00:00
Andreas KaratzasandOpenAI Codex 8ec00ec2bc [Tests] Release HF embedding weights after prompt extraction (#57868) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-21 05:27:29 +00:00
Andreas KaratzasandOpenAI Codex 76ee20c1ec [ROCm][Tests] Avoid pooling cleanup waits against live shared engines (#57864) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-21 05:21:35 +00:00
Chaunceyandzjy0516 3df4ae153e [Core][KDA] Generalize Mamba prefill checkpoint builder and exporter (#57783) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-09-21 04:46:11 +00:00
Isotr0py 27757dde02 [DSV4.1] Add encoder cuda graph support for deepseek-v4.1-flash (#56625) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-20 21:51:13 +08:00
Chauncey bf01fc4a31 [GLM-5.3-Flash] Route kpool indexer top-k through the shared SparseIndexerTopk dispatcher (#57546) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-09-20 11:17:59 +00:00
yinfengLiuandShanshan Shen 0e110f696d [ROCm][DSV4][Perf] Use FP8 WO_A output projection (#54894) 
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-20 09:13:55 +00:00
Cyrus Leung 751f6807d9 [Refactor] Use dynamic MM cache in processor (#57674) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-19 15:38:00 +00:00
vllmellm 211e252d0b [Bugfix][Model] Fix MiMo-V2.5 fused fp8 qkv_proj sharding (pre-shard count is num_key_value_heads; MTP path too) (#57508) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2026-09-19 06:05:00 -07:00
Jiang WuandJiang Wu 01c7bf8813 [Bugfix][Model] Fix Aria expert weight names and layout (#57487) 
Signed-off-by: Jiang Wu <245355730+JiangLLM@users.noreply.github.com>
Co-authored-by: Jiang Wu <245355730+JiangLLM@users.noreply.github.com>
 2026-09-18 21:06:30 -07:00