iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
4236 Commits
Author SHA1 Message Date
vllmellm e33ee23ee3 [Bugfix] [AITER] [ROCm] Fix Quark MoE Quant Config and AITER Fused MoE quant type logic (#27029) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-10-17 12:51:10 -06:00
Aleksandr MalyshevandAleksandr Malyshev 0925b28a8e [ROCM] MoE fp4 CK kernel (#26545) 
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
 2025-10-17 14:06:33 -04:00
燃and松灵 4c91a28e30 [bugfix] Qwen3-VL fix video incorrect timestamp calculations while do_sample_frames=True (#27104) 
Co-authored-by: 松灵 <wpf272043@alibaba-inc.com>
 2025-10-17 16:26:33 +00:00
Luka Govedič bd7157a071 [torch.compile] Enable attention and allreduce fusion without custom ops enabled (#24604) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-10-17 08:10:23 -06:00
Reima Karhila (AMD)andxaguilar c253745eb8 [Harware][AMD][Model] Triton MoE tuning configs for GLM-4.5 for MI350 and MI355 (#25586) 
Signed-off-by: Reima Karhila <reima.karhila@amd.com>
Signed-off-by: xaguilar <Xavier.AguilarFruto@amd.com>
Co-authored-by: xaguilar <Xavier.AguilarFruto@amd.com>
 2025-10-17 04:56:12 -07:00
Jee Jee Li daec4d2624 [Model]Improve Qwen3VLMoeForConditionalGeneration packed_modules_mapping (#27096) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-17 04:47:00 -07:00
Harry Mellor 6c9fdbf725 [Docs] Replace rst style double-backtick with md single-backtick (#27091) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-17 02:47:34 -07:00
Mengqing Cao e20eba753b [VLM][Refactor] Remove useless func get_input_positions in MRotaryEmbedding (#27088) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-10-17 02:00:30 -07:00
zhrrr 75c7ad9918 [Kernel][Performance] Fuse float cast and renormalize to topk softmax kernel (#26717) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: izhuhaoran <izhuhaoran@qq.com>
 2025-10-17 07:30:35 +00:00
Said TaghadouiniandCyrus Leung 3aeb19a39e [Model] Add support for LightOnOCR (#26916) 
Signed-off-by: Said Taghadouini <taghadouinisaid@gmail.com>
Signed-off-by: Said Taghadouini <84044788+staghado@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-10-17 05:05:24 +00:00
Cyrus Leung 8c017b3490 [Model] Always use Transformers backend for PaliGemma and Gemma3-MM (#26715) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-17 05:03:35 +00:00
Tao He bde9e2272a [Bugfix][Qwen] fixes the weights dtype in qwen3_next: it is actually a bfloat16 (#27030) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
 2025-10-17 03:37:52 +00:00
Boyuan FengandLuka Govedič 08405609cc disable graph partition in custom op (#26952) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Boyuan Feng <fby.1994@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-10-17 11:08:47 +08:00
Lukas Geiger 4d055ef465 Remove unused imports (#26972) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-10-16 19:51:17 -07:00
Cyrus Leung 4d4d6bad19 [Chore] Separate out vllm.utils.importlib (#27022) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-17 00:48:59 +00:00
jiahancandMichael Goin 41d3071918 [NVIDIA] [Perf] Update to leverage flashinfer trtllm FP4 MOE throughput kernel (#26714) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-10-16 16:20:25 -07:00
Harry Mellor fb5e10d3fb Refactor Transformers backend to use mixins (#26906) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-16 21:50:39 +00:00
Bram Wasti b2f78cbad4 [small][batch invariance] Rename the env and internal flags to simplify usage (#26855) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
 2025-10-16 21:40:25 +00:00
Wentao Yeandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> b3dda72c23 [Feature] Migrate DeepGEMM API from get_m_alignment_for_contiguous_layout to get_mk_alignment_for_contiguous_layout (#26935) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-16 16:46:48 -04:00
Varun Sundar RabindranathandVarun Sundar Rabindranath fb0571b077 [GPTOSS][DP/EP][Marlin] Enable GPTOSS Batched DP/EP using Marlin kernels (#25997) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-10-16 12:53:11 -07:00
Kyle SayersandMichael Goin a5464dcf92 [Compressed Tensors] Always clone output for compile robustness (#26849) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-10-16 19:29:59 +00:00
ZiTian Zhao 7bb736d00e Fix Qwen2.5 VL image grid docstring (#27033) 
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
 2025-10-16 09:57:36 -07:00
Jee Jee Li 9f4e30904b [Model] Fix Qwen3VL mm mapping (#27027) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-16 09:45:59 -07:00
bogdanmandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> dcbb3f1871 [Bugfix] Correct LayerNorm epsilon parameter in modernbert.py (#27008) 
Signed-off-by: bogdanm <152898065+bogdan01m@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-16 12:27:44 +00:00
Sungjae LeeandMengqing Cao 00417f4e44 [MISC] fix import violations for re and triton modules (#26654) 
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>
 2025-10-16 03:38:27 -07:00
CSWYF3634076 e51928793e [Model][Bugfix] fix ernie45 vl run failed from shared experts optimization (#26885) 
Signed-off-by: wangyafeng <wangyafeng@baidu.com>
 2025-10-16 03:37:35 -07:00
Cyrus Leung d2740fafbf [Chore] Separate out vllm.utils.collections (#26990) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 08:35:35 +00:00
Bram WastiandWentao Ye 7d8975de84 Deepseek-v3 Batch Invariant on 8xH100 (#26609) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-10-15 22:06:02 -07:00
Vadim Gimpelson 785d8b6410 [PERF] Qwen3-next MTP speedup (change bool mask indexing to index_select / index_copy to reduce d2h) (#26437) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-10-16 12:18:31 +08:00
Cyrus Leung f6cdc9a02f [Chore] Rename utils submodules (#26920) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 03:58:13 +00:00
kliuaevalarLipTJian
 		1317034379 [ROCm][FEAT] Fuse DeepSeek shared experts into AITER fused_moe ops (#24097) 
Signed-off-by: chenjun <junchen2@amd.com>
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: valarLip <103567126+valarLip@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2025-10-16 10:41:34 +08:00
felixzhu555 f96bc3649c [Qwen3-Next] Add tuned MoE config for Qwen3-Next FP8 on H100 tp2 (#26887) 
Signed-off-by: Felix Zhu <felixzhu555@gmail.com>
 2025-10-15 18:55:05 -07:00
XiaobingZhangandMichael Goin 0b99f5d302 support flashinfer_fp4 moe for 5090 gpu (#26669) 
Signed-off-by: XiaobingSuper <xiaobingzhangupc@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-10-15 15:06:47 -04:00
Kaixi Houandmgoin de92d916fe [NVIDIA] Add support for cudnn fp4 gemm via flashinfer (#26107) 
Signed-off-by: kaixih <kaixih@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-10-15 13:53:00 -04:00
XiaobingZhang d796375258 [ModelOpt] Remove NVFP4 MoE K%16==0 constraint (#26891) 
Signed-off-by: XiaobingSuper <xiaobingzhangupc@gmail.com>
 2025-10-15 13:06:17 -04:00
Harry Mellor d3cbaa08dc Lower sevarity of log when model info cache misses due to exception (#26917) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-15 09:01:09 -07:00
Cyrus Leung 136a17fe6e [Chore] Separate out vllm.utils.func (#26904) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 13:03:58 +00:00
wangxiyuan 8f4b313c37 [Misc] rename torch_dtype to dtype (#26695) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-10-15 12:11:48 +00:00
Cyrus Leung f93e348010 [Misc] Remove isort and yapf ignores (#26888) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 12:09:03 +00:00
wang.yuqi f54f85129e [Model][2/N] Improve all pooling task | Support multi-vector retrieval (#25370) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-10-15 11:14:41 +00:00
ant-yy 5c3bae1a6a [Fix] Remove divisibility requirement between num_kv_heads and tp_size in bailing_moe (#26876) 
Signed-off-by: vito.yy <vito.yy@antgroup.com>
 2025-10-15 16:44:04 +08:00
Yongye Zhu f5ed68ef63 [Deepseek-V3.2][Kernel] Integrate cuda indexer k cache gather (#26456) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2025-10-15 16:05:01 +08:00
Mengqing Cao 302ef403a2 [DSA][MLA] Tiny refactor on DeepSeek to make it reusable for different backends (#26656) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-10-15 00:16:44 -07:00
sangho.lee 8865da157b [Bugfix][Multi Modal] Fix incorrect Molmo token processing (#26873) 
Signed-off-by: sanghol <sanghol@allenai.org>
 2025-10-15 07:13:59 +00:00
Isotr0py 8c851f6d04 [Bugfix] Fix qwen3-omni audio truncation issue (#26815) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-10-15 05:38:36 +00:00
Morrison TurnanskyandLuka Govedič 96b9aa5aa0 [Frontend][torch.compile] CompilationConfig Overhaul (#20283): name change compilation level to compilation mode, deprecation compilation level (#26355) 
Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-10-15 02:51:16 +00:00
Michael Goin 7e0ef4084a [CI Failure] Fix torchao dep failure for Quantization Test (#26824) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-10-14 16:41:43 -07:00
Dhruvil Bhatt 0e65818910 Added MoE configs for llama 4, H200 device with tp=4/8 tuning (#26837) 
Signed-off-by: Dhruvil Bhatt <bhattdbh@amazon.com>
 2025-10-14 14:21:03 -07:00
Huamin Li 87efc681db llama4_vision_rope: add HIP override to accept (q, k) and avoid (positions, q, k) mismatch (#26790) 
Signed-off-by: Huamin Li <3ericli@gmail.com>
 2025-10-14 11:54:12 -07:00
Ze'ev Klapow aba48f7db1 [Kernel][MoE] Add MoE tunings for GLM 4.6-FP8 and GLM 4.5 Air on NVidia B200 (#26818) 2025-10-14 11:20:39 -07:00