iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,991 Commits 3 Branches 0 Tags
c19c263ac8ea3fd6e1d2d689f0e4e725855d89cf
Commit Graph
200 Commits
Author SHA1 Message Date
Dakai AnYifan QiaoOpenAI CodexOpenAI CodexLucas WilkinsonClaude Opus 5.5Lucas Wilkinson
 		02e55d2af3 [KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-09 15:48:37 -04:00
wangxiyuanandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 072d49bb6e [Platform] Support platform-specific MLA prefill backend selection (#56959) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-08 15:25:21 +00:00
aditi-amd 385d86a6cf [Attention] Add UltraQuant 4-bit KV cache backend (FlyDSL D=256) (#57057) 
Signed-off-by: Aditi Rana <aditi.rana@amd.com>
 2026-10-06 20:42:52 +00:00
Jee Jee Li e3af5bf3ce [LoRA] Code cleanup (#60017) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-10-05 09:15:53 +00:00
DivyClaude Opus 5.5Wentao Ye
 		bd42276e1e [Misc][Platform] Check aligned KV block sizes against every attention backend (#58457) 
Signed-off-by: Divy <divy@coralbricks.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-04 11:13:20 -04:00
Jiangyun ZhuClaude Opus 5.5Thomas Parnell
 		cb6458c5ce [Core] Remove deprecated mamba_cache_mode "all" (#58997) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
 2026-10-01 02:59:37 -07:00
Shanshan Shen 7ba3df63cb [ROCm][Refactor] Move DeepSeek-V4/V4.1 multi-stream overlap gate to ROCm platform (#58983) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-09-28 05:46:28 -07:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
wangxiyuan 44257c2e35 [Feature] Triton kernel dispatcher (#43048) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-09-25 04:07:43 +00:00
DivyandClaude Opus 5 e26547adfb [Bugfix] Pick a KV block size supported by every attention backend (#49845) 
Signed-off-by: Divy <divy@coralbricks.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-23 16:36:35 -04:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
wangxiyuanandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 118e17f5ff [Platform] Move env check function to platform interface (#48599) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-16 10:08:36 +00:00
zwang86Zeyu Wangmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b6ce714354 [Bugfix] Format kernel-import errors eagerly so warning_once does not retain them (#54098) 
Signed-off-by: Zeyu Wang <zwang86@users.noreply.github.com>
Co-authored-by: Zeyu Wang <zwang86@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-15 16:35:54 -04:00
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
+5
Harry Huangzjy0516KungYorkThien TranYifan Qiaoandyluo7Isotr0pyandyluo7OpenAI Codexlishunyang12Vadim GimpelsonIsotr0pyLucas WilkinsonClaude Opus 5OpenAI Codex
 		e126687a9a [Model] Support Qwen3.8-Flash-Next (#53896) 
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: KungYork <firelikest@163.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-31 05:57:56 +00:00
Roger Wang 755492e37d Revert "[Kernel] Gemma-4 FA4 FP8 Kernel" (#52987) 2026-08-19 10:48:46 -07:00
Lucas WilkinsonandClaude 57bd0ed441 [5/N][KV-Cache Layout Refactor] Backend-published KV packing via customize_spec (#51704) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-14 09:47:55 -04:00
Jhao-Ting ChenQixiang LinClaude Opus 4.8Yongye Zhu
 		827a2af806 [Kernel] Gemma-4 FA4 FP8 Kernel (#48666) 
Signed-off-by: Qixiang Lin <qixiangl@nvidia.com>
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
Co-authored-by: Qixiang Lin <qixiangl@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-08-13 17:42:00 -07:00
wangxiyuan 73b83949f7 [Platform] Add check_runner_kv_caches_multi_layer (#51633) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-08-13 19:25:07 +00:00
+13
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodexWei Zhao
 		aeeb36b1f1 [New model] Kimi K3 (#50000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-07-30 03:49:58 -07:00
Alejandro Paredes La TorreandShengqi Chen 0a684ab0c0 [Bugfix] Fix WSL circular import from pin_memory warning_once (#48444) 
Signed-off-by: AlejandroParedesLT <alejandroparedeslatorre@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-07-20 18:30:55 -07:00
Tyler Michael Smith 68b4a1d582 Fix NVML capability lookup for visible devices (#47892) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-07-08 09:07:44 -04:00
sychen52andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 67ff0ae30f Support nvfp4 kv with kv-cache-dtype-skip-layers sliding_window (#42890) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-04 02:29:13 +00:00
Wentao YeandTJian 007b5a52ed [Log] Update to log once (#46511) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-06-24 14:45:16 +00:00
Tyler Michael SmithClaudeCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>kourosh hakhamaneshi
 		ebfbcfe46a Stop setting CUDA_VISIBLE_DEVICES internally in vLLM, add device_ids arg (#45026) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: kourosh hakhamaneshi <kouroshHakha@users.noreply.github.com>
 2026-06-20 13:38:10 -07:00
Jimmy Lee 8bf374955f [Bug Fix] Allow pinned memory for WSL2 (#41496) 
Signed-off-by: Jimmy Lee <hirejimmylee@gmail.com>
 2026-06-16 05:56:26 +00:00
Chris LeonardandShengqi Chen 6e64c1bab1 [10c/n] Migrate MoE kernels to torch stable ABI (#44565) 
Signed-off-by: Chris Leonard <chleonar@redhat.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-06-10 23:02:26 -07:00
Yan Ma 3bb46975bd [XPU][Feature] transparent sleep mode support for XPU platform (#37149) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-06-07 10:45:31 +08:00
Mengqing Caozjy0516mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0c6631f02a [KVCache] Support Pluggable KVCacheSpec (#37505) 
Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-03 09:05:16 -07:00
wangxiyuanandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 02a01496fc [Platform] Add is_cumem_allocator_available (#43838) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-03 10:54:50 +08:00
Raj JoshiandCursor 9090368b65 [Feat] Add support for per GPU worker RDMA NIC selection (#42083) 
Signed-off-by: Raj Joshi <rajjoshi@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-28 12:45:23 -07:00
TJian 0ba46d4b11 [ROCm][DSV4] Enable Tilelang MHC replacing torch/triton mhc (#43679) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-05-28 07:05:28 +00:00
Viktor PusCursorWentao Ye
 		0ce6613b9c platforms: add uses_cpu_device() hook to Platform for DeviceConfig (#42313) 
Signed-off-by: Viktor Pus <viktorpus@tenstorrent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-05-12 12:39:17 -07:00
JartXJim SmithSandermageClaude
 		4f2af1a7c0 [Feature] TurboQuant: support hybrid models and uniform quantization (#39931) 
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: Jim Smith <jhsmith0@me.com>
Co-authored-by: Jim Smith <jhsmith0@me.com>
Co-authored-by: Sandermage <sandermage@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-04 20:14:01 -04:00
Matthew BonanniMichael GoinLucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f3fef12350 [Attention] Abstract the MLA prefill backends and eliminate cuDNN (#32623) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-01 13:36:20 -04:00
haosdentandDarkLight1337 995e9a209e [Bugfix] Use is_integrated to detect UMA GPUs for memory reporting (#35356) 
Signed-off-by: haosdent <haosdent@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-04-13 11:07:40 -07:00
Yan Ma ec68d53b2b Add platform manual_seed_all API (#38468) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-04-10 13:43:50 +08:00
Xiaoshuang Wang f8c3af2d85 [vLLM IR] add import_ir_kernels() to support OOT platforms (#38807) 
Signed-off-by: Icey <1790571317@qq.com>
 2026-04-03 17:25:19 +00:00
JartXyangyang4991Wentao YeIsotr0py
 		2ce3d0ce36 [Feature] KV cache per-token-head INT8/FP8 quantization (#38378) 
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: yangyang4991 <yangyang4991@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2026-04-02 08:13:26 -04:00
Luka GovedičXinyu ChenChaojun ZhangLuka Govedič
 		40bb175027 [vLLM IR] 1/N Implement IR skeleton and rms_norm op (#33825) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Signed-off-by: chzhang <chaojun.zhang@intel.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Luka Govedič <ProExpertProg@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
 2026-03-31 22:15:05 -04:00
Chendi.XueMatthew BonanniNicolò Lucchesi
 		3b1dbaad4e [HMA]Fix corner case when hybrid page_size can not be evenly divided issue (blk_size=64,tp=4) (#37467) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-03-30 16:47:30 +00:00
Cyrus Leung ba2f0acc2d [Misc] Reorganize inputs (#35182) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-25 10:22:54 -07:00
SherryC41andsherryC41 6ec5e9fd37 refactor: abstract deepgemm support into platform (#37519) 
Co-authored-by: sherryC41 <sherry.c.c41@gmail.com>
 2026-03-20 17:54:08 +00:00
Lalithnarayan CChinmay-Kulkarni-AMDTyler Michael SmithTyler Michael SmithClaude Sonnet 4.6
 		7acaea634c In-Tree AMD Zen CPU Backend via zentorch [1/N] (#35970) 
Signed-off-by: Lalithnarayan C <Lalithnarayan.C@amd.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Chinmay-Kulkarni-AMD <Chinmay.Kulkarni@amd.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-15 23:35:35 +00:00
Itay Alroy 5353c9b016 platforms: Fix Ray DP startup crash (#36665) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
 2026-03-11 03:08:55 -07:00
Matthew Bonanni 77a73458e3 Reapply [Attention] Refactor check_and_update_config (#35122) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-09 07:17:14 -07:00
nkm-meta 792cbd64ca Add platform method to enable custom collective ops registration (#34760) 
Signed-off-by: Naina Kuruballi Mahesh <nainakm@meta.com>
 2026-03-05 00:50:32 +00:00
Gregory Shtrasberg 6042e66cd5 [ROCm] Add extra step in config initialization to populate custom ops before compilation config init (#34848) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2026-02-26 16:05:40 +08:00
Kunshang Ji 8ad54a991b [Platform] Add current_platform.num_compute_units interface (#35042) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-02-24 22:22:49 -08:00