iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
19,362 Commits 3 Branches 0 Tags
f1899b2ffdf87598350c416a074c8c291c845902
Commit Graph
508 Commits
Author SHA1 Message Date
+13
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodexWei Zhao
 		aeeb36b1f1 [New model] Kimi K3 (#50000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-07-30 03:49:58 -07:00
+12
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodex
 		7c6729b769 [Model] Add Kimi K3 support: model files and kernels [1/N] (#50089) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
 2026-07-29 14:10:58 +08:00
Bugen Zhao 01661cc57f [Rust][Benchmark] Make vllm bench serve Rust delegation opt-in (#50081) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-28 16:21:12 +00:00
Nick HillandAndreas Karatzas 6453fc0b8c [Bugfix] Don't reuse engine core payload buffer while zmq is sending it (#50053) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-28 16:09:28 +00:00
Nils Mattesonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 149daf0d72 [Bugfix] Exclude location-derived path vars from torch.compile cache factors (#47573) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-22 16:56:09 -07:00
gnovack 85f638a2b8 skip cudagraph/DP padding in topk (#48979) 
Signed-off-by: gnovack <novackgm@gmail.com>
 2026-07-21 15:20:30 -07:00
sungsoo haClaude Opus 4.8Lucas WilkinsonMatthew Bonanni
 		2396a61108 [Attention][MLA][DCP] Query replication for MLA decode (DeepSeek-V2/R1 + Kimi-K2.5) (#45964) 
Signed-off-by: Sungsoo Ha <sungsooh@nvidia.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-20 23:51:27 +00:00
Daoyuan Li b88abb5036 [Misc] Remove orphaned env vars and stale env-var references (#44749) 
Signed-off-by: Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com>
 2026-07-17 00:00:47 +00:00
liranschourandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 615834ee58 [KVOffload][P2P] Well-known default host/port env vars and per-DP-rank control port (#47636) 
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-15 15:22:56 +03:00
Artur FierkaandClaude 50ac1c7bab [Misc] Rename VLLM_TRITON_ATTN_USE_TD to VLLM_TRITON_USE_TD (#45781) 
Signed-off-by: Artur Fierka <artur.fierka@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-14 10:32:57 +00:00
omerpaz95andOr Ozeri 8df14cfc8c [EC Connector] Add EC Transfer Params (#42433) 
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-07-12 14:35:33 +03:00
Michael GoinandClaude Fable 5 29fd688892 Add VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS and skip CuTeDSL fp4_gemm autotuning by default (#48268) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-10 18:13:08 -07:00
Nicklas Frahm c0e8e1f12a [Bugfix] Register VLLM_BUILD_* and VLLM_IMAGE_TAG provenance env vars (#45313) 
Signed-off-by: Nicklas Frahm <nicklas.frahm@gmail.com>
 2026-07-08 06:21:12 +00:00
Nils MattesonNils Mattesonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e97c3cb303 [Core] Persist and reuse the memory-profiling result across boots (opt-in) (#47388) 
Signed-off-by: Nils Matteson <nils@thaw.sh>
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Co-authored-by: Nils Matteson <nils@thaw.sh>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-08 00:53:02 +00:00
Juan Pérez de Algaba 675f4295cd fix(security): bound completion prompt list to prevent unbounded engine fan-out (#47845) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-07-07 22:48:20 +00:00
BadrBasowid 740f379fae [ROCm][AITER] Directly Implement AITER Custom All-reduce in CudaCommunicator (#46065) 
Signed-off-by: BadrBasowid <badr.basowid@gmail.com>
 2026-07-06 12:16:32 +00:00
Juan Pérez de Algaba 364ee36af1 fix(security): prevent image decompression bomb OOM denial of service (#47010) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-06-30 09:39:22 +00:00
Nick Hill 4e07ca2c92 [Core] Add VLLM_GPU_SYNC_CHECK env var (#44800) 2026-06-26 08:24:33 -07:00
haoyangli0109 1744adc256 [ROCM] [Communication] Add INT3 quantization method for quickreduce (#45666) 
Signed-off-by: Haoyang Li <lihaoyang0109@gmail.com>
 2026-06-25 15:14:15 +00:00
Yifan QiaoClaudemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		9e88e969c0 [Perf][KVConnector][Mooncake] Parallelize KV load with a receive-thread pool (#45971) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-24 18:25:12 -07:00
Lucas Wilkinson e7df232288 [KV Offload] Gate packed HMA KV cache on cross-layer config (#46252) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-06-24 11:55:30 -04:00
Woosuk Kwonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 0a3e2dbc09 [Optimization] Skip DP padding tokens in MoE (#46428) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-23 14:54:46 -07:00
Yongye ZhuClaude Opus 4.8Thien Tran
 		6691f087a6 [Minimax-M3] BF16/FP8 Indexer using MSA (#45892) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-06-23 10:28:49 -07:00
Umut PolatandFlora Feng 8dd1b702f2 [Misc] Fix stale doc URL and docstring module path (#35530) 
Signed-off-by: umut-polat <52835619+umut-polat@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-06-20 23:57:01 +00:00
Lucas WilkinsonOpenAI CodexTyler Michael Smith
 		cc22621b51 [KV Offload] Support packed HMA KV cache layout (#46205) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-06-20 21:19:40 +00:00
Francesco FuscoandClaude ced32bb474 [Perf] Add VLLM_TRITON_FORCE_FIRST_CONFIG to skip Triton autotuning (#42425) 
Signed-off-by: Francesco Fusco <ffu@zurich.ibm.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-16 15:16:45 +02:00
Jimmy Lee 8bf374955f [Bug Fix] Allow pinned memory for WSL2 (#41496) 
Signed-off-by: Jimmy Lee <hirejimmylee@gmail.com>
 2026-06-16 05:56:26 +00:00
Cyrus Leung 9096659edb [Cleanup] Remove dead env (#45777) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-06-15 22:56:23 -07:00
gitbisectorClaudeCyrus Leung
 		9d808e2309 [Core] Use fastsafetensors ParallelLoader for weight loading (#40183) 
Signed-off-by: Git Bisector <gitbisector@gmail.com>
Signed-off-by: gitbisector <gitbisector@gmail.com>
Signed-off-by: git bisector <gitbisector@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-06-15 22:32:05 -07:00
youkaichaoOpenAI CodexIsotr0pyThien TranBugen ZhaoJee Jee LiRoger WangfunctionstackxYongye ZhuJee Jee Li
 		0a1c5034f5 [Model] Add MiniMax M3 support (#45381) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-16 01:01:25 +08:00
Juan Pérez de AlgabaandHarry Mellor 2b3006076c [Security] Add timeout guard for regex compilation in structured outp… (#45118) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-13 09:52:56 +00:00
Ryan Rockandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> aab639c705 [Core][AMD] Propagate shutdown timeout to MultiprocExecutor (#43154) 
Signed-off-by: Ryan Rock <ryan.rock@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-06-12 15:13:31 -05:00
Chaunceycjackalmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		2043258dec [Frontend] Support strict mode for tool calling (#45003) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-12 07:51:48 +00:00
Ekagra Ranjanandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> e0b9fb1290 [ASR] Optimize CPU preproc to get 2.5x RTFx via multi-threading (#44612) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-11 21:05:11 -07:00
Harry Mellor 03878d1c22 Deprecations for v0.23 and v0.24 (#44992) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-11 14:35:38 +00:00
Harry Mellor 3cc9fecd58 Deprecated 1st generation Qwen and QwenVL models (#45131) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-10 14:55:33 +00:00
Juan Pérez de Algaba 1b1359c332 [Security] Fix DoS via audio decompression bomb in speech-to-text endpoint (#44970) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-06-10 00:18:53 +08:00
Tyler Michael SmithandClaude Opus 4.6 e2f993dc41 [WideEP] Integrate DeepEP v2 (#41183) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-06-08 18:07:29 -07:00
Walter Beller-Morales c73b0d0db9 [Core][Engine] allow DP ray placement groups to be set on specific nodes (#44669) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
 2026-06-05 20:07:47 +00:00
Han Lingemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>TJian
 		165b7864d0 [ROCM] [FEAT] Integrate Aiter hipBLASLt GEMM online tuning (#40426) 
Signed-off-by: hanlin12 <hanlin12@amd.com>
Signed-off-by: Han Lin <hanlin12@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-06-04 23:45:36 -07:00
Tushar JainandTushar Jain 38fd2405f3 use split_group for pytorch process group creation (#41980) 
Signed-off-by: Tushar Jain <tushar00jain@users.noreply.github.com>
Co-authored-by: Tushar Jain <tushar00jain@users.noreply.github.com>
 2026-06-04 14:36:07 -04:00
Wei ZhaoandYifan Qiao a6183563b6 [Prefix Caching] DeepSeekv4 - Support selective prefix-cache retention for sliding-window KV cache (#43447) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-06-04 00:48:31 -07:00
Daoyuan Li bd98e97557 [Misc] Remove dead VLLM_RPC_TIMEOUT env var and fix profiling doc that references it (#44128) 
Signed-off-by: Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com>
 2026-06-03 00:22:10 +00:00
Raj JoshiandCursor 9090368b65 [Feat] Add support for per GPU worker RDMA NIC selection (#42083) 
Signed-off-by: Raj Joshi <rajjoshi@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-28 12:45:23 -07:00
Nico Holmberg 7b54690244 [ROCm][Perf] Expose AITER MoE sorting dispatch policy via env var (#39177) 
Signed-off-by: nholmber <nholmber@users.noreply.github.com>
 2026-05-27 13:11:02 +08:00
Simon DanielssonandTJian 681d7dd38b [Misc][Refactor][ROCm] Convert MoRI-related envvars to extra config args (#43303) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-05-26 03:33:35 -07:00
Jee Jee Li ec5de7fa7d [LoRA] Add one shot triton kernel For MoE LoRA (#42290) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-05-25 19:47:04 -07:00
daniserebandBanani Ghosh d56285c747 Tuning script and configs for Triton Mamba SSU kernel (#43083) 
Signed-off-by: Banani Ghosh <bg2502@nyu.edu>
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
Co-authored-by: Banani Ghosh <bg2502@nyu.edu>
 2026-05-24 20:12:44 +03:00
Nick Hill 0f66623b0d [Frontend] Rework fastokens integration (#43168) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-05-21 15:36:58 -07:00
Michael Goin 9b54e50e2c [Deprecation] Mark env vars covered by --moe-backend / --linear-backend (#43148) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-05-21 12:51:12 -07:00