iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
19,868 Commits 3 Branches 0 Tags
4eef91c03cad3f080e46db1eeeadf121c9e6c136
Commit Graph
2575 Commits
Author SHA1 Message Date
Aaron HaoandClaude Opus 5 793ca6998a [Docs][RL] Rewrite weight-transfer docs; standardize examples (#51729) 
Signed-off-by: haoaaron <ahao@anyscale.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-11 18:53:15 -07:00
鐘天楽 a367cdbca0 [Profiler] Add minimal Triton Proton profiling backend (#48789) 
Signed-off-by: Luossu <zhluosuu@outlook.com>
 2026-08-11 14:51:57 -07:00
Louie Tsai 5b184f775a connects vLLM Recipes with vLLM's native config-based deployment and benchmark (#51308) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-08-11 13:53:51 +00:00
Kyungmin LeeandHarry Mellor 529d010351 [Doc] Fix typos in speculative decoding docs (#51500) 
Signed-off-by: lkm2835 <lkm2835@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-11 09:17:05 +01:00
Yizhou a311916a29 [MRV2][Spec] Fuse AR speculator multi-step decodes back into one CUDA graph (#46849) 
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
 2026-08-11 15:06:22 +08:00
Khushali Desaiandwang.yuqi dc5101fb1b replace batch_norm to numerically identical without cudnn (#51734) 
Signed-off-by: khushali9 <khushali.desai9@gmail.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-11 12:43:20 +08:00
Michael GoinOpenAI CodexOpenAI Codex
 		21c667aa64 [Docker] Cache test dependencies before vLLM install (#51184) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-10 10:34:59 -07:00
Juan Pérez de AlgabaandCursor 640a09086d Fix DoS via sample-rate forgery bypassing audio decode duration guard (#49948) 
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-10 15:49:17 +00:00
Emmanuel Acheampong 3a79957b62 [Doc] Add Crusoe Managed Inference deployment guide (#49353) 
Signed-off-by: Emmanuel Acheampong <achampion.emma@gmail.com>
 2026-08-10 11:31:34 +01:00
Rehan KhanandLi, Jiang <jiang1.li@intel.com> b22afe45ac [CPU] Enable GPTQ and AWQ quantization for s390x (#51148) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-08-10 11:41:43 +08:00
ChaunceyandOr Ozeri c423998642 [KV Offload] Emit self-describing events for partial recurrent blocks (#51243) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-08-09 11:28:44 +03:00
Isotr0py 073c510c91 [Migration] Migrate bitsandbytes support to OOT plugin (#43529) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-09 10:27:31 +08:00
Tyler Michael SmithOpenAI CodexMichael Goin
 		cb1a52aee3 [Build] Upgrade runtime image to Ubuntu 24.04, pick up rdma-core > 44 (#51058) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-08-08 18:17:50 -07:00
fxmarty-amd 3518110f2f [Online quantization] Add online MXFP4 quantization support (#49347) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-08-07 12:39:46 -07:00
Kunshang Ji 0406ba22c4 fix pre-commit broken (#51341) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-06 18:17:39 -07:00
Simon Mo 9bca7d840d docs(governance): refresh committers list, add TSC note, update project leads (#51300) 2026-08-06 16:28:17 -07:00
Lyu Han 22013f74ff Interns2mobius support (#51149) 
Signed-off-by: Lyu Han <lvhan_028@163.com>
 2026-08-06 10:39:35 +00:00
wang.yuqiandCyrus Leung 2fa490470d [Model] Fused mm preprocess normalisation on the Device (#50411) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-08-06 03:11:14 -07:00
Qiuyang Yue 777b01d1a8 [MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference (#45254) 
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com>
 2026-08-06 05:04:19 +00:00
Chauncey 22170354d8 [Feature] Parse request priority from HTTP header (#51089) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-08-05 21:10:29 -07:00
baodi 2e09247c2d [Docs] List Intel XPU attention backends (#51215) 
Signed-off-by: baodii <di.bao@intel.com>
 2026-08-06 10:58:28 +08:00
Silen NaihinDakai AnJiangyun Zhu
 		b50fdebce0 [Bugfix] Fix level-2 sleep/wake/reload with enable_lora=True (#39935) 
Signed-off-by: Silen Naihin <silen.naihin@gmail.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-08-06 09:13:10 +08:00
LZWandRoger Wang d36f24b66a [KV Connector][Mooncake] Add tenant ID support to MooncakeStoreConnector (#48069) 
Signed-off-by: lzw <2994587066@qq.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-08-05 17:31:18 -07:00
Zhewen LiZhewen LiClaude Opus 5
 		14e57ad47d [Docker][KVConnector] Install mooncake from official wheels instead of a custom build (#51067) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-05 11:58:44 -07:00
Michael GoinandShengqi Chen e8586b2878 [Build] Remove Ubuntu build-stage option from the CUDA dockerfile (#51060) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-08-05 10:50:58 -07:00
Ronen SchafferandOr Ozeri 8543522ca7 [KV Offload] Support out-of-tree secondary tier managers via module_path (#51007) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-08-05 20:30:53 +03:00
zexplorerhj d4da0c55af [Model][Frontend] Add Ling 3.0 Flash BF16, MTP, and parser support (#51045) 
Signed-off-by: zexplorerhj <zhjoneson@163.com>
 2026-08-06 00:38:28 +08:00
Rehan KhanandLi, Jiang <jiang1.li@intel.com> 0187f4c88e [CPU] Enable tcmalloc for s390x (#50841) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-08-05 10:55:39 +08:00
Harry MellorandClaude Opus 5 1eb3694521 [Docs] Fix two docs build warnings (#51014) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-04 05:29:04 -07:00
Tzu-Ling KanGPT-5.6 SolCursor Grok 4.5Claude Opus 4.8
 		952694e384 [Bugfix] Validate NIXL speculative config compatibility (#49230) 
Signed-off-by: Tzu-Ling <tzulingk@nvidia.com>
Co-authored-by: GPT-5.6 Sol <noreply@openai.com>
Co-authored-by: Cursor Grok 4.5 <noreply@cursor.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-04 02:51:04 +08:00
wangxiyuan dd11df04f3 [Misc] Remove deprecated calculate_kv_scales runtime KV scale calculation (#49389) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-08-03 11:15:13 -04:00
Nicolò Lucchesi 8ba87e0181 [CI] Mooncake PD integration tests (#46844) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-03 06:55:51 -07:00
Fede KamelharandHarry Mellor 005fa01756 docs: document reasoning_content output removal as a breaking client change (#50624) 
Signed-off-by: fede-kamel <fkamelhar@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-03 13:45:23 +00:00
Kyungmin Leeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 9ae11a6b89 [Model] Add K-EXAONE-2.0-750B-A37B (#50524) 
Signed-off-by: lkm2835 <lkm2835@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-03 06:33:02 -07:00
Li, Jiang c8602c7906 [CPU] Refine CPU kernel dispatch (#50801) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-08-03 17:42:53 +08:00
Li, JiangClaude Opus 5Fadi Arafeh
 		0a6446005d [CPU] Migrate unquantized MoE to the modular-kernel experts structure (#50133) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-08-03 14:55:04 +08:00
zofiaandKunshang Ji e481da9508 [XPU] [Linear] add torch as xpu linear backend (#49664) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-03 14:30:21 +08:00
Omkar Shewale 9a4fd57cac [Model] Support jina-embeddings-v5-text-nano (EuroBERT encoder backbone) (#50688) 
Signed-off-by: Omkar Shewale <omkarshewale2001@gmail.com>
 2026-08-03 01:37:08 +00:00
Isotr0pyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> e2fa28594f [1/N] Unify multiple-path encoder cuda graph support (#49934) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-02 10:33:23 +08:00
bnellnm c67fe497a2 [Bugfix][Doc] Fix references to FusedMoE in doc (#50701) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-08-02 09:13:06 +08:00
Guan-Ming Chiu 63e78ce365 [Benchmark] Add probe requests to vllm bench serve (#49611) 
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
 2026-08-01 22:10:56 +08:00
Michael Goin 4ee9702bee Add Understanding the Latency Metrics docs (#50600) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-31 23:22:40 -07:00
LG c036cb257f [Doc] Add BgeM3EmbeddingModel to embedding supported models (#50571) 2026-07-31 15:56:01 +00:00
Nicolò Lucchesi 17beffd55f [Misc] Clarify mono audio requirement (#50141) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-07-31 10:46:53 +00:00
Rehan KhanandLi, Jiang <jiang1.li@intel.com> 88bc8fb56f [CPU][s390x] Optimize inference perf and add oneDNN INT8 GEMM for s390x (#50219) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-07-31 17:49:31 +08:00
Isotr0py 1180b604f8 [Multimodal] Expose mm hash algothrim selection to cli args (#49686) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-07-31 05:25:19 +00:00
Kevin H. LuuandMichael Goin 5f8f72866c [Build] Fix CUDA release wheel builds (#50243) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-07-30 11:48:09 -07:00
Juan Pérez de Algaba 89d97d9a16 docs(security): document Ray cluster trust model and env var propagation (#50397) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-07-30 11:52:55 +00:00
+13
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodexWei Zhao
 		aeeb36b1f1 [New model] Kimi K3 (#50000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-07-30 03:49:58 -07:00
Fadi Arafeh 8122a10536 [DOC][CPU] remove tcmalloc warning from CPU docs (#50308) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-07-30 06:30:21 +00:00