iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,583 Commits 3 Branches 0 Tags
7a100bb617471801ee1d5525bfbb8fb238a345ea
Commit Graph
20583 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Kevin H. Luu 7a100bb617 [CI] Restore gpu_1_queue routing for torch-abi audit (#54468) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-08-30 14:59:36 -07:00
Kevin H. Luu b2dc864bb6 [Bugfix][Spec Decode] Keep default CUDA graph sizes memory-safe (#54418) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-08-30 18:02:31 +00:00
Aditya JhaandClaude 9d0fe9bac8 [Bugfix][Quantization][MoE] Route weight only NVFP4 checkpoints through W4A16 (#54427) 
Signed-off-by: Aditya Jha <4adityajha@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-30 13:02:15 -04:00
Juan Pérez de Algaba e79961c946 [codeowners] Add jperezdealgaba to security file ownership (#54358) 
Adds Juan Pérez de Algaba (`@jperezdealgaba`) alongside Russell Bryant as a CODEOWNER for the vLLM security guide, security policy, and vulnerability-management documentation.
 2026-08-30 16:24:21 +00:00
waizuichougouandOpenAI Codex f6895a5fcb [Bugfix][Multimodal] Avoid caching full prompts in fallback (#54439) 
Signed-off-by: waizuichougou <2082431897@qq.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-30 16:19:38 +00:00
Yongji WuCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Jiangyun Zhu
 		56058fd572 [Bugfix][Kernel] Keep packed GDN decode beta in FP32 (#53877) 
Signed-off-by: CherryLemon <lemon_cn@alumni.sjtu.edu.cn>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-08-30 22:15:36 +08:00
macheroandClaude 87b9b5b8d9 [Test][VLM] Add batch-invariance tests for Qwen3-VL (#53531) 
Signed-off-by: machero <sr1123640465@126.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-30 09:49:41 -04:00
Taneem Ibrahim fe6db3ed5e [Bugfix] Validate scale-out transfer params (#54324) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-30 09:46:12 -04:00
Harry MellorandClaude Opus 5 488b6da105 [Doc] Fix griffe warnings in HYV4 tool parser (#54412) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-30 13:21:22 +00:00
Kevin H. Luu 5e71a11eb2 [CI] Mark L4 GPU test steps with device: l4 for EKS migration (#54326) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-08-30 04:40:41 -07:00
Kevin H. Luu 78fa18910e ci: add MIG slice size to H200 job labels (#54420) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-08-30 04:35:01 -07:00
Jiangyun ZhuandOpenAI Codex 8c51b92654 [Bugfix] Avoid global config lookup in sparse indexer forward (#54400) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-30 03:31:47 -07:00
Simon Danielssonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Shanshan Shen
 		2c7d7dd64a [Perf][ROCm] Dual-stream decode with hipgraphs (#52033) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-08-30 08:24:20 +00:00
Devon KrismanandClaude Fable 5 4f78a8fdd0 [Model] Honor cap_pixels_per_frame in Qwen3-VL memory profiling (#54380) 
Signed-off-by: Devon Krisman <git@krisman.dev>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-30 08:07:29 +00:00
Canlin Guoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> b383e16396 [Bugfix] Reset cached Mamba align metadata on profiling teardown (#54044) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-30 00:43:20 -07:00
Kevin H. LuuandKimi Code 8fa4c6cdb3 [CI] Add explicit step keys to 18 hardware test steps (#54330) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-08-30 00:08:25 -07:00
Juan Pérez de Algaba 1dc464d426 [Bugfix] Bound cache_salt length to prevent DoS via scheduler CPU exhaustion (#54353) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-08-30 04:28:16 +00:00
Yizhou 1ebff996a1 [Fix] Improve ROCm detection in WSL environments (#38434) 
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
 2026-08-30 04:23:10 +00:00
Connor Carpenter 7a67941c1d [Rust Frontend][gRPC] Add audio and video media inputs (#53760) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
 2026-08-30 11:31:37 +08:00
waizuichougouOpenAI CodexOpenAI Codex
 		b016ed8ea3 [Bugfix][Multimodal] Release Qwen2.5-VL and Qwen3-VL RoPE caches with the model (#54346) 
Signed-off-by: waizuichougou <2082431897@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-30 11:10:29 +08:00
Matt Qinyudigege86CursorAndreas Karatzas
 		dbf662c9e8 [ROCm][MLA] Reach FULL cudagraphs for AITER MLA speculative decoding (#51171) 
Signed-off-by: yudigege86 <naqin@amd.com>
Co-authored-by: yudigege86 <naqin@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-29 19:49:55 -07:00
hcl b5707bf994 [Bugfix][MoE] Enable cuBLAS out_dtype router GEMM on all CUDA archs (fixes family-120/GB10) (#54048) 
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com>
 2026-08-29 19:18:42 -07:00
yzong-rh 680e2177e4 [CI][Ray] Fix flaky multi-node assignment test after placement-group teardown (#53621) 2026-08-29 23:48:22 +00:00
Oxana KorzhandClaude Opus 5 6cddad414e [CI][Test] Deflake test_mem.py sleep-mode asserts via allocator bookeeping (#54312) 
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-29 12:05:17 -07:00
Giancarlo Delfin fe755c8899 [Bugfix][Model Runner V2][Spec Decode] Decouple the draft's gumbel noise stream from the target's (#54282) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-08-29 09:58:29 -07:00
Kaif KohariandCyrus Leung 7fbfca2670 [Bugfix][Frontend] Only echo the assistant turn in batched chat completions (#52529) 
Signed-off-by: Kaif <kaifkohari10@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-29 09:44:17 -07:00
jackandQwertyJack 5b0e5b69ac [Bugfix][Frontend] Validate stop_token_ids against vocab size (#54196) 
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
 2026-08-29 16:25:35 +00:00
Shenglei FuandClaude Fable 5 129087ddab [Perf] Reuse topk SparseMatrix routing metadata in GPT-OSS MoE forward (#45457) 
Signed-off-by: Shenglei Fu <sfu@confluent.io>
Signed-off-by: Shenglei Fu <117230642+ShengleiFu@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-29 12:14:39 -04:00
Nicolò Lucchesi 7f4793eaa3 [Nixl][PD] DCP support for MLA models (#50611) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-29 16:03:53 +00:00
Isotr0py 4fc943b867 [Multimodal] Deprecate PyAV video decoder backend (#54231) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-29 15:13:28 +00:00
Taneem Ibrahim fd5d3aea94 [Mypy] Fix typing for J models (#54130) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-29 08:50:15 -04:00
Andreas KaratzasandOpenAI Codex cacc429f62 [ROCm][CI] Stage E gating (#50920) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-28 23:52:23 -07:00
Rahul ChalamalaJanelle CaiDevin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>Devin
 		d3d79ffc1e [Bugfix][Spec Decode] Capture the widest uniform decode batch by default (#50488) 
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Rahul Chalamala <rahul@modal.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin <devin@cognition.ai>
 2026-08-28 22:38:13 -07:00
Keyi LiKeyi LiClaude Opus 4.7
 		a758a9f671 [Perf][KV Connector] Pin token_indices before non_blocking H2D in hf3fs helper (#54293) 
Signed-off-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-08-28 22:20:17 -07:00
aoshen02aoshen02Claude Fable 5
 		f4f3bcd4b4 [Test] Assert co-located RayExecutorV2 stores publish distinct ports (#54310) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-28 22:13:31 -07:00
Oxana KorzhandClaude Opus 5 fb68025138 [Bugfix][MRV2] Release model and KV cache on in-process engine shutdown (#54162) 
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-28 22:09:08 -07:00
Hotragn Pettugani 3958a420f0 [Bugfix][V1] Keep an encoder cache entry until its last occurrence is freed (#54284) 
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in>
 2026-08-29 05:08:01 +00:00
Summer Yang 6d4562c59b [Attention][DCP] Enable FlashInfer MLA for DSpark drafting (#54277) 
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
 2026-08-28 21:11:28 -07:00
Cheng JiangchengvjiangJee Jee LiBugen ZhaorussellfengJee Jee Li
 		b2f685834a [Hy4] support Hy4-preview model (#54160) 
Signed-off-by: chengvjiang <chengvjiang@tencent.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: russellfeng <russellfeng@tencent.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-29 12:07:28 +08:00
Nick Hill 6c18a54648 [Perf] Avoid h2d copies from non-pinned CPU tensors (#54299) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-29 12:06:30 +08:00
Stellar鱼 5182f2705b [CI/Build] Use file rendezvous for UniProc loader fixtures (#52367) 
Signed-off-by: yu-xin-c <2182712990@qq.com>
 2026-08-29 11:53:04 +08:00
kaiyuanxieandCalvinXKY 738bc8811a [BugFix] Disable TP for Qwen3-Omni audio encoder when heads % TP != 0 (#50858) 
Signed-off-by: CalvinXKY <xky@users.noreply.github.com>
Co-authored-by: CalvinXKY <xky@users.noreply.github.com>
 2026-08-29 11:18:04 +08:00
Keyi LiKeyi LiClaude Opus 4.7
 		46a83642f6 [Perf] Pin CPU tensors before non_blocking H2D in three MM paths (#54292) 
Signed-off-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-08-28 20:13:40 -07:00
Oxana Korzhokorzh-amdClaude Opus 5Douglas Lehr
 		93ab92be0c [Bugfix][AMD] Annotate draft KV cache groups on the hybrid grouping path (#52047) 
Signed-off-by: okorzh-amd <okorzh-amd@users.noreply.github.com>
Co-authored-by: okorzh-amd <okorzh-amd@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-08-28 21:53:46 -05:00
Yifan QiaoClaude Fable 5OpenAI CodexDao Le
 		6b110badbb [Bugfix][Mooncake] Save exact Mamba boundary states (#51358) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Dao Le <daole@inferact.ai>
 2026-08-29 02:40:53 +00:00
Ronaldandaoshen02 16d6c376bc [Bugfix][Models] Register sleep-managed runtime buffers (#53507) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-08-29 10:32:24 +08:00
Wei ZhaoandYifan Qiao 7fd9cc036e [KV Connector] Support MooncakeStore with hybrid DCP prefix caching (#53324) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: wzhao18 <weizha@nvidia.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-08-29 02:10:23 +00:00
Keyi LiKeyi LiClaude Opus 4.7
 		43196f2458 [Perf][MLA Sparse] Pin req_id_per_token before non_blocking H2D on XPU and ROCm (#54295) 
Signed-off-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-08-29 02:00:46 +00:00
RoachZhaoClaude Opus 5Bugen ZhaoOpenAI Codex
 		4c6c9d569f [Rust Frontend] Take the raw buffer in mm tensor lowering when possible (#53528) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: roachsinai <roachsinai@qq.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-29 01:05:14 +00:00
JulianCodexSummer Yang
 		99013d77d3 [Bugfix][Distributed] Gate cross-node MNNVL custom all-reduce by group capability (#53253) 
Signed-off-by: JulianZJN <148066842+JulianZJN@users.noreply.github.com>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
 2026-08-29 08:49:24 +08:00