iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
20,617 Commits 3 Branches 0 Tags
76ff0cdff2defded8d912d550299c902ff33cdca
Commit Graph
20617 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
andyluo7andOpenAI Codex 76ff0cdff2 [Bugfix][ROCm] Preserve AITER unified-attention metadata during graph replay (#53821) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-31 21:53:12 +08:00
Itay EtelisItay EtelisOr Ozeri
 		c5d840ff6a [KV Connector][Offloading] Certify attention-only hybrids in the canonical portability gate (#51689) 
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-08-31 15:53:19 +03:00
Itay EtelisItay EtelisNickLucche
 		4c58a0c398 [Bugfix][KV Offload] Unlink /dev/shm region after all workers map it (barrier variant of #51317) (#52596) 
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-31 15:53:00 +03:00
Uttam KumarandOr Ozeri e0d27040dd [Bugfix][KV Offload][P2P] Preserve aborted loads until abort completion (#52571) 
Signed-off-by: Uttam Kumar <uttam.kumar912@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-08-31 14:34:48 +03:00
Jee Jee Li e2c8eeac40 [kernel] Fused embedding kernel (#53677) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-08-31 11:34:14 +00:00
Itay EtelisItay Etelismergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		bed3280f50 [KV offload] Order CPU->GPU loads against the compute stream (#50696) 
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-31 14:31:18 +03:00
waizuichougou 810bc3250c [Frontend][Performance] Resolve async media across modalities concurrently (#54537) 
Signed-off-by: waizuichougou <2082431897@qq.com>
 2026-08-31 10:50:17 +00:00
maithilijoshi20 9debcd5990 [Test][Qwen3-VL] Cover compiled DeepStack input contract (#53529) 
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com>
 2026-08-31 10:46:50 +00:00
Moein Khazraee 9acbc5360a [KV Offload] Preserve KV event metadata until final residency removal (#52068) 
Signed-off-by: Moein Khazraee <moein@nvidia.com>
 2026-08-31 13:28:30 +03:00
wang.yuqiandCyrus Leung eeb549a74d [Frontend] Move engine/protocol.py out openai folder (#54492) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-08-31 10:13:50 +00:00
liranschour d8de4ae322 [Bugfix][KVOffload] P2P tier declares REQUEST_LEVEL on the producer leg (#52912) 
Signed-off-by: Liran Schour <lirans@il.ibm.com>
 2026-08-31 10:07:20 +00:00
Gabriel WuWentao Yemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		699e180df4 [Bugfix][SM120] DSv4: pass contiguous C128A decode topk indices on SM120 (#53574) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-31 09:44:42 +00:00
Hotragn Pettugani 28bf75c9a9 [Bugfix][Frontend] Truncate prompt_is_token_ids with the prompt (#54509) 
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in>
 2026-08-31 09:29:01 +00:00
Andreas Karatzas 399247cc88 [Bugfix][MM] Fix MiniCPM-o image processor reuse on Transformers v5 (#54501) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-31 17:07:52 +08:00
wang.yuqi 5bfd76372d [Renderer] Shutdown the renderer properly. (#52124) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-31 08:40:37 +00:00
jl9876andKunshang Ji 1b9539d37c [Quantization][Autoround][XPU] Support AutoRound MXFP8 MoE models (#51248) 
Signed-off-by: jl9876 <jia.li@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-31 16:37:01 +08:00
Tianmu LiandOpenAI Codex c6c33f2b1f [CPU] Support FP16/BF16 persisted GDN state on AMX (#52191) 
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-31 16:32:20 +08:00
zofiaandopencode 2a61f060d3 [XPU] Ensure unquantized linear weight is N-contiguous (#53536) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: opencode <noreply@opencode.ai>
 2026-08-31 16:01:24 +08:00
Li, Jiang fdbf2ddbd2 [Bugfix][CPU] Fix several bugs (#54042) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-08-31 15:33:06 +08:00
Yunxiao NingMikhail KostryukovLucas Wilkinson
 		2cf82bcdd1 [Bugfix][DCP] Fix NVIDIA DeepSeek-V3.2 / GLM-5.2 fused attention (#50005) 
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-08-31 00:30:45 -07:00
Moein KhazraeeNicolò LucchesiCyrus Leung
 		f9d666f917 [KV Offload] Forward ownership in KV cache events (#52067) 
Signed-off-by: Moein Khazraee <moein@nvidia.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-31 10:28:41 +03:00
Jee Jee Li 8e92248f79 [Kernel] Retire the DSv3 router GEMM CUDA kernel (#54040) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-08-31 07:15:31 +00:00
范裕达Jee Jee LiJee Jee Li
 		da0b2d8b17 [Performance] Optimize Dots3 NOTE runtime (#53517) 
Signed-off-by: KurodaKanbei <mistergalahad@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-31 00:05:18 -07:00
Prudhvi Vuda 7292ee2791 [Bugfix][Multimodal] Honor modality-scoped mm_processor_kwargs (#53808) 
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
 2026-08-31 06:37:12 +00:00
eligottsandOpenAI Codex 4ae172231c [Frontend] Forward cache salt for content parts (#54315) 
Signed-off-by: Eli Gotts <78387377+eligotts@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-30 23:29:53 -07:00
Hotragn Pettugani 8fd9eb85d5 [Bugfix][Frontend] Truncate prompt_token_offsets with the prompt (#54407) 
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in>
 2026-08-31 06:13:37 +00:00
+5
Harry Huangzjy0516KungYorkThien TranYifan Qiaoandyluo7Isotr0pyandyluo7OpenAI Codexlishunyang12Vadim GimpelsonIsotr0pyLucas WilkinsonClaude Opus 5OpenAI Codex
 		e126687a9a [Model] Support Qwen3.8-Flash-Next (#53896) 
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: KungYork <firelikest@163.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-31 05:57:56 +00:00
Hotragn Pettugani 687db59744 [Bugfix][Frontend] Truncate pooling prompts before padding them (#54364) 
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in>
 2026-08-30 22:12:03 -07:00
LironKesem 44fe2a392b [CPU] add CPU support for Voxtral (#53921) 
Signed-off-by: Liron Kesem <52330564+LironKesem@users.noreply.github.com>
 2026-08-31 04:20:37 +00:00
Jee Jee Li 648b7468b8 [CI/Build] Fix Kimi K3 Eagle3 test fixture (#54482) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-08-31 03:29:45 +00:00
Isotr0py 555ea65e8c [Frontend] Add video embeds input support (#54242) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-31 03:20:47 +00:00
Lucas Wilkinson 5707355209 [Bugfix][MLA] Fix BLHNC addressing for FlashInfer sparse MLA (#54465) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-31 01:13:42 +00:00
AI-Infra Rookie c92b29a1d4 [Bugfix] Make metadata send non-blocking in GroupCoordinator.isend_tensor_dict (#49274) 
Signed-off-by: z00897177 <zhouxinyi6@huawei.com>
 2026-08-30 23:51:25 +00:00
Wei ZhaoJee Jee LiYongye ZhuJiangyun Zhu
 		7ab2923489 [Flashinfer] Upgrade Flashinfer version to 0.6.18 (#54313) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-08-30 16:01:05 -07:00
Kevin H. Luu 7a100bb617 [CI] Restore gpu_1_queue routing for torch-abi audit (#54468) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-08-30 14:59:36 -07:00
Kevin H. Luu b2dc864bb6 [Bugfix][Spec Decode] Keep default CUDA graph sizes memory-safe (#54418) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-08-30 18:02:31 +00:00
Aditya JhaandClaude 9d0fe9bac8 [Bugfix][Quantization][MoE] Route weight only NVFP4 checkpoints through W4A16 (#54427) 
Signed-off-by: Aditya Jha <4adityajha@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-30 13:02:15 -04:00
Juan Pérez de Algaba e79961c946 [codeowners] Add jperezdealgaba to security file ownership (#54358) 
Adds Juan Pérez de Algaba (`@jperezdealgaba`) alongside Russell Bryant as a CODEOWNER for the vLLM security guide, security policy, and vulnerability-management documentation.
 2026-08-30 16:24:21 +00:00
waizuichougouandOpenAI Codex f6895a5fcb [Bugfix][Multimodal] Avoid caching full prompts in fallback (#54439) 
Signed-off-by: waizuichougou <2082431897@qq.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-30 16:19:38 +00:00
Yongji WuCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Jiangyun Zhu
 		56058fd572 [Bugfix][Kernel] Keep packed GDN decode beta in FP32 (#53877) 
Signed-off-by: CherryLemon <lemon_cn@alumni.sjtu.edu.cn>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-08-30 22:15:36 +08:00
macheroandClaude 87b9b5b8d9 [Test][VLM] Add batch-invariance tests for Qwen3-VL (#53531) 
Signed-off-by: machero <sr1123640465@126.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-30 09:49:41 -04:00
Taneem Ibrahim fe6db3ed5e [Bugfix] Validate scale-out transfer params (#54324) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-30 09:46:12 -04:00
Harry MellorandClaude Opus 5 488b6da105 [Doc] Fix griffe warnings in HYV4 tool parser (#54412) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-30 13:21:22 +00:00
Kevin H. Luu 5e71a11eb2 [CI] Mark L4 GPU test steps with device: l4 for EKS migration (#54326) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-08-30 04:40:41 -07:00
Kevin H. Luu 78fa18910e ci: add MIG slice size to H200 job labels (#54420) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-08-30 04:35:01 -07:00
Jiangyun ZhuandOpenAI Codex 8c51b92654 [Bugfix] Avoid global config lookup in sparse indexer forward (#54400) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-30 03:31:47 -07:00
Simon Danielssonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Shanshan Shen
 		2c7d7dd64a [Perf][ROCm] Dual-stream decode with hipgraphs (#52033) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-08-30 08:24:20 +00:00
Devon KrismanandClaude Fable 5 4f78a8fdd0 [Model] Honor cap_pixels_per_frame in Qwen3-VL memory profiling (#54380) 
Signed-off-by: Devon Krisman <git@krisman.dev>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-30 08:07:29 +00:00
Canlin Guoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> b383e16396 [Bugfix] Reset cached Mamba align metadata on profiling teardown (#54044) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-30 00:43:20 -07:00
Kevin H. LuuandKimi Code 8fa4c6cdb3 [CI] Add explicit step keys to 18 hardware test steps (#54330) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-08-30 00:08:25 -07:00