iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,363 Commits 3 Branches 0 Tags
7b7e5cbfcd99bc286ba2ae13e1f967b66a1a5c00
Commit Graph
7395 Commits
Author SHA1 Message Date
kiroxuandOpenAI Codex 07ef21bc69 [Kernel][Perf] Enable fused QK-norm + partial MRoPE + gate for Qwen3.6 (#52676) 
Signed-off-by: kiroxu <148877251+BabyDrangoner@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-24 18:28:49 -07:00
drakoshaandClaude 9bf0d4717e [Bugfix] Keep grid dims for XD-RoPE models on a prefix-cache hit (#53456) 
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-25 01:14:32 +00:00
Harry MellorXianbao QIANClaude Opus 5
 		7ca336929c [Model] Remove ten deprecated model architectures (#53608) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Xianbao QIAN <xianbao.qian@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-24 17:06:00 -07:00
rasmith a0f1b9ad05 [CI][The Rock] Increase flex attention abs tol (#53646) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-08-24 23:57:03 +00:00
Giancarlo Delfin d154d90d6c [Model Runner V2] batch-sharded sample (#50465) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-08-24 15:49:57 -07:00
Juan Pérez de Algaba 0d7d5ed0b2 fix(security): enforce VLLM_MAX_AUDIO_CLIP_FILESIZE_MB on all audio paths (#53561) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-08-24 21:37:02 +00:00
Khushali Desai 23ab0cfdbc speculative decoding under tensor parallelism (TP>1) , workspace creation select max hidden dim of target and draft model (#52193) 
Signed-off-by: khushali9 <khushali.desai9@gmail.com>
 2026-08-24 21:33:36 +00:00
Wentao Ye f59bb0bd8c [Refactor] Refactor batch invariance folder (#53619) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-24 14:31:36 -07:00
Guan-Ming Chiu 6a9c69fa85 [Attention][Spec Decode] Support varlen trtllm-gen decode for adaptive verification (#52157) 
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
 2026-08-24 14:43:57 -04:00
Prudhvi Vudaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 342b8ebd8b [Bugfix][Multimodal] Encode text in mixed CLIP/SigLIP pooling batches (#53165) 
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-24 10:45:42 -07:00
Jee Jee LiandCyrus Leung 4f686e182a [MISC] Cleanup deprecated parameters (#53559) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-24 10:19:04 -07:00
Lio Einaudiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 7797b6022c [Kernel][Perf] Per-architecture tuned configs for batch-invariant persistent matmul (~3x decode kernels on RTX 4090D/H20) (#53247) 
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-24 11:28:51 -04:00
Kevin H. LuuandCodex 4c56e62c85 [Bugfix][Kimi K3] Skip absent metadata during CUDA graph profiling (#53581) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
 2026-08-24 15:16:28 +00:00
Misha Goin f620499ee3 [Bugfix][Spec Decode] Reapply group geometry for FlashAttention metadata (#53336) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-24 08:08:43 -07:00
wang.yuqi 9cef631f30 [Frontend] Move run_batch.py out openai folder (#53500) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-24 15:04:10 +00:00
Cyrus Leung ecfa7bb373 [MM] Cache common token sequences (#53560) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-24 07:48:10 -07:00
石皮幼鸟andOpenAI Codex 4ca856b0b5 [Bugfix] Release worker RPC payload before next dequeue (#51979) 
Signed-off-by: 石皮幼鸟 <2960474346@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-24 07:27:36 -07:00
Taneem Ibrahim 79bb395eea [Pooling] Improve BGE-M3 sync pooling throughput by up to 3.13% (#53464) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-24 08:33:32 -04:00
andrewbcohere 41ae917c12 Add Cohere ChatV2 render endpoint (#53219) 2026-08-24 12:27:57 +00:00
Tan ChaoandClaude e239947777 [Bugfix][Frontend] Fix run_batch upload retrying on success and unawaited error body (#50588) 
Signed-off-by: Chao Tan <chaos.tc@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-24 11:33:44 +00:00
Nicolò Lucchesi 0ecc284790 [Bugfix][Kernel] Handle kernel block sizes in V2 DCP slot mapping (#51031) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-24 07:28:38 +00:00
Ace EldeibandCodex a047e2543d Fix MNNVL Lamport mailbox publication and cleanup (#53000) 
Signed-off-by: Ace Eldeib <aeldeib@coreweave.com>
Co-authored-by: Codex <codex@openai.com>
 2026-08-23 23:56:54 -07:00
Nick HillAnh Trananhtra3889
 		a4d70bef37 [Model Runner V2] Reserve CUDA graph memory (#53306) 
Signed-off-by: Anh Tran <anh.tran.3889@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Anh Tran <anh.tran.3889@gmail.com>
Co-authored-by: anhtra3889 <anhtra@nvidia.com>
 2026-08-23 23:11:19 -07:00
Summer Yang e6e1af4ca1 [Perf] Tune FlashInfer all-reduce selection on SM103 (#53318) 
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
 2026-08-24 05:19:59 +00:00
ThunGuoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> e8888b2d68 [Model] Add FP8 quantization support for ModernBERT (#53101) 
Signed-off-by: thunguo <tew@apache.org>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-24 03:04:19 +00:00
Clinton ThomasandLucas Bourtoule 2ec6f0d71e Reject oversized media before fully downloading it (#51896) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
 2026-08-24 01:15:09 +00:00
Dieter Schwarzmann 585bb07c70 feat: add SSE keep-alive comments for idle streaming responses (#51034) 
Signed-off-by: fuzzifikation <40043768+fuzzifikation@users.noreply.github.com>
 2026-08-23 18:13:02 -07:00
FrankandCodex cd329413e2 [Misc] Use VLLMValidationError in Cohere request validation (#52467) 
Signed-off-by: frank-suwen <suwenw2@outlook.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-08-24 00:52:27 +00:00
Aaron Hao 6cd9713463 [RL] P2P RDT weight sync (#43375) 2026-08-23 09:42:22 -07:00
Jensen ChenandClaude Opus 5 39eba6ac36 [Misc] Use VLLMValidationError in offline inference input validation (#53432) 
Signed-off-by: Jensen Chen <a1043904820@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-23 08:43:51 -07:00
Wyett b26039b09f Add routed expert loading for gpt-oss (#52209) 
Signed-off-by: Wyett <wyettzeng@gmail.com>
Signed-off-by: wyett <wyettzeng@gmail.com>
 2026-08-23 17:59:08 +08:00
Cyrus LeungandQwen Code ebcd606467 [MM] Simplify prompt updates: replace PromptSeq with list[int] (#53372) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Qwen Code <noreply@alibaba-inc.com>
 2026-08-23 14:50:32 +08:00
anmolgupt a3561ef8e4 [Bugfix][Model] Mistral3: fix image placeholder grid for processor size overrides (#52874) 
Signed-off-by: Anmol Gupta <14880251+anmolgupt@users.noreply.github.com>
 2026-08-22 21:34:38 -07:00
Andreas Karatzas 30b34171b1 [ROCm][CI] Restore attention coverage after KV-cache layout refactor (#53351) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-22 20:00:25 -05:00
Brian WoodsClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		bbe8b23e1a [Deprecation] Remove dead use_prefill_decode_attention flag (#52557) 
Signed-off-by: Brian Woods <me@brian.biz>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-22 22:38:10 +00:00
Zhou248 2f55ef254c [Model] Add Qwen3-Omni DSpark support (#52560) 
Signed-off-by: Zhou248 <630563665@qq.com>
 2026-08-22 15:35:22 -07:00
Shuolei Wangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 236f78cc5c [Core][RL] Support sparse checkpoint updates through native weight loaders (#50723) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-22 17:39:58 +00:00
Wentao YeZiming HuangOpenAI Codex
 		9eb9d9d395 [Perf] Support internal prefill checkpoints for Mamba prefix caching, 9%~25% TTFT improvement (#52789) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-22 10:32:40 -07:00
Cyrus Leung 0b19ebcacd [MM] Simplify _apply_hf_processor_main (#53275) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-08-22 01:49:37 -07:00
Tianyu Guo 7ca49fbe4b [Refactor][Model Runner V2][Multimodal] Move the encoder-only path out of the shared runner (#53176) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-08-22 05:11:44 +00:00
Yongye ZhuandOpenAI Codex e9d1398d9e [Bugfix][Kimi K3] Enable deferred MoE finalization before weight loading (#53327) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-22 11:23:32 +08:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 b2db227a7c [Bugfix][R3] Unwrap UniformTypeKVCacheSpecs when selecting the routed-experts KV group (#53240) 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
 2026-08-21 19:13:53 -07:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00
Misha Goin 9ff7041b55 [Bugfix][Spec Decode] Use group geometry for FlashAttention metadata (#53002) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-21 17:07:19 -07:00
Misha Goin a34f2ab70a [Test] Add focused hybrid MTP prefix-cache regressions (#53189) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-21 17:05:39 -07:00
Maroon Ayoub a556f3fccb Forward Anthropic vllm_xargs to sampling params (#53308) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
 2026-08-21 15:23:21 -07:00
andrewbcohere f94dcde5c5 Fix Cohere ChatV2 citation and tool handling issues (#52175) 
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
 2026-08-21 15:23:17 -07:00
stefankoncarevic f37d5868b9 [CI/Build][ROCm] Run the TileLang HIP symbol checks in their own interpreter (#53117) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-08-21 16:17:42 -05:00
Kevin H. LuuandAndreas Karatzas 592e06f2ae Revert "[ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill" (#53294) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-21 13:34:13 -07:00
Lucas WilkinsonandOpenAI Codex 0a3a4f7f35 [Bugfix][KV Connector][NIXL] Support PCP producers (#52779) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-21 16:22:01 -04:00