iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
2339 Commits
Author SHA1 Message Date
Michael Goin e66d787bce Disable FlashInfer sampler by default (#26859) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-10-15 02:35:18 +00:00
Jialin Ouyang 07ca70af8d [Core][Easy] Use envs.__getattr__ for all Unify to environment variable access (#26810) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-10-15 01:41:18 +00:00
Tyler Michael Smith 579d2e5458 [WideEP][P/D] Add usage stats for DP+EP and KV Connector (#26836) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-10-14 23:51:54 +00:00
Nick Hill 4aed506b65 [Core] Streamline some structured output related code (#26737) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-14 23:27:44 +00:00
Boyuan Fengandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> a86b4c58e8 remove attn output view kernel (#26680) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Boyuan Feng <fby.1994@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-14 22:53:10 +00:00
Nick Hill ff4810ba73 [Minor] Group async_scheduling related fields in model runner init (#26736) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-14 14:46:37 -07:00
Jialin Ouyang 380f17527c [Perf] Cache vllm.env.__getattr__ result to avoid recomputation (#26146) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-10-14 17:03:21 -04:00
Jialin Ouyang acaa2c0a4a [Core] Reuse empty block lists whenever possible in KVCacheBlocks to mitigate GC costs (#24964) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-10-14 12:58:43 -07:00
Matthew Bonanni 82af928c41 [Attention][Spec Decode] FlashMLA spec decode support (#26541) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-14 19:38:20 +00:00
Wentao Ye 6d87a2838c [Config] Remove Unused Environment Variable VLLM_DISABLE_PAD_FOR_CUDAGRAPH (#26743) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-14 11:47:49 -04:00
Qier Li 720394de43 [KVConnector][Metrics] Aggregate scheduler-side KVConnectorStats (#26046) 
Signed-off-by: Qier Li <kevin44036@gmail.com>
 2025-10-14 14:38:07 +00:00
Jaya Yuan ea97940d6c [DCP] Support Decode Context Parallel (DCP) for GQA with FlashAttention (#24864) 
Signed-off-by: yuanyongjie.yyj <yuanyongjie.yyj@antgroup.com>
Signed-off-by: FENP <32334296+FENP@users.noreply.github.com>
Signed-off-by: Jaya Yuan <yuanyongjie.yyj@antgroup.com>
 2025-10-14 13:07:50 +00:00
XiongfeiWei 4497c8f821 Fix lora tests failure in TPU CI due to the removal of LoRA bias (#26723) 
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
 2025-10-14 13:04:23 +08:00
Michael Goin 3e051bda82 [UX] Replace VLLM_ALL2ALL_BACKEND with --all2all-backend (#26732) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-10-13 18:12:52 -07:00
Lucia Fang 8317f72354 [Misc][DP] support customized aggregated logger for dp (#24354) 
Signed-off-by: Lu Fang <fanglu@fb.com>
 2025-10-13 17:45:59 -07:00
Wentao Ye e251e457c5 [Log] Optimize Startup Log (#26601) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-14 02:06:57 +08:00
Will Eaton 53c9a7cee2 [P/D] [NixlConnector] kv load recovery integration (#26171) 
Signed-off-by: Will Eaton <weaton@redhat.com>
 2025-10-13 08:48:04 -07:00
ihb2032 4a61950f4d [Hardware][CPU] Disable torch.compile for RISC-V to prevent APIError (#26693) 
Signed-off-by: lyd1992 <liuyudong@iscas.ac.cn>
Signed-off-by: ihb2032 <1355790728@qq.com>
Signed-off-by: lyd1992 <liuyudong@iscas.ac.cn
 2025-10-13 07:56:01 -07:00
Bram Wasti 3263799056 [unrevert] Add batch invariant kernel override for FlashInfer backend [2/n] (#26373) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
Signed-off-by: Bram Wasti <bwasti@fb.com>
 2025-10-13 10:24:53 -04:00
yyzxwandwang.yuqi 46ad73955a [FIX] Throwing an exception when the model does not support pool tasks (#25840) (#25855) 
Signed-off-by: zxw <1020938856@qq.com>
Co-authored-by: wang.yuqi <noooop@126.com>
 2025-10-12 20:56:21 -07:00
quanliu 41f3884438 [Bugfix][Core]Fix block table out-of-range issue in priority scheduling (#26661) 
Signed-off-by: quanliu <18646313696@163.com>
 2025-10-13 01:25:42 +00:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
wang.yuqi 76852017ea [MISC] Rename the torch profiler filename as instance_id+rank_id for merging the Profiler results of each Rank (#25867) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-10-12 09:29:08 +00:00
dsinghvigemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>DarkLight1337wwl2755
 		727144bed1 [Refactor]: Use M-RoPE interface directly while defining model class instead of maintaining model specific M-RoPE implementation in mrope.py (#24172) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-10-11 07:21:04 +00:00
Roger Wangandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> ddaff2938e [MM] Move Qwen3Omni MRoPE impl to model file (#26608) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-10 22:17:24 -07:00
Nick Hill 5bc26c438d [BugFix] Make penalties and bad_words work with async scheduling (#26467) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-10 23:27:04 +00:00
Nick Hill 949cb0170d [BugFix] Fix async scheduling + request preemption (#26385) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-10 20:29:57 +00:00
Vadim Gimpelson e94cfd51da [BUG] Qwen3-next MTP. Fix attn metadata build bug (#26564) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-10-10 14:59:03 -04:00
Mark McLoughlin e519281920 [Metrics] Add test for multi-modal cache stats logging (#26588) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-10-10 16:00:50 +00:00
Sage Moore ae9d0e7da5 [Bugfix] Make DP padding optional in coordinate_batch_across_dp (#26375) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-10-10 10:53:33 -04:00
Mark McLoughlin 784c231151 [NIXL] Ignore abort on already-finished request (#25067) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-10-10 12:21:56 +02:00
Chen Zhang 606b00e80f [bugfix][DCP] fix block_size of hash in DCP prefix caching (#26296) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-10-10 03:02:49 -07:00
Cyrus Leung ad430a67ca [Metrics] Log multi-modal cache stats and fix reset (#26285) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-10 01:45:55 -07:00
Chen Zhang 6f0f570c43 [deepseek] kernel block size for UniformTypeKVCacheSpecs (#26559) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-10-10 16:40:41 +08:00
Lucas WilkinsonandBenjamin Chislett 29255cfc3b [Spec-Decode] Support piecewise cudagraphs for Eagle head (#25109) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2025-10-10 01:20:31 -04:00
Nick Hill aafb99a4d4 [Core] Small simplification in GPUModelRunner._update_states() (#26508) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-10 10:53:58 +08:00
Rui Qiao 757fa4a4da [DP][ray] Support different VLLM_RAY_DP_PACK_STRATEGY (#23849) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-10-09 19:53:43 -07:00
Julien Denize c6187f55f7 Refactor MistralTokenizer (#26358) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
 2025-10-09 22:48:58 +00:00
Benjamin Chislett 6e783bc54b [Bugfix] Fix CUDA graph selection bug in FlashInfer at high concurrency (#26499) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2025-10-09 17:12:34 -04:00
Nick Hill 2e54db4d2b [Core] Remove unused prev_sampled_token_ids_invalid_indices input batch field (#26514) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-09 20:22:14 +00:00
Ming Yang 3b736e1c38 [Attention][DCP] Support DCP with query length > 1 (MTP) with FA3 (#25049) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-10-09 08:06:29 -07:00
Cyrus Leung 4bdf7ac593 [Bugfix] Fix SHM cache initialization (#26427) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-09 02:48:04 -07:00
Cyrus Leung dc7976dd9f [Misc] Upgrade more code to Python 3.10 (#26463) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-09 10:43:53 +01:00
Nick Hill ddcbc2f334 [Misc] Misc code simplifications (#26450) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-09 02:10:06 -07:00
elvischenv 5e49c3e777 Bump Flashinfer to v0.4.0 (#26326) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2025-10-08 23:58:44 -07:00
Jee Jee Li 1b2c440cd6 [Core] Relax the LoRA max rank (#26461) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-08 23:47:14 -07:00
Zhiyuan Li d24cf322e1 [Hybrid]: Decouple Kernel Block Size from KV Page Size (#24486) 
Signed-off-by: lizhiyuan <uniartisan2017@gmail.com>
Signed-off-by: Zhiyuan Li <uniartisan2017@gmail.com>
 2025-10-08 23:43:39 -07:00
Qier LiandQier Li d17f0fbf30 [Core][KVConnector] Propagate all tokens on resumed preemptions (#24926) 
Signed-off-by: Qier Li <kevin44036@gmail.com>
Co-authored-by: Qier Li <qier@fb.com>
 2025-10-09 14:43:31 +08:00
Nick Hill bb6d8c21f9 [Bugfix] Catch and log invalid token ids in detokenizer #2 (#26445) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-08 21:20:25 -07:00
Naveenraj KamalakannanandLuka Govedič e614ab7806 Separate MLAAttention class from Attention (#25103) 
Signed-off-by: Naveenraj Kamalakannan <therealnaveenkamal@gmail.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-10-08 17:11:11 -07:00