iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,968 Commits 3 Branches 0 Tags
f83ea61cfe77f0b200f2ded1fda9be5ff591af2c
Commit Graph
1165 Commits
Author SHA1 Message Date
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
Yongye ZhuandSummer Yang 442bd00d9f [Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 (#55072) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
 2026-09-25 03:27:21 +00:00
Yifan QiaoOpenAI CodexKimi Code
 		b3a87b2d04 [Bugfix][KVConnector] Finalize saves on steps without a forward (#57775) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
 2026-09-25 00:06:36 +00:00
Zijing LiuYifan QiaoCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7a9406690e [KV Connector] Fix DecodeBench fp8 fill values and add a startup fill mode (#58472) 
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 23:55:47 +00:00
stefankoncarevicandAndreas Karatzas 9ac3b7c69f [ROCm][CI] Mirror the DSv4-Flash disaggregated DP EP group on MI355 (#58558) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 18:38:25 -05:00
snadampalandNicolò Lucchesi 6a86bf626e [PD][PushConnector] Record last activity of remotes on the D side (#52245) 
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
 2026-09-24 23:15:32 +00:00
Vincent CavéShiksha PatelDouglas LehrAndreas Karatzas
 		92e9c6bf13 [ROCm] Fix misrouting race-condition in multi-decode P/D disagg with mori-io (#51681) 
Signed-off-by: Vincent Cave <vincent.cave@amd.com>
Signed-off-by: Shiksha Patel <shikpate@amd.com>
Co-authored-by: Shiksha Patel <shikpate@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 19:52:43 +00:00
Kapil AryaandOr Ozeri 70fc359d25 [Bugfix][KV Offload] Retain offload event metadata through batch translation (#57453) 
Signed-off-by: Kapil Arya <kapila@nvidia.com>
Signed-off-by: Kapil Arya <kapil.arya.17@gmail.com>
Co-authored-by: Or Ozeri <or@ozery.com>
 2026-09-24 07:09:12 +03:00
Lucas WilkinsonQiuChunshuoClaude Fable 5.1CodexRobert ShawNick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e6dc16cebd [PCP] Support prefill context parallelism with data parallelism (#57075) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-23 22:32:36 -04:00
Dao007foreverandCodex c4f6ce4874 [Bugfix][NIXL] Restore successful push completion reporting (#58188) 
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-23 09:39:04 -07:00
Nicolò LucchesiandYifan Qiao 9488318c6a [Mooncake] Address review nits from #56855 (#57174) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Yifan Qiao <17067717+ivanium@users.noreply.github.com>
 2026-09-23 10:15:45 +02:00
Tianyu GuoandCyrus Leung 3ee2906f47 [EPD] Support metadata-only audio inputs (#57887) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-23 09:03:50 +08:00
Tony LinandFlora Feng e4340e41c9 [Feat][XPU] VLLM_BATCH_INVARIANT support for Dense/MoE models (#55881) 
Signed-off-by: Tony Lin <tony.lin@intel.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-22 11:06:54 -04:00
Karen Chung 986e217158 [Feature] Add first-class KV hints request envelope for programmatic KV management (#53423) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-09-21 09:56:11 -07:00
Lucas WilkinsonandOpenAI Codex 8c0825090d [Bugfix][NIXL] Fix DCP pulls across MLA cache regions (#57389) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-21 11:57:49 -04:00
Hongxin Xuaoshen02OpenAI CodexRoger WangNick HillCodex
 		0b7f11a1ee [AuxOutput] Add block-keyed storage for routed-expert outputs (#45635) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-21 10:25:05 +00:00
YiSheng5andKunshang Ji 17e50b9b76 [XPU][UT]Bugfix when the process can't see all the world_size meet accuracy issue. (#57779) 
Signed-off-by: yisheng <yi.sheng@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-21 06:50:25 +08:00
YukioZzz 9679173788 [KVConnector][MoRIIO] Transfer hybrid mamba/KDA recurrent state in READ mode (#51052) 
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
 2026-09-20 17:30:34 -04:00
hangangqiangmukeshbhandarkaraoshen02Claude Fable 5.1aoshen02
 		90531daf6d [Core][Frontend] Bind KV-event publishers at port 0 and expose the bound endpoints (#55844) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: mukeshbhandarkar <shell00035@gmail.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-18 18:11:05 -07:00
ys2025-AI 6aab78f477 [Bugfix][KVConnector] Make ExampleHiddenStatesConnector abort-safe (#56841) 
Signed-off-by: ys2025-AI <yuanshuaien@163.com>
 2026-09-18 21:53:02 +00:00
Nicolò LucchesiandOpenAI Codex 49e0427ca5 [Bugfix][NIXL] Avoid receive reports for notification-only requests (#57570) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-18 13:33:02 -07:00
Nicolò Lucchesi 164dca8d1e [Nixl] Separate transport-failure metrics from KV expiry (#55854) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-09-18 18:08:59 +02:00
Kevin Liandkevinli 911548247a [Distributed][MoonEP] BF16 integration of MoonEP balanced EP backend (#52101) 
Signed-off-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com>
Co-authored-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com>
 2026-09-18 10:34:01 -04:00
Nicolò LucchesiOpenAI CodexLucas Wilkinson
 		67e5b0acc9 [Bugfix][HiSparse][NIXL] Import full blocks without tail prefill on D (#57049) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-09-17 15:26:10 -04:00
Alexander Leeaoshen02Claude Opus 5aoshen02mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		9e1cd262cb [Bugfix] Use DP index for dense DP weight updates and EC CPU region (#56950) 
Signed-off-by: lxy-alexander <lxy_alexander@outlook.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-17 22:44:36 +08:00
Tianyu GuoZhou zihengOpenAI Codex
 		0d72627565 [Perf][EPD] Batch image requests per encoder (#57095) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-17 03:45:52 -07:00
Itay EtelisandOr Ozeri b6e7c1f1f0 [kv_offload] Skip scratch groups (#57145) 
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-09-17 13:22:49 +03:00
drakoshaClaudeColton OttleyOr Ozeri
 		40b40d1d39 [Bugfix][KV Offload] Register the offload region in chunks (#51081) 
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Colton Ottley <coltonottley@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-09-17 13:19:56 +03:00
Qiming Zhang e6b1a5e5e3 [XPU] Enable XPU eplb (#44987) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com>
 2026-09-17 13:19:13 +08:00
Zhou zihengTianyu Guojiangkuaixue123Codex
 		2743dc4f92 [Feature][Multimodal] Add cross-encoder caching to Mooncake P2P (#56242) 
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-17 13:16:41 +08:00
Matt 2d8a5c5741 [ROCm][Bugfix] Reduce CUDA graph divergences (#57229) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
 2026-09-17 05:12:13 +00:00
Kevin H. Luu 0bfc7a15d0 [CI] Fix ruff docstring violations in EC connector files (#57283) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-09-16 21:13:15 -07:00
Robert ShawandRobert Shaw c38f92aae7 [A2A] Allow EPLB + SharedExpert Overlap + DeepEPv2 (#57236) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-09-17 04:02:39 +00:00
Itay EtelisandOr Ozeri 9854b580df [Bugfix][KV Offload] Compact canonical MLA rows (#56799) 
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-09-17 06:24:06 +03:00
omerpaz95 fa3622a4a1 [EC Connector] Add Metrics Collection (#54960) 
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
 2026-09-17 10:36:23 +08:00
Nicolò Lucchesi fc8132a5e5 [Bugfix] Fix np.float64 leaking into the KV transfer metrics log (#57068) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-09-16 18:09:05 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Nicolò Lucchesi 804e5377aa [Bugfix][HiSparse][PD] Align region-mapped pulls across logical block sizes (#57077) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-09-16 18:39:07 +02:00
Nicolò Lucchesi 8b1d188046 [Bugfix][Mooncake] Report request-level KV load failures under HMA (#56855) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-09-16 14:55:40 +02:00
MINJUN GILandOr Ozeri f12fe10c5c [Bugfix][KV Offload] Track cache recency once per request (#51787) 
Signed-off-by: mindungil <alswnsrlf12@naver.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-09-16 10:47:26 +03:00
AlexHuangOpenAI CodexItay Etelis
 		0f8fa53acf [Bugfix][KV Offload] Check cgroup memory before SHM allocation (#54014) 
Signed-off-by: Alex <jihui.huang@daocloud.io>
Signed-off-by: AlexHuang <alex.tech.lab@outlook.com>
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
 2026-09-16 09:17:50 +03:00
Liqian LinCodexOr Ozeri
 		961c5b6b7e [Bugfix][KV Offload] Restore MTP-retained sliding-window history (#56709) 
Signed-off-by: jacklin78911-collab <jacklin78911@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-09-16 08:14:09 +03:00
alberto 75dc588269 [KV Offload] Add per-request max_load_tokens control (#55885) 
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
 2026-09-16 06:57:38 +03:00
Giancarlo Delfin 85c1f58d50 [BugFix][Model Runner V2][Spec Decode] Fix decode instance's multi-layer MTP kv caches during P/D (#55055) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-09-15 17:31:25 -07:00
qizixiClaudeYiliu DongCodex
 		0fefffc934 [KVConnector][NIXL] Support attention-HMA layouts in pipeline-parallel push prefill (#50494) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-15 15:43:56 -07:00
Nicolò LucchesiandNick Hill 45779ef99f [Bugfix][NIXL] Fix multiple handles xfer race (#56104) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-15 04:27:10 +00:00
Yizheng JiaoandNick Hill 381c61e008 [Bugfix][NIXL] Report a full prefix cache hit as finished (#56640) 
Signed-off-by: Yizheng Jiao <jyizheng@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-14 20:15:19 -07:00
Liqian Lin e45bb43984 [Bugfix][KV Offload] Ignore pending chunks in invalid sliding windows (#56486) 
Signed-off-by: Liqian Lin <jacklin78911@gmail.com>
 2026-09-15 05:48:38 +03:00
Andreas KaratzasandCodex a7576447b8 [ROCm][Bugfix] Initialize Ray NIXL agents for sharded RDT (#56687) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-15 00:34:21 +00:00