Cyrus Leung
c8824a1e42
[MM] Move supports_multimodal_inputs and cache out of registry ( #57913 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-21 07:43:07 -07:00
1185254851
[Bugfix][KV Connector] Propagate cache reset failure during sleep ( #54581 )
...
Signed-off-by: CrazyCoder <18235787078@163.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-09-21 09:25:02 +08:00
092bdd6d57
[Frontend][Core] Add release_kv_cache_memory() API ( #44890 )
...
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: AlanFokCo <alanfok2868@gmail.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com >
Co-authored-by: AlanFokCo <alanfok2868@gmail.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-18 07:12:12 +08:00
Wei Zhao
ccde9c61e9
[Bugfix] Restore KV cache metadata GET method for external event consumer ( #56925 )
...
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-09-17 23:01:35 +00:00
Oxana Korzh and Andreas Karatzas
2307a1d16c
[Core] Don't issue a blocking collective RPC during engine handshake ( #57226 )
...
Signed-off-by: okorzh <okorzh@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-16 21:30:59 -05:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
f37c550bf6
[Feature][Frontend] Expose effective attention block size for DCP ( #56538 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Signed-off-by: Julien Darve <jdarve@NVIDIA.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-16 08:29:43 +00:00
Chaojun Zhang and Kunshang Ji
622934e61f
[XPU] Honor device_ids for worker device placement ( #56015 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-16 11:53:23 +08:00
Wentao Ye
6b15bea080
[Refactor] Remove utils dead code ( #53941 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-08 11:22:09 -04:00
aoshen02
039ea82660
[Core] Sync DP state on the first step of a wave ( #52957 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
2026-09-06 02:16:58 +00:00
Jiangyun Zhu
98ed0856f3
[Model] add GLM-5.3-Flash support ( #53906 )
2026-09-03 09:40:35 -07:00
aoshen02
17da48596c
[Bugfix][DP] Synchronize the device on pause completion ( #52914 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
2026-08-26 09:50:33 -07:00
Ning Xie
23310bc3de
[sleep functionality] code refactor about sleep/wake_up ( #50431 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
2026-08-25 07:45:20 +00:00
Lucas Wilkinson
8bdc70ec7b
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout ( #51718 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-08-21 17:47:49 -07:00
lxy and “新颐”
0c8c3f41cf
[Bugfix][V1] Sync mamba_block_size via EngineCoreReadyResponse ( #50809 )
...
Signed-off-by: “新颐” <lxy537258@antgroup.com >
Co-authored-by: “新颐” <lxy537258@antgroup.com >
2026-08-19 16:44:36 +00:00
3a386cfaf5
[ROCm] Give EngineCore cleanup grace after request abort ( #52281 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-19 11:39:59 -05:00
aoshen02 and Claude Opus 5
7ddb50788d
[Bugfix][DP] Don't assume the engines started when forwarding a wake ( #51481 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-18 13:49:35 -07:00
Connor Carpenter and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f08a95f8d8
[Rust Frontend][gRPC] Advertise LoRA capabilities ( #52031 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-17 13:27:56 -07:00
fa9d67f782
[EC Connector] Added Build Connector Worker Meta for EC Connector ( #49585 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: omerpaz95 <omerpaz95@gmail.com >
Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com >
Co-authored-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-08-15 15:40:38 -07:00
Connor Carpenter and Nick Hill
f473870ecf
[Rust Frontend][gRPC] Add RL lifecycle control ( #51316 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-14 11:32:39 -07:00
3962042304
[Bugfix][V1][Multimodal] Recover from P0/P1 processor cache drift ( #46747 ) ( #46747 )
...
Signed-off-by: William Zhao <williamzh@meta.com >
Signed-off-by: WillZZZy <93454470+WillZZZy@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-12 12:06:57 +08:00
aoshen02 and Andy Lo
8cfa01cdd2
[BugFix] Dense multinode DP rescope with regression test ( #49212 )
...
Signed-off-by: Andy Lo <andy@mistral.ai >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Andy Lo <andy@mistral.ai >
2026-08-05 18:12:03 -07:00
5f2ee2fa8c
[Bugfix][Core] Log KV cache capacity after block-size resolution ( #50462 )
...
Signed-off-by: Tanmay Dixit <22117787+tandixit95@users.noreply.github.com >
Signed-off-by: Michael Goin <mgoin64@gmail.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Tanmay Dixit <22117787+tandixit95@users.noreply.github.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-04 12:53:32 +00:00
Connor Carpenter and Nick Hill
48fc2e2797
feat(grpc): add KV event source discovery ( #50033 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-29 13:46:40 -07:00
Nick Hill and Andreas Karatzas
6453fc0b8c
[Bugfix] Don't reuse engine core payload buffer while zmq is sending it ( #50053 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-28 16:09:28 +00:00
Itay Alroy
35efdf6b34
[Elastic EP] Async preparation ( #47288 )
...
Signed-off-by: Itay Alroy <ialroy@nvidia.com >
2026-07-28 05:19:13 -07:00
Shuolei Wang
9069a57139
[Core][Frontend] Add weight version tagging for RL rollouts ( #49040 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Shuolei Wang <948904026@qq.com >
2026-07-28 14:35:29 +08:00
Connor Carpenter and Nick Hill
e3c2fc3b3c
[Rust Frontend][gRPC] Add server and model discovery ( #49491 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-27 09:53:27 -07:00
Nick Hill and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
59a6b0411d
[Core] Fix internal LB load-balancing ( #49204 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-27 14:00:46 +00:00
0b0bd2b5f6
[Feature] Add fault tolerance framework (simplified) for DP+EP external LB deployments ( #44428 )
...
Signed-off-by: fangyuchu <fangyuchu@qq.com >
Signed-off-by: a798347923 <2645302020@qq.com >
Signed-off-by: TianZhuo <2770730562@qq.com >
Signed-off-by: a798347923 <39047817+a798347923@users.noreply.github.com >
Signed-off-by: 205150940 <112750056+205150940@users.noreply.github.com >
Signed-off-by: w00689259 <wangzhuo66@huawei.com >
Signed-off-by: zWaNg3 <37772915+zWaNg3@users.noreply.github.com >
Signed-off-by: zWaNg3 <389750525@qq.com >
Signed-off-by: yzchang-plus <1078477584@qq.com >
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com >
Co-authored-by: zWaNg3 <37772915+zWaNg3@users.noreply.github.com >
Co-authored-by: a798347923 <2645302020@qq.com >
Co-authored-by: TianZhuo <2770730562@qq.com >
Co-authored-by: 205150940 <112750056+205150940@users.noreply.github.com >
Co-authored-by: a798347923 <39047817+a798347923@users.noreply.github.com >
Co-authored-by: w00689259 <wangzhuo66@huawei.com >
Co-authored-by: zWaNg3 <389750525@qq.com >
Co-authored-by: yzchang-plus <1078477584@qq.com >
Co-authored-by: Jade Zheng <zheng.shoujian@outlook.com >
2026-07-25 11:49:09 -04:00
zhrrr and Nick Hill
12213c6795
[Bugfix] handle grammar compilation failures to avoid engine crash ( #47312 )
...
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-23 18:13:49 +08:00
5a65ba5f17
[Refactor] Move iteration logging to the frontend ( #46647 )
...
Signed-off-by: maxyanghu <hyoung2991@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Shang Wang <shangw@nvidia.com >
2026-07-15 17:59:05 -07:00
Taneem Ibrahim and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
9182e86971
Log fully resolved pooling config at startup ( #48030 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-14 22:00:46 +00:00
omerpaz95 and Or Ozeri
8df14cfc8c
[EC Connector] Add EC Transfer Params ( #42433 )
...
Signed-off-by: omerpaz95 <omerpaz95@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-07-12 14:35:33 +03:00
Robert Shaw
c5e3c40877
Fix P/D with DP Supervisor ( #46628 )
...
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-06-25 13:13:08 -07:00
Wentao Ye
1cd3e0e945
[Bug] Fix IndentationError: expected an indented block after 'with' statement ( #46627 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-06-24 23:14:17 +08:00
93ec645878
[Bugfix] Fix illegal memory access from a forward during a partial wake_up ( #44483 )
...
Signed-off-by: Meihan-chen <zr010426ztt@outlook.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-06-24 22:12:23 +08:00
ebfbcfe46a
Stop setting CUDA_VISIBLE_DEVICES internally in vLLM, add device_ids arg ( #45026 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: kourosh hakhamaneshi <kouroshHakha@users.noreply.github.com >
2026-06-20 13:38:10 -07:00
Ashar
4cb5e746b6
[Rust Frontend]: Add /get_world_size route with static parallel size ( #44801 )
2026-06-18 13:10:20 +00:00
Nick Hill
d8d95998dc
[Core] Add prefill step cadence for better non-PD DP balancing ( #44558 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-16 13:17:18 -07:00
yifei wu and Harry Mellor
a52205bccf
[Model] Add HrmTextForCausalLM (Hierarchical Reasoning Model — Text) ( #43098 )
...
Signed-off-by: Wuyifei <wuyifei@me.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-16 22:41:41 +08:00
Ethan Feng
b7f9b6ab27
[Metrics] Add group-aware KV cache capacity to vllm:cache_config_info ( #42206 )
...
The startup log already reports the correct group-aware KV cache capacity for
hybrid models, but Prometheus did not expose matching info in 'vllm:cache_config_info`.
This PR adds kv_cache_size_tokens and kv_cache_max_concurrency.
Signed-off-by: Ethan Feng <ethan.fengch@gmail.com >
2026-06-12 11:49:44 +00:00
eb28452b10
[Model] Add DiffusionGemma Support ( #45163 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Martin Kukla <martin.kukla@cantab.net >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Dipika Sikka <dsikka@redhat.com >
Co-authored-by: NickLucche <nlucches@redhat.com >
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com >
Co-authored-by: Alec Kohlhoff <134344302+aleckohlhoff@users.noreply.github.com >
Co-authored-by: Porras Huang <20535584+porrashuang@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com >
2026-06-11 22:17:35 -07:00
qizixi
96229fa99e
[KVConnector][1/N] PP-aware handshake aggregation and intermediate-PP output plumbing ( #43720 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
2026-06-04 22:04:19 -07:00
Wentao Ye
41a4829f22
[Logs Refactor] Optimize shutdown logs, easier to follow and consistent ( #43707 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-06-04 14:36:32 -04:00
0c6631f02a
[KVCache] Support Pluggable KVCacheSpec ( #37505 )
...
Signed-off-by: MengqingCao <cmq0113@163.com >
Signed-off-by: Mengqing Cao <cmq0113@163.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-03 09:05:16 -07:00
Nick Hill
e15f20258b
[ModelRunnerV2] Avoid pipeline parallel bubbles ( #42187 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-02 14:02:01 -07:00
Nick Hill
cab5c9a2a9
[Core] Move max_concurrent_batches to VllmConfig ( #44274 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-02 08:57:25 -07:00
654bd2bca4
[Bugfix] Sync block_size from EngineCore to frontend for hybrid Mamba… ( #42967 )
...
Signed-off-by: Amit Gruner <agruner@crusoe.ai >
Co-authored-by: Amit Gruner <agruner@crusoe.ai >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-06-02 13:41:00 +00:00
Cyrus Leung