 Russell Bryant
|
78aa341d12
|
[CI] Fix race condition in test_kv_cache_events test (#18169)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
|
2025-05-14 16:27:48 -07:00 |
|
|
bnellnm
|
f9c069c85e
|
Modularize fused experts and integrate PPLX kernels (#15956)
|
2025-05-14 13:11:54 -07:00 |
|
|
Nick Hill
|
59dd311cf5
|
[KVConnector] Keep KVTransferParams as a dict (#18033)
|
2025-05-14 08:05:57 -07:00 |
|
|
Harry Mellor
|
dc372b9c8a
|
Update deprecated type hinting in vllm/device_allocator and vllm/distributed (#18126)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-05-14 04:07:57 -07:00 |
|
|
youkaichao
|
6266c57bae
|
[core][distributed] add ep group and all2all interface (#18077)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2025-05-14 10:46:49 +08:00 |
|
|
Nick Hill
|
d5af47a149
|
[P/D] Add some more debug logs to NixlConnector (#18102)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-05-13 19:33:03 -07:00 |
|
|
Nick Hill
|
55aa7af994
|
[V1] DP scale-out (2/N): Decouple engine process management and comms (#15977)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-05-13 10:48:21 -07:00 |
|
|
|
d19110204c
|
[P/D] NIXL Integration (#17751)
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Brent Salisbury <bsalisbu@redhat.com>
|
2025-05-12 09:46:16 -07:00 |
|
|
Harry Mellor
|
4b2ed7926a
|
Improve configs - the rest! (#17562)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-05-09 15:18:44 -07:00 |
|
|
Isotr0py
|
6e4a93e3f7
|
[Bugfix][CPU] Fix broken AVX2 CPU TP support (#17252)
Signed-off-by: Isotr0py <2037008807@qq.com>
|
2025-05-09 08:55:14 +00:00 |
|
|
Tyler Michael Smith
|
68e1ee0072
|
[Bugfix][Easy] Fix whitespace in shm_broadcast.py logging (#17635)
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2025-05-04 19:20:19 -07:00 |
|
|
Harry Mellor
|
d6484ef3c3
|
Add full API docs and improve the UX of navigating them (#17485)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-05-03 19:42:43 -07:00 |
|
|
AlecandMark McLoughlin
|
0be6d05b5e
|
[V1][Metrics] add support for kv event publishing (#16750)
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
|
2025-04-30 07:44:45 -07:00 |
|
|
Russell Bryant
|
a0304dc504
|
[Security] Don't bind tcp zmq socket to all interfaces (#17197)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
|
2025-04-28 10:08:20 -07:00 |
|
|
cascadeandTyler Michael Smith
|
690fe019f0
|
[Feature] support sequence parallelism using compilation pass (#16155)
Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2025-04-27 06:29:35 -07:00 |
|
|
Yihua Cheng
|
5e83a7277f
|
[v1] [P/D] Adding LMCache KV connector for v1 (#16625)
|
2025-04-26 03:03:38 +00:00 |
|
|
Russell BryantandShangming Cai
|
a5450f11c9
|
[Security] Use safe serialization and fix zmq setup for mooncake pipe (#17192)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-04-25 16:53:23 +00:00 |
|
|
jglaser
|
0d6e187e88
|
Use custom address for listening socket (#15988)
Signed-off-by: Jens Glaser <glaserj@ornl.gov>
|
2025-04-25 01:57:16 +00:00 |
|
|
Areeb Syed
|
ed2e464653
|
Addendum Fix to support FIPS enabled machines with MD5 hashing (#17043)
Signed-off-by: sydarb <areebsyed237@gmail.com>
|
2025-04-23 19:55:00 -07:00 |
|
|
|
3408e47159
|
[P/D][V1] KV Connector API V1 (#15960)
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: remi <remi@mistral.ai>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
|
2025-04-17 13:22:40 -07:00 |
|
|
|
2b05b8ce69
|
[V1][Frontend] Improve Shutdown And Logs (#11737)
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
|
2025-04-16 19:48:34 -07:00 |
|
|
billishyahao
|
3ac98edcb1
|
[Feature] add model aware kv ops helper (#16020)
Signed-off-by: billishyahao <bill.he@amd.com>
|
2025-04-15 23:00:43 -07:00 |
|
|
shangmingc
|
1dd23386ec
|
[Misc] Update usage with mooncake lib for kv transfer (#16523)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-04-14 11:31:37 +00:00 |
|
|
Huazhong Ji
|
68bb122eb4
|
[MISC] Make GroupCoordinator compatible with out-of-tree devices (#16464)
Signed-off-by: hzji210@gmail.com <hzji210@gmail.com>
|
2025-04-12 09:20:25 +00:00 |
|
|
Kebe
|
b4ac449a83
|
[Misc] Merge the logs of pp layers partitions (#16225)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2025-04-08 00:18:15 -07:00 |
|
|
Chengji Yao
|
01b6113659
|
[TPU] optimize the all-reduce performance (#15903)
Signed-off-by: Chengji Yao <chengjiyao@google.com>
|
2025-04-03 00:25:14 +00:00 |
|
|
Li, Jiang
|
550b2801ad
|
[CPU][Bugfix] Using custom allreduce for CPU backend (#15934)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2025-04-02 07:46:47 -07:00 |
|
|
Ilya Markovandilmarkov
|
b7b7676d67
|
[Distributed] Add custom allreduce support for ROCM (#14125)
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
|
2025-03-31 22:49:12 -07:00 |
|
|
shangmingc
|
6fa7cd3dbc
|
[Feature][Disaggregated] Support XpYd disaggregated prefill with MooncakeStore (#12957)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-03-29 04:01:46 -07:00 |
|
|
Nick Hill
|
6d531ad7b8
|
[Misc][V1] Misc code streamlining (#15723)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-03-28 20:59:47 -07:00 |
|
|
Kebeandyoukaichao
|
4e0f6076be
|
[Bugfix] Fix failure to launch in Tensor Parallel TP mode on macOS. (#14948)
Signed-off-by: Kebe <mail@kebe7jun.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
|
2025-03-28 10:13:41 +08:00 |
|
|
Robert Shaw
|
bd45912b99
|
[TPU] Lazy Import (#15656)
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
|
2025-03-28 09:57:01 +08:00 |
|
|
Nick Hill
|
15dac210f0
|
[V1] AsyncLLM data parallel (#13923)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-03-27 16:14:41 -07:00 |
|
|
Yi LiuandYi Liu
|
9cc645141d
|
[MISC] Refine no available block debug msg (#15076)
Signed-off-by: Yi Liu <yiliu4@habana.ai>
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Co-authored-by: Yi Liu <yiliu4@habana.ai>
|
2025-03-25 00:01:10 +08:00 |
|
|
youkaichao
|
9606d572ed
|
[distributed] fix dp group (#15355)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2025-03-24 14:54:27 +00:00 |
|
|
Russell Bryant
|
038de04d7b
|
Fix zmq IPv6 URL format error (#15341)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
|
2025-03-24 09:30:41 -04:00 |
|
|
billishyahaoandZhai Feiyue
|
742369d35a
|
[Frontend][Bugfix] support prefill decode disaggregation on deepseek (#14824)
Signed-off-by: billishyahao <bill.he@amd.com>
Co-authored-by: Zhai Feiyue <80079571+ZhaiFeiyue@users.noreply.github.com>
|
2025-03-20 00:00:33 -07:00 |
|
|
Alexander Matveev
|
cfbca8a2f2
|
[V1] TPU - Tensor parallel MP support (#15059)
|
2025-03-20 00:55:18 +00:00 |
|
|
hoshi-hiyougaandyoukaichao
|
414919138b
|
[Bugfix] torchrun compatibility (#14899)
Signed-off-by: hiyouga <hiyouga@buaa.edu.cn>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
|
2025-03-18 05:49:27 -07:00 |
|
|
Nick Hill
|
b82662d952
|
[BugFix] Fix torch distributed stateless PG backend init (#14870)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-03-15 20:26:19 -07:00 |
|
|
yasu52
|
3fb17d26c8
|
[Doc] Fix typo in documentation (#14783)
Signed-off-by: yasu52 <tsuguro4649@gmail.com>
|
2025-03-13 20:33:09 -07:00 |
|
|
Mathis Felardos
|
1bd32bc8dd
|
[Config][Disaggregated] Add timeout configuration for the torch.store and add KVTransferConfig.kv_connector_extra_config (#14367)
Signed-off-by: Mathis Felardos <mathis@mistral.ai>
|
2025-03-12 20:15:20 -07:00 |
|
|
gnovack
|
d6123170d5
|
[Neuron] Add Neuron device communicator for vLLM v1 (#14085)
|
2025-03-10 18:37:04 -07:00 |
|
|
Jiayi Yao
|
6d7f037748
|
[Feat] Support chunked prefill for LMCache connector (#14505)
Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn>
|
2025-03-08 19:30:06 -08:00 |
|
|
Mathis Felardos
|
980385f8c1
|
[Bugfix][Disaggregated] Add a check in send_kv_caches_and_hidden_states and fix the reshape of the KVCache (#14369)
Signed-off-by: Mathis Felardos <mathis@mistral.ai>
|
2025-03-07 22:39:31 -08:00 |
|
|
Kuntai Du
|
288ca110f6
|
[Security] Serialize using safetensors instead of pickle in Mooncake Pipe (#14228)
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
|
2025-03-04 21:10:32 +00:00 |
|
|
Mathis Felardos
|
b9e41734c5
|
[Bugfix][Disaggregated] patch the inflight batching on the decode node in SimpleConnector to avoid hangs in SimpleBuffer (nccl based) (#13987)
Signed-off-by: Mathis Felardos <mathis@mistral.ai>
|
2025-02-28 07:53:45 +00:00 |
|
|
Harry Mellor
|
145944cb94
|
Improve pipeline partitioning (#13839)
|
2025-02-25 18:53:56 -08:00 |
|
|
Jiayi Yao
|
2f42a4888c
|
[Feature] Support KV cache offloading and disagg prefill with LMCache connector. (#12953)
|
2025-02-25 00:38:42 -08:00 |
|
|
cjackal
|
51010a1807
|
[Misc] set single whitespace between log sentences (#13771)
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
|
2025-02-25 10:26:12 +08:00 |
|