MOVZX and Hinata
2b544b7ba0
[Mini-VLLM] Cut dead subsystems across serving, connectors, parsers
...
- Remove P/D transfer connectors: nixl, nixl_chat, mooncake x2, hf3fs,
moriio, hisparse, example and hidden-states connectors
- Reduce weight_transfer to the base API used by model loading
- Remove kv_offload tiering, simple_kv_offload, ec cpu/example,
snapshot command, engine snapshot server
- Remove transformers model fallback, dsv4 tokenizers, dead platforms
cpu/tpu/xpu, flashmla third_party, dsa, megamoe/flydsl, dead attn
ops, dead layers (short_conv, fused_embed_norm, indexer_topk,
relu2 gemma_ar, lightning_attn, minimax_rms_norm), gemm_rs_ar
- Keep MoE LoRA op (imported eagerly by punica_gpu); drop fp8 variant
- Rebuild parser hubs around Qwen3 adapters only (36 dead adapters)
- Remove dead tests with the code they covered
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 02:19:03 +07:00
6b6a6c5c98
[RL] Support per-parameter source layouts for NCCL M2N ( #60934 )
...
Signed-off-by: pkousha <pkousha@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-16 21:41:17 -04:00
Woosuk Kwon and Claude Opus 5.5
e57bb4ace7
[Core] Remove the legacy Ray compiled-graph executor ( #61712 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 18:13:27 -07:00
aoshen02
bd5bbce6e4
[RL] Release weight-transfer communicators without stopping inference ( #61060 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: fh3 <a@b>
2026-10-11 21:15:14 +08:00
3c062d391d
[Bugfix] Support 0-dim tensors in packed weight transfer ( #52235 )
...
Signed-off-by: subhash polisetti <subhashr161347@gmail.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 01:41:06 +00:00
Harry Mellor and Claude Opus 5.5
21d9758459
Remove Python 3.10 support now that it is EOL ( #60402 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:30:52 -07:00
4187ade7b8
[Frontend] Consolidate rl entrypoints. ( #57849 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 21:37:11 -07:00
a810316c08
[Bugfix] Keep batch-invariance NCCL pins out of the weight-transfer group ( #59500 )
...
Signed-off-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-02 11:19:28 +08:00
Hyunjae Woo and aoshen02
5a661487e2
[Feature] Add native ModelExpress weight transfer backend ( #58399 )
...
Signed-off-by: Hyunjae Woo <hwoo@nvidia.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-02 02:29:35 +00:00
b6d2b4fc25
[RL] Add sharding-aware NCCL M2N weight transfer ( #51520 )
...
Signed-off-by: Ke Wen <kwen@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-26 01:38:53 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Andreas Karatzas and Codex
a7576447b8
[ROCm][Bugfix] Initialize Ray NIXL agents for sharded RDT ( #56687 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-15 00:34:21 +00:00
dharak-cohere and Cursor
d0e695a91b
[Distributed] Support pre-shared ncclUniqueId rendezvous for weight transfer ( #53784 )
...
Signed-off-by: Dharak Kharod <dharak.kharod@cohere.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-01 14:55:57 +08:00
6c4aa8bec3
[RL] Support checkpoint-coordinate sparse NCCL weight updates ( #53751 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-27 18:19:51 +08:00
ee5ae2ae70
[RL] Add rank-local IPC weight updates ( #52497 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-08-26 15:36:57 +08:00
Quentin Gallouédec
26858770ec
[Bugfix] Reuse CUDA streams in packed weight transfer to cap reserved-memory waste ( #52951 )
...
Signed-off-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co >
2026-08-23 23:54:41 -07:00
Aaron Hao
6cd9713463
[RL] P2P RDT weight sync ( #43375 )
2026-08-23 09:42:22 -07:00
Aaron Hao and Claude Opus 5
793ca6998a
[Docs][RL] Rewrite weight-transfer docs; standardize examples ( #51729 )
...
Signed-off-by: haoaaron <ahao@anyscale.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-11 18:53:15 -07:00
acmore
f37dff22fc
[Bugfix] Import each packed IPC export once on the consumer side ( #51259 )
...
Signed-off-by: acmore <hezj07+github@gmail.com >
2026-08-11 10:05:27 +08:00
Aaron Hao and Claude Opus 5
21ea5b4fa1
[rl] Stateful Trainer Send: NCCL + Sparse NCCL [3/N] ( #50902 )
...
Signed-off-by: haoaaron <ahao@anyscale.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-07 00:35:59 -07:00
Aaron Hao
30b4e7f479
[rl] Stateful Trainer Send: IPC [2/N] ( #48981 )
...
Signed-off-by: haoaaron <ahao@anyscale.com >
2026-07-30 21:59:08 +08:00
Shuolei Wang
9069a57139
[Core][Frontend] Add weight version tagging for RL rollouts ( #49040 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Shuolei Wang <948904026@qq.com >
2026-07-28 14:35:29 +08:00
aoshen02 and aoshen02
ba18929079
[Bugfix][SpecDecode] Scope MTP completeness checks outside bucketed updates ( #49178 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
2026-07-21 18:56:39 -07:00
Aaron Hao and Sumanth R Hegde
fb1d8ccaf5
[rl] Stateful Trainer Send: New Abstractions [1/N] ( #48042 )
...
Signed-off-by: haoaaron <ahao@anyscale.com >
Signed-off-by: Aaron Hao <ahao@anyscale.com >
Co-authored-by: Sumanth R Hegde <39546518+SumanthRH@users.noreply.github.com >
2026-07-17 15:11:06 +08:00
fc1c548093
Runtime Draft Weight Update for Speculative Decoding ( #46725 )
...
Signed-off-by: vx120 <893600387@qq.com >
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: crp0128 <191679376@qq.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-07-11 22:51:53 -07:00
Aaron Hao and Claude Sonnet 4.6
77a9c5ae28
Weight sync refactor + move sparse nccl engine ( #44353 )
...
Signed-off-by: hao-aaron <ahao@anyscale.com >
Signed-off-by: haoaaron <ahao@anyscale.com >
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-07-01 01:25:19 -07:00
Siddharth Bedekar and OpenAI Codex
266b9d9c64
[Frontend][Core] Add sparse NCCL weight transfer support for in-place updates ( #40096 )
...
Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-06-01 15:37:30 -04:00
Sumanth R Hegde and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
3cb83c9592
Add model to WeightTransferEngine.__init__ ( #42922 )
...
Signed-off-by: SumanthRH <sumanthrh99@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-22 17:52:15 -07:00
Aaron Hao
e0a45f1455
[Feat][RL] IPC weight sync optimizations: multigpu support and chunked packed tensors ( #37476 )
...
Signed-off-by: ahao-anyscale <ahao@anyscale.com >
Signed-off-by: hao-aaron <ahao@anyscale.com >
2026-05-15 22:53:06 +08:00
Sumanth R Hegde
e3b65a5ba0
[feat] Add explicit /start_weight_update and /finish_weight_update APIs for weight transfer ( #39212 )
2026-05-08 18:03:33 -07:00
Isotr0py
a836524d20
[Chore] Replace all base64 usages with faster pybase64 package ( #37290 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-03-17 14:44:19 +00:00
Aaron Hao
5e1a373d2e
[BUG] Fix rank calculation in NCCLWeightTransferEngine ( #36940 )
...
Signed-off-by: hao-aaron <ahao@anyscale.com >
2026-03-13 01:56:51 +00:00
Kunshang Ji
53ec16a705
[Hardware] Replace torch.cuda.device_count/current_device/set_device API ( #36145 )
...
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com >
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
2026-03-12 07:57:47 -07:00
Simon Mo and Cursor Agent
f678c3f61a
[RL] [Weight Sync] Guard IPC update-info pickle deserialization behind insecure serialization flag ( #35928 )
...
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2026-03-04 17:05:32 -05:00
Aaron Hao
2ce6f3cf67
[Feat][RL][2/2] Native Weight Syncing API: IPC ( #34171 )
...
Signed-off-by: hao-aaron <ahao@anyscale.com >
Signed-off-by: Aaron Hao <ahao@anyscale.com >
Signed-off-by: ahao-anyscale <ahao@anyscale.com >
2026-02-27 13:45:21 -07:00
Aaron Hao and SumanthRH
c1858b7ec8
[Feat][RL][1/2] Native Weight Syncing API: NCCL ( #31943 )
...
Signed-off-by: ahao-anyscale <ahao@anyscale.com >
Signed-off-by: Aaron Hao <ahao@anyscale.com >
Co-authored-by: SumanthRH <sumanthrh99@gmail.com >
2026-02-05 12:13:23 -05:00