[feat] Add explicit /start_weight_update and /finish_weight_update APIs for weight transfer (#39212)

This commit is contained in:
Sumanth R Hegde
2026-05-08 18:03:33 -07:00
committed by GitHub
parent 30f519e947
commit e3b65a5ba0
19 changed files with 287 additions and 53 deletions
+2 -4
View File
@@ -43,16 +43,14 @@ update_request = WeightTransferUpdateRequest(
### WeightTransferUpdateInfo
The base `WeightTransferUpdateInfo` includes an `is_checkpoint_format` flag:
The base `WeightTransferUpdateInfo` is a marker class for backend-specific update info:
```python
@dataclass
class WeightTransferUpdateInfo(ABC):
is_checkpoint_format: bool = True
pass
```
When `is_checkpoint_format=True` (the default), vLLM applies layerwise weight processing (repacking, renaming, etc.) on the received weights before loading them. Set to `False` if the trainer has already converted weights to the kernel format expected by the model.
## Implementing a Custom Engine
To create a custom weight transfer backend: