[Bugfix] Re-sync parameter tp_rank after process_weights_after_loading (fix replicated / disable_tp weight reload) (#48025)

Signed-off-by: Alex Xu <alexxu@roblox.com>
Co-authored-by: YQ-Wang <yiqingwang@roblox.com>
Co-authored-by: alexhxu <alex.xu1015@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
alexxu-roblox
2026-07-18 08:40:06 +00:00
committed by GitHub
co-authored by YQ-Wang alexhxu Cursor
parent c71a583aa9
commit d96aee0951
3 changed files with 23 additions and 7 deletions
@@ -111,6 +111,13 @@ def process_weights_after_loading(
# parameters onto device for processing and back off after.
with device_loading_context(module, target_device):
quant_method.process_weights_after_loading(module)
# process_weights_after_loading may swap in freshly-created
# Parameters (e.g. FP8 requantization), which are stamped with the
# global rank in BasevLLMParameter.__init__. Re-reconcile their TP
# state to the layer so a later weight reload / RL weight-refit
# narrows replicated (disable_tp) weights at the correct offset.
if hasattr(module, "update_param_tp_status"):
module.update_param_tp_status()
# Repacking transients above can leave large amounts of memory in
# the caching allocator, which starves the OS on UMA devices.
release_device_memory_under_pressure(target_device)