[Feature] Add first-class KV hints request envelope for programmatic KV management (#53423)

Signed-off-by: Karen Chung <karenc@nvidia.com>
This commit is contained in:
Karen Chung
2026-09-21 09:56:11 -07:00
committed by GitHub
parent 4f14516790
commit 986e217158
30 changed files with 340 additions and 18 deletions
@@ -553,6 +553,7 @@ def _create_req_context(req: Request) -> ReqContext:
return ReqContext(
req_id=req.request_id,
kv_transfer_params=params,
kv_hints=req.kv_hints,
load_tier_filter=load_filter,
)