[Feature][DSpark]: Logprobs adaptive verification (#52242)

Signed-off-by: Naveenraj Kamalakannan <therealnaveenkamal@gmail.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
This commit is contained in:
Naveenraj Kamalakannan
2026-08-25 10:22:33 -04:00
committed by GitHub
co-authored by Benjamin Chislett
parent ca29cd48ba
commit d5cadcee86
10 changed files with 61 additions and 25 deletions
@@ -36,7 +36,7 @@ Set `enable_adaptive_verification: false` to verify the full block for every req
- The attention backend must tolerate device-decided query lengths, since the CPU lengths only bound them from above. Backends that plan off the CPU lengths are excluded by the attention selector, and rejected at startup for models that hard-wire their backend.
- Full cudagraphs are required: step costs are profiled from captured graphs, so `--enforce-eager` is rejected at startup.
- Not supported with LoRA (the per-token LoRA mapping is built from CPU-side boundaries), pipeline parallelism (cost curves and confidences exist only on the last rank), or output logprobs (to be fixed).
- Not supported with LoRA (the per-token LoRA mapping is built from CPU-side boundaries) or pipeline parallelism (cost curves and confidences exist only on the last rank).
## Tuning the cost profile