[MRV2][SD] Make Dynamic SD comatible with Full Cuda Graphs (#45953)

Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
This commit is contained in:
Ekagra Ranjan
2026-07-03 23:58:27 -04:00
committed by GitHub
co-authored by Benjamin Chislett Nick Hill
parent 67ff0ae30f
commit 07516fda67
8 changed files with 436 additions and 36 deletions
@@ -71,8 +71,5 @@ VLLM_USE_V2_MODEL_RUNNER=0 vllm serve meta-llama/Llama-3.1-8B-Instruct \
## Limitations
* only tested with Eagle and Eagle-3. Other SD methods may or may not work out of the box
* only usable with Model Runner V1
* not compatible with full cuda graph so we force piece-wise cuda graph with this feature
We are working on enabling it on MRv2 with full cuda graph support.
* Tested with Eagle, Eagle-3, and DFlash. Other SD methods may or may not work out of the box
* Full Cudagraph only works with Model Runner V2. MRv1 only supports piece-wise cuda graph with this feature