[MISC] cudagraph_capture_sizes related improvements (#26016)
Signed-off-by: fhl <2410591650@qq.com> Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
co-authored by
Luka Govedič
Harry Mellor
parent
435be10db9
commit
284cc92275
@@ -104,7 +104,7 @@ class EagleProposer:
|
||||
)
|
||||
|
||||
self.cudagraph_batch_sizes = (
|
||||
list(reversed(self.vllm_config.compilation_config.cudagraph_capture_sizes))
|
||||
(sorted(self.vllm_config.compilation_config.cudagraph_capture_sizes))
|
||||
if self.use_cuda_graph
|
||||
else []
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user