[Bugfix][Spec Decode] Keep heterogeneous-vocab draft models on Model Runner V1 (#59541)
Signed-off-by: Nishant Revur <kumrfni@amazon.com> Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -3159,6 +3159,11 @@ class VllmConfig:
|
||||
):
|
||||
unsupported.append("parallel drafting for EAGLE speculative decoding")
|
||||
|
||||
# The V2 draft-model speculator has no token mapping between the
|
||||
# draft and target vocabularies (TLI is TBD in #47172).
|
||||
if getattr(speculative_config, "use_heterogeneous_vocab", False):
|
||||
unsupported.append("heterogeneous-vocabulary draft models")
|
||||
|
||||
if self.parallel_config.use_ubatching:
|
||||
unsupported.extend(self._get_dbo_unsupported_features())
|
||||
|
||||
|
||||
Reference in New Issue
Block a user