[BugFix] Fix cache issue in compilation_config (#31376)

Signed-off-by: Boyuan Feng <boyuan@meta.com>
This commit is contained in:
Boyuan Feng
2025-12-27 09:30:39 -05:00
committed by GitHub
parent 40a8756224
commit 2f12cd32c0
2 changed files with 47 additions and 0 deletions
+5
View File
@@ -1360,6 +1360,11 @@ def set_current_vllm_config(
num_models_seen = compilation_counter.num_models_seen
try:
# Clear the compilation config cache when context changes.
# This is needed since the old config may have been accessed
# and cached before the new config is set.
get_cached_compilation_config.cache_clear()
_current_vllm_config = vllm_config
_current_prefix = prefix
yield