[RFC] Replace shared-memory routed experts with ModelRunnerOutput transfer and HTTP support (#39568)

Signed-off-by: xhx1022 <1737006628@qq.com>
Signed-off-by: arlenxu <arlenxu@tencent.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: arlenxu <arlenxu@tencent.com>
Co-authored-by: Junjie Zhang <junj.jay.zhang@gmail.com>
This commit is contained in:
Hongxin Xu
2026-05-14 14:12:30 +00:00
committed by GitHub
co-authored by Claude Opus 4.6 arlenxu Junjie Zhang
parent 2317682f95
commit c7560af424
12 changed files with 605 additions and 369 deletions
+7
View File
@@ -294,6 +294,13 @@ class SamplingParams(
"""Arbitrary additional args, that can be used by custom sampling
implementations, plugins, etc. Not used by any in-tree sampling
implementations."""
routed_experts_prompt_start: int = 0
"""When enable_return_routed_experts is active, skip the first
routed_experts_prompt_start prompt tokens from the returned routing
data. In multi-turn agent scenarios, set this to the length of the
already-returned prefix to avoid duplicating routing for prompt tokens
covered by earlier turns. Default 0 returns routing for all prompt
tokens."""
# Fields used for bad words
bad_words: list[str] | None = None