[Bugfix][EC Connector] Don't stop an encoder-instance request before its images are encoded (#50275)

Signed-off-by: Tianyu Guo <guoty@inferact.ai>
This commit is contained in:
Tianyu Guo
2026-08-05 05:53:58 +00:00
committed by GitHub
parent 7794b1e08b
commit 50c51682a1
4 changed files with 121 additions and 5 deletions
@@ -108,8 +108,8 @@ async def fanout_encoder_primer(
"messages": [
{"role": "user", "content": [item]},
],
# Only need 1 token so the server actually runs the encoder path
"max_tokens": 1,
# No max_tokens cap: the encoder instance never samples, it finishes
# once the prompt is encoded and its embeddings are published.
"stream": False,
}
tasks.append(