[Bugfix][EC Connector] Don't stop an encoder-instance request before its images are encoded (#50275)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
This commit is contained in:
@@ -108,8 +108,8 @@ async def fanout_encoder_primer(
|
||||
"messages": [
|
||||
{"role": "user", "content": [item]},
|
||||
],
|
||||
# Only need 1 token so the server actually runs the encoder path
|
||||
"max_tokens": 1,
|
||||
# No max_tokens cap: the encoder instance never samples, it finishes
|
||||
# once the prompt is encoded and its embeddings are published.
|
||||
"stream": False,
|
||||
}
|
||||
tasks.append(
|
||||
|
||||
Reference in New Issue
Block a user