[EPD] Support metadata-only audio inputs (#57887)
Signed-off-by: Tianyu Guo <guoty@inferact.ai> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
This commit is contained in:
co-authored by
Cyrus Leung
parent
826e300cde
commit
3ee2906f47
@@ -61,6 +61,19 @@ and compares the disaggregated outputs against it. Set `MODEL` to use another mo
|
||||
or `MOONCAKE_EC_PROTOCOL=rdma` to use RDMA on supported hardware. See the script for
|
||||
additional configuration options.
|
||||
|
||||
### Audio inputs
|
||||
|
||||
The Python EPD proxy also rewrites audio in `/v1/chat/completions` into
|
||||
metadata-only references for Qwen2-Audio, AudioFlamingo3, Ultravox,
|
||||
Qwen2.5-Omni, and Qwen3-Omni. The encoder publishes `audio_num_tokens`
|
||||
(the placeholder token count of each audio) together with any feature
|
||||
lengths the model needs; the consumer uses them to reconstruct the audio
|
||||
placeholders and receives the embeddings through its EC connector without
|
||||
repeating audio preprocessing.
|
||||
|
||||
This covers pure audio inputs, not video with an embedded audio track. It does
|
||||
not add `/v1/audio/transcriptions` or realtime routes to the example proxy.
|
||||
|
||||
---
|
||||
|
||||
## 3 Test Script
|
||||
|
||||
Reference in New Issue
Block a user