[Feature][Frontend] Expose effective attention block size for DCP (#56538)
Co-authored-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: Julien Darve <jdarve@NVIDIA.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
This commit is contained in:
co-authored by
Bugen Zhao
OpenAI Codex
parent
0d8173d153
commit
f37c550bf6
@@ -2,6 +2,9 @@
|
||||
|
||||
This directory is the canonical source for vLLM's gRPC schema.
|
||||
|
||||
See [context parallel deployment](../../docs/serving/context_parallel_deployment.md)
|
||||
for effective attention block-size metadata in Python and `Control.GetServerInfo`.
|
||||
|
||||
Schema updates are no longer published to the Buf Schema Registry. Rust consumers
|
||||
should use `vllm-proto` from crates.io; consumers in other languages can generate
|
||||
bindings from the `.proto` files in this directory. Buf still builds and lints
|
||||
@@ -34,8 +37,7 @@ On pull requests and releases, `cargo-semver-checks` compares the crate with its
|
||||
latest published version. Include any required version bump in the protocol
|
||||
change PR. This check becomes available after the first manual publication.
|
||||
|
||||
1. Update the crate version and the `vllm-proto` workspace dependency together,
|
||||
and update `rust/Cargo.lock`.
|
||||
1. Update the crate version in `rust/proto/Cargo.toml` and update `rust/Cargo.lock`.
|
||||
2. Run `cargo publish --manifest-path rust/proto/Cargo.toml --locked --dry-run`
|
||||
and the frontend gRPC tests. Record the tested vLLM releases or revisions in
|
||||
the release notes; matching crate versions alone do not establish runtime compatibility.
|
||||
|
||||
@@ -50,6 +50,8 @@ message ServerInfo {
|
||||
uint64 max_batched_tokens = 9;
|
||||
uint32 max_loras = 10;
|
||||
RlCapabilities rl_capabilities = 11;
|
||||
// Full-attention block size in tokens after initialization; absent if unavailable.
|
||||
optional uint64 effective_attention_block_size = 14;
|
||||
}
|
||||
|
||||
message RlCapabilities {
|
||||
|
||||
Reference in New Issue
Block a user