Signed-off-by: lizhiyuan <lizhiyuan@moonshot.cn> Signed-off-by: Zhiyuan Li <uniartisan2017@gmail.com>
Optional[x]
x | None
Union[x, y]
x | y
get_kv_cache_spec
AttentionLayerBase
vllm.utils