[ROCm] Fix OOB During Model Warmup With ROCM_ATTN and MRV2 (#46548)

Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
This commit is contained in:
Micah Williamson
2026-06-24 10:53:21 -05:00
committed by GitHub
co-authored by Matthew Wong
parent 1cd3e0e945
commit b3a688cb9e
3 changed files with 8 additions and 8 deletions
@@ -60,10 +60,8 @@ def test_rocm_compressed_tensors_w8a8(
vllm_runner, example_prompts, model_path, max_tokens, num_logprobs
):
dtype = "bfloat16"
# Pin to TRITON_ATTN, see https://github.com/vllm-project/vllm/issues/46179
with vllm_runner(
model_path, dtype=dtype, attention_backend="TRITON_ATTN"
) as vllm_model:
with vllm_runner(model_path, dtype=dtype) as vllm_model:
vllm_model.generate_greedy_logprobs(example_prompts, max_tokens, num_logprobs)