[Bugfix] serving_llama70B_tp4 benchmark was silently running at tensor_parallel_size=1 (#50766)

Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>
2026-08-03 02:26:48 +00:00
parent 9a4fd57cac
commit e42c230e4f
1 changed files with 1 additions and 0 deletions
@@ -76,6 +76,7 @@
      "test_name": "serving_llama70B_tp4_random_128_128",
      "server_parameters": {
        "model": "meta-llama/Llama-3.3-70B-Instruct",
+        "tensor_parallel_size": 4,
        "async_scheduling": "",
 	"no_enable_prefix_caching": "",
        "max_num_batched_tokens": 8192