Files

T

Harry Mellor 28bffe9466 Fix docs build warning (#34686 )

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

2026-02-17 02:31:40 -08:00

metrics

[Model Runner V2] Support num NaNs in logits (#30187 )

2025-12-09 10:00:49 -08:00

[ModelRunner V2] Misc minor simplifications and optimizations (#33467 )

2026-02-01 22:17:14 +00:00

sample

[Model Runner V2] Minor refactoring for penalties (#34662 )

2026-02-16 21:43:00 -08:00

spec_decode

[Model Runner V2] Init cuda graph pool when necessary (#33217 )

2026-02-11 09:12:13 -08:00

__init__.py

GPU Model Runner V2 (#25266 )

2025-11-21 08:20:55 -08:00

async_utils.py

[ModelRunner V2] Misc minor simplifications and optimizations (#33467 )

2026-02-01 22:17:14 +00:00

attn_utils.py

[ModelRunner V2][BugFix] Fix max_query_len calculation (#34167 )

2026-02-09 21:47:17 +00:00

block_table.py

[ModelRunner V2] Misc minor simplifications and optimizations (#33467 )

2026-02-01 22:17:14 +00:00

buffer_utils.py

[Model Runner V2] Use pinned memory for write_contents (#34222 )

2026-02-10 08:55:22 -08:00

cudagraph_utils.py

[Model Runner V2] Init cuda graph pool when necessary (#33217 )

2026-02-11 09:12:13 -08:00

dp_utils.py

[ModelRunner V2] Misc minor simplifications and optimizations (#33467 )

2026-02-01 22:17:14 +00:00

input_batch.py

[Model Runner V2] Fix unintended CPU-GPU sync in make_dummy (#34667 )

2026-02-16 19:00:29 -08:00

kv_connector.py

[ModelRunner V2] Misc minor simplifications and optimizations (#33467 )

2026-02-01 22:17:14 +00:00

lora_utils.py

[ModelRunner V2] Misc minor simplifications and optimizations (#33467 )

2026-02-01 22:17:14 +00:00

model_runner.py

[Model Runner V2] Minor refactoring for penalties (#34662 )

2026-02-16 21:43:00 -08:00

pp_handler.py

Fix docs build warning (#34686 )

2026-02-17 02:31:40 -08:00

README.md

GPU Model Runner V2 (#25266 )

2025-11-21 08:20:55 -08:00

states.py

[Model Runner V2] support bad_words sampling param (#33433 )

2026-02-16 16:36:06 -08:00

structured_outputs.py

[ModelRunner V2] Misc minor simplifications and optimizations (#33467 )

2026-02-01 22:17:14 +00:00

README.md

[Experimental] Model Runner V2

This directory contains the new model runner which is under active development. Ping Woosuk Kwon for any changes.