Files

T

Lucas WilkinsonandBenjamin Chislett abe93bce59 [Attention] Make seq_lens_cpu optional in CommonAttentionMetadata to enable true async spec-decode (#29624 )

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>

2025-12-09 17:18:10 -08:00

metrics

[Model Runner V2] Support num NaNs in logits (#30187 )

2025-12-09 10:00:49 -08:00

sample

[Model Runner V2] Support num NaNs in logits (#30187 )

2025-12-09 10:00:49 -08:00

spec_decode

[Model Runner V2] Implement get_num_sampled_and_rejected kernel (#30029 )

2025-12-04 07:19:42 -08:00

__init__.py

GPU Model Runner V2 (#25266 )

2025-11-21 08:20:55 -08:00

async_utils.py

[Model Runner V2] Support num NaNs in logits (#30187 )

2025-12-09 10:00:49 -08:00

attn_utils.py

[Attention] Make seq_lens_cpu optional in CommonAttentionMetadata to enable true async spec-decode (#29624 )

2025-12-09 17:18:10 -08:00

block_table.py

[CI] Bug: Fix triton import issue (#29202 )

2025-11-21 17:14:49 -08:00

cudagraph_utils.py

[Model Runner V2] Refactor prefill token preparation (#29712 )

2025-11-28 19:49:17 -08:00

dp_utils.py

[Model Runner V2] Minor code cleanup (#29570 )

2025-11-26 19:47:17 -08:00

input_batch.py

[Model Runner V2] Implement get_num_sampled_and_rejected kernel (#30029 )

2025-12-04 07:19:42 -08:00

model_runner.py

[Model Runner V2] Support num NaNs in logits (#30187 )

2025-12-09 10:00:49 -08:00

README.md

GPU Model Runner V2 (#25266 )

2025-11-21 08:20:55 -08:00

states.py

[Model Runner V2] Support min-p sampling (#30171 )

2025-12-05 21:42:47 -08:00

structured_outputs.py

GPU Model Runner V2 (#25266 )

2025-11-21 08:20:55 -08:00

README.md

[Experimental] Model Runner V2

This directory contains the new model runner which is under active development. Ping Woosuk Kwon for any changes.