Roy f1c0fc3919 Migrate logits computation and gather to model_runner (#3233)

2024-03-20 23:25:01 +00:00

..

attention

[1/n][Chunked Prefill] Refactor input query shapes (#3236)

2024-03-20 14:46:05 -07:00

fused_moe

[Kernel] change benchmark script so that result can be directly used; tune moe kernel in A100/H100 with tp=2,4,8 (#3389)

2024-03-14 08:11:48 +00:00

ops

[1/n] Triton sampling kernel (#3186)

2024-03-20 14:45:08 -07:00

quantization

fix marlin config repr (#3414)

2024-03-14 16:26:19 -07:00

__init__.py

Change the name to vLLM (#150)

2023-06-17 03:07:40 -07:00

activation.py

[1/n][Chunked Prefill] Refactor input query shapes (#3236)

2024-03-20 14:46:05 -07:00

layernorm.py

Revert "Refactor llama family models (#2637)" (#2851)

2024-02-13 09:24:59 -08:00

linear.py

[Minor] Fix bias in if to remove ambiguity (#3259)

2024-03-13 09:16:55 -07:00

logits_processor.py

Migrate logits computation and gather to model_runner (#3233)

2024-03-20 23:25:01 +00:00

rejection_sampler.py

[Speculative decoding 3/9] Worker which speculates, scores, and applies rejection sampling (#3103)

2024-03-08 23:32:46 -08:00

rotary_embedding.py

Fix lint (#3388)

2024-03-13 13:56:49 -07:00

sampler.py

Migrate logits computation and gather to model_runner (#3233)

2024-03-20 23:25:01 +00:00

vocab_parallel_embedding.py

Remove hardcoded device="cuda" to support more devices (#2503)

2024-02-01 15:46:39 -08:00