Tao He d69ff0cbbb Fixes assertion failure in prefix caching: the lora index mapping should respect prefix_len (#2688) ...

Signed-off-by: Tao He <sighingnow@gmail.com>

2024-01-31 18:00:13 +01:00

..

spec_decode

[Speculative decoding 2/9] Multi-step worker for draft model (#2424)

2024-01-21 16:31:47 -08:00

__init__.py

Change the name to vLLM (#150)

2023-06-17 03:07:40 -07:00

cache_engine.py

Support FP8-E5M2 KV Cache (#2279)

2024-01-28 16:43:54 -08:00

model_runner.py

Fixes assertion failure in prefix caching: the lora index mapping should respect prefix_len (#2688)

2024-01-31 18:00:13 +01:00

worker.py

Support FP8-E5M2 KV Cache (#2279)

2024-01-28 16:43:54 -08:00