Woosuk Kwon 2f77b6cfec [TPU] Implement prefix caching for TPUs (#10307) ...

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

2024-11-20 13:54:15 -08:00

..

__init__.py

…

cache_engine.py

…

cpu_embedding_model_runner.py

[Bugfix][CPU] Fix CPU embedding runner with tensor parallel (#10394)

2024-11-16 23:12:04 -08:00

cpu_enc_dec_model_runner.py

[Bugfix][Hardware][CPU] Fix broken encoder-decoder CPU runner (#10218)

2024-11-11 12:37:58 +00:00

cpu_model_runner.py

[Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355)

2024-11-20 10:57:39 +00:00

cpu_worker.py

…

embedding_model_runner.py

…

enc_dec_model_runner.py

[Platform][Refactor] Extract func get_default_attn_backend to Platform (#10358)

2024-11-19 11:22:26 +08:00

hpu_model_runner.py

[Hardware] [HPU]add mark_step for hpu (#10239)

2024-11-17 00:44:44 -08:00

hpu_worker.py

…

model_runner_base.py

…

model_runner.py

[5/N][torch.compile] torch.jit.script --> torch.compile (#10406)

2024-11-18 23:20:06 +08:00

multi_step_model_runner.py

…

multi_step_tpu_worker.py

…

multi_step_worker.py

…

neuron_model_runner.py

[1/N] Initial prototype for multi-modal processor (#10044)

2024-11-13 12:39:03 +00:00

neuron_worker.py

…

openvino_model_runner.py

[1/N] Initial prototype for multi-modal processor (#10044)

2024-11-13 12:39:03 +00:00

openvino_worker.py

…

tpu_model_runner.py

[TPU] Implement prefix caching for TPUs (#10307)

2024-11-20 13:54:15 -08:00

tpu_worker.py

[TPU] Implement prefix caching for TPUs (#10307)

2024-11-20 13:54:15 -08:00

utils.py

…

worker_base.py

…

worker.py

[7/N] torch.compile, reduce compilation time (#10460)

2024-11-20 11:20:38 -08:00

xpu_model_runner.py

[1/N] Initial prototype for multi-modal processor (#10044)

2024-11-13 12:39:03 +00:00

xpu_worker.py

…