Itay EtelisandItay Etelis d63108fb18 [kv_offload] Skip decode-phase blocks in CPU offload (#43797) ...

Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>

2026-05-29 06:39:43 +03:00

..

attention

[Attention][AMD] Standardize kv layout to blocks first for AMD (#43660)

2026-05-28 12:28:50 -05:00

core

[EC Connector] Add shutdown API to EC Connector. (#42423)

2026-05-28 12:28:01 +00:00

engine

[Rust Frontend] Add /version endpoint using engine-reported value (#43854)

2026-05-29 00:32:27 +00:00

executor

[Feat] Add support for per GPU worker RDMA NIC selection (#42083)

2026-05-28 12:45:23 -07:00

kv_offload

[kv_offload] Skip decode-phase blocks in CPU offload (#43797)

2026-05-29 06:39:43 +03:00

metrics

[CPU] Migrate cpu_awq into awq_marlin (#43841)

2026-05-28 22:36:31 +08:00

pool

[Misc] Added missing return type annotations to improve mypy and IDE tooling (#43383)

2026-05-23 13:28:22 +08:00

sample

[ROCm] Enable the aiter top-k/top-p sampler by default (#43331)

2026-05-28 13:19:59 -05:00

simple_kv_offload

[Feat][KVConnector] Support DSV4 in SimpleCPUOffloadBackend (#42296)

2026-05-25 14:04:30 +08:00

spec_decode

[Model]Support Step-3.7-Flash (#43859)

2026-05-28 17:01:48 -07:00

structured_output

[Bug][Structured Outputs] Fix bug that leads to unconstrained generations with structural tags (#42452)

2026-05-20 07:08:58 -07:00

worker

Adjust design around encoder_cudagraph_forward (#42288)

2026-05-29 03:02:52 +00:00

__init__.py

[V1] AsyncLLM Implementation (#9826)

2024-11-11 23:05:38 +00:00

cudagraph_dispatcher.py

[Experimental] Breakable CUDA graph (#42304)

2026-05-16 22:04:12 +08:00

kv_cache_interface.py

[Bugfix][Hybrid][NemotronH] Fix mamba_cache_mode=all + speculative decoding crash (#41233)

2026-05-18 14:54:00 +03:00

outputs.py

[Core] Cleanup KVConnector handling with PP + fix MRV2 (#43732)

2026-05-28 13:12:03 -07:00

request.py

[bug] AsyncScheduler drops first post-resume token after pause_generation + clear_cache (#42117)

2026-05-19 01:06:21 -07:00

serial_utils.py

Add tensor IPC transfer mechanism for multimodal data (#32104)

2026-03-21 20:10:20 +00:00

utils.py

[Bug] Fix tests/distributed/test_elastic_ep.py - assert False (#43813)

2026-05-28 11:00:56 -04:00