Varun Sundar RabindranathandVarun Sundar Rabindranath 7951d78738 [Core] Enable CUDA graphs for DP + All2All kernels (#18724) ...

Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

2025-05-28 22:55:30 +00:00

..

attention

[BugFix] FA2 MLA Accuracy Issue (#18807)

2025-05-28 08:59:39 +00:00

core

[BugFix][Spec Decode] Improve Prefix Caching Logic in Speculative Decoding (#18668)

2025-05-24 17:33:46 -07:00

engine

[V1][Metrics] Add API for accessing in-memory Prometheus metrics (#17010)

2025-05-27 09:37:06 +00:00

executor

[BugFix] Increase TP execute_model timeout (#18558)

2025-05-23 10:22:11 +08:00

metrics

[V1][Metrics] Add API for accessing in-memory Prometheus metrics (#17010)

2025-05-27 09:37:06 +00:00

sample

[V1][Sampler] Improve performance of FlashInfer sampling by sampling logits instead of probs (#18608)

2025-05-26 11:49:36 -04:00

spec_decode

[V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (2) (#18781)

2025-05-28 05:09:14 +00:00

structured_output

[Bugfix] Migrate to REGEX Library to prevent catastrophic backtracking (#18454)

2025-05-23 16:16:26 -07:00

worker

[Core] Enable CUDA graphs for DP + All2All kernels (#18724)

2025-05-28 22:55:30 +00:00

__init__.py

[V1] AsyncLLM Implementation (#9826)

2024-11-11 23:05:38 +00:00

kv_cache_interface.py

[v1] Redo "Support multiple KV cache groups in GPU model runner (#17945)" (#18593)

2025-05-23 09:39:47 -07:00

outputs.py

[P/D] NIXL Integration (#17751)

2025-05-12 09:46:16 -07:00

request.py

[Feature][V1]: suupports cached_tokens in response usage (#18149)

2025-05-23 01:41:03 -07:00

serial_utils.py

[Core] Improve Tensor serialisation (#18774)

2025-05-28 09:46:21 +08:00

utils.py

[V1] DP scale-out (2/N): Decouple engine process management and comms (#15977)

2025-05-13 10:48:21 -07:00