4 people 66dfee7121 [Bugfix] Fix degenerate KV cache stride causing TMA cudaErrorIllegalInstruction (#40737) ...

Signed-off-by: David Oy <david@baseten.co>
Signed-off-by: David Oy <58150256+the-david-oy@users.noreply.github.com>
Signed-off-by: David Oy <david.oy@baseten.co>
Co-authored-by: David Oy <david@baseten.co>
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>

2026-05-03 23:52:18 +00:00

..

__init__.py

[MoE Refactor] Remove SharedFusedMoE class (#35782)

2026-04-21 18:12:12 -04:00

argparse_utils.py

[Misc] Support Human-readable (k/K/m/M..) json cli arg (#40473)

2026-04-23 09:42:23 +02:00

async_utils.py

[Bugfix] Offload blocking tokenizer ops to shared thread pool to unblock event loop (#34789)

2026-03-26 22:17:00 -07:00

cache.py

[Docs] Mock all imports for docs (#27873)

2025-11-01 10:02:23 +00:00

collection_utils.py

[Plugin] Simplify IO Processor Plugin interface (#34236)

2026-02-10 19:47:39 -08:00

counter.py

[Misc] Separate out utils.counter and move utils.Device to engine (#27588)

2025-10-28 12:20:46 +00:00

cpu_resource_utils.py

[Bugfix] Fix degenerate KV cache stride causing TMA cudaErrorIllegalInstruction (#40737)

2026-05-03 23:52:18 +00:00

cpu_triton_utils.py

feat(cpu): add CPU support for draft model speculative decoding (#32662)

2026-04-10 11:49:52 +08:00

deep_gemm.py

[Feat] DeepSeek V4 Rebased (#40860)

2026-04-26 18:31:08 -07:00

flashinfer.py

[compile] Add FlashInfer FP8 async TP fusion and preserve allreduce fusion ordering #27893 (#39505)

2026-05-01 05:08:34 +00:00

func_utils.py

[ROCm][Engine] Fix GPU memory leaks in engine shutdown and test workaround for async KV prefix cache reset (#38503)

2026-04-25 05:25:20 +00:00

gc_utils.py

[Doc]: fixing typos in multiple files. (#29685)

2025-11-28 08:41:41 -08:00

hashing.py

[Core] Add xxHash as a high-performance hash option for accelerating prefix caching (#29163)

2025-12-03 16:06:57 +00:00

import_utils.py

[Refactor] Clean up log once scope="local" (#40540)

2026-04-22 16:42:43 -04:00

jsontree.py

[mypy] Pass type checking for vllm/utils and vllm/v1/pool (#29666)

2025-11-28 20:40:47 +08:00

math_utils.py

[Refactor] Remove unused utils (#38153)

2026-03-26 17:08:19 +08:00

mem_constants.py

[Tests] Add Qwen3-VL multimodal memory leak check (#39268)

2026-04-09 04:54:46 -07:00

mem_utils.py

[Bugfix] Use is_integrated to detect UMA GPUs for memory reporting (#35356)

2026-04-13 11:07:40 -07:00

mistral.py

[Feature] Avoid eager import of the "mistral_common" package. (#40043)

2026-04-24 02:49:16 +00:00

multi_stream_utils.py

[DSV4] Add knob to enable pre-attn gemm (#41443)

2026-05-01 12:23:17 -07:00

nccl.py

[mypy] Pass type checking for vllm/utils and vllm/v1/pool (#29666)

2025-11-28 20:40:47 +08:00

network_utils.py

[Fix] Add missing space in IP fallback warning (#40359)

2026-04-20 20:26:06 -07:00

numa_utils.py

[Feature] NUMA binding support for GPU workers (#38635)

2026-04-08 09:55:24 -07:00

numa_wrapper.sh

[Feature] NUMA binding support for GPU workers (#38635)

2026-04-08 09:55:24 -07:00

nvtx_pytorch_hooks.py

[Feature] Add Layer-wise NVTX Support (#29990)

2025-12-05 11:20:07 +00:00

ompmultiprocessing.py

[CPU] Refactor CPU affinity and memory management (#39781)

2026-04-17 21:01:08 +08:00

platform_utils.py

[Platform] Add current_platform.num_compute_units interface (#35042)

2026-02-24 22:22:49 -08:00

print_utils.py

[Examples][1/n] Resettle basic examples. (#35579)

2026-03-08 20:22:53 -07:00

profiling.py

[Deprecation] Deprecate cprofile and cprofile_context (#39100)

2026-04-21 11:25:22 +08:00

registry.py

[mypy] Pass type checking for vllm/utils and vllm/v1/pool (#29666)

2025-11-28 20:40:47 +08:00

serial_utils.py

[Frontend] Add multimodal support to /inference/v1/generate endpoint (#38405)

2026-04-17 20:31:56 -07:00

system_utils.py

[Feature] NUMA binding support for GPU workers (#38635)

2026-04-08 09:55:24 -07:00

tensor_schema.py

Update Optional[x] -> x | None and Union[x, y] to x | y (#26633)

2025-10-12 09:51:31 -07:00

torch_utils.py

[Bugfix] Fix degenerate KV cache stride causing TMA cudaErrorIllegalInstruction (#40737)

2026-05-03 23:52:18 +00:00

tqdm_utils.py

[Renderer] Move InputPreprocessor into Renderer (2/2) (#34560)

2026-02-17 05:29:01 -08:00