3 people 560ef78bfe [Perf][Model Runner V2] Compact sampling masks on GPU instead of unpacking the full-vocab bitmask on CPU (#54901) ...

Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: vx120 <893600387@qq.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: vx120 <893600387@qq.com>

2026-09-03 19:03:36 -07:00

..

cpu

[CPU] Ship triton-cpu wheel and fix several hardcoded pin_memory=True (#52092)

2026-08-13 14:40:53 +08:00

gpu

[Perf][Model Runner V2] Compact sampling masks on GPU instead of unpacking the full-vocab bitmask on CPU (#54901)

2026-09-03 19:03:36 -07:00

sentinel

[Feature] Add fault tolerance framework (simplified) for DP+EP external LB deployments (#44428)

2026-07-25 11:49:09 -04:00

__init__.py

[V1] Implement vLLM V1 [1/N] (#9289)

2024-10-22 01:24:07 -07:00

block_table.py

[3/N][Feat][Perf] Add new warmup infrastructure for JITs. Add provider registry and orchestration for JIT warmup (#50174)

2026-08-17 10:11:14 -07:00

cp_utils.py

[Core] Add MRV2 virtual-batch PCP for MLA (#46570)

2026-07-19 02:53:15 +00:00

cpu_model_runner.py

[Bugfix][MRV2] Isolate sleep-mode KV allocations (#53508)

2026-08-27 15:08:28 -07:00

cpu_worker.py

[CPU] Ship triton-cpu wheel and fix several hardcoded pin_memory=True (#52092)

2026-08-13 14:40:53 +08:00

dp_utils.py

[Core] Check for GPU<->CPU syncs during CI (#43107)

2026-08-14 17:42:05 -07:00

ec_connector_model_runner_mixin.py

[Refactor] Remove dead code in kv connector and model runner (#38383)

2026-03-31 17:05:23 -04:00

encoder_cudagraph_defs.py

[1/N] Unify multiple-path encoder cuda graph support (#49934)

2026-08-02 10:33:23 +08:00

encoder_cudagraph.py

[Bugfix][ROCm] Capture CUDA graphs on the current stream (#53818)

2026-08-26 23:16:45 +08:00

gpu_input_batch.py

[Pooling] Improve BGE-M3 sync pooling throughput by up to 3.13% (#53464)

2026-08-24 08:33:32 -04:00

gpu_model_runner.py

[Model] add GLM-5.3-Flash support (#53906)

2026-09-03 09:40:35 -07:00

gpu_ubatch_wrapper.py

[Bugfix] Fix UBatchWrapper CUDA graph key to sum all ubatches, not just first two (#43161)

2026-07-07 12:42:31 +00:00

gpu_worker.py

[Bugfix][Core] Wait for the previous PP tensor sends before the next forward pass (#54962)

2026-09-03 08:01:36 +00:00

kv_connector_model_runner_mixin.py

[Core][KV Connector] Start async KV loads after the forward launch when no sync loads are scheduled (#53333)

2026-08-28 21:58:00 +00:00

lora_model_runner_mixin.py

[Bugfix] Fix level-2 sleep/wake/reload with enable_lora=True (#39935)

2026-08-06 09:13:10 +08:00

mamba_utils.py

[K3 Bug] Fix Kimi-K3 RecoverSSM startup failure 'MambaAttentionBackendEnum.GDN_ATTN declares 4 states, but provides 2 state copy funcs' (#54634)

2026-08-31 23:22:49 +00:00

mm_encoder_model_runner.py

[Refactor][Model Runner V2][Multimodal] Move the encoder-only path out of the shared runner (#53176)

2026-08-22 05:11:44 +00:00

startup_plan.py

[Core] Persist and reuse the memory-profiling result across boots (opt-in) (#47388)

2026-07-08 00:53:02 +00:00

tpu_input_batch.py

fix: improve token_ids_cpu swap to copy only valid indices (#36255)

2026-08-28 17:22:39 -07:00

ubatch_utils.py

[BugFix] Preserve max_seq_len in ubatch metadata during CUDA graph capture (#40961)

2026-05-05 04:29:34 +00:00

ubatching.py

[Misc] Replace Optional[X] with X | None syntax (#33332)

2026-01-30 01:56:59 -08:00

utils.py

[Model] add GLM-5.3-Flash support (#53906)

2026-09-03 09:40:35 -07:00

worker_base.py

[Bugfix][DP] Synchronize the device on pause completion (#52914)

2026-08-26 09:50:33 -07:00

workspace.py

[Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark (#51538)

2026-08-15 07:01:32 -07:00

xpu_model_runner.py

[XPU] Alias is_current_stream_capturing to XPU in cuda wrapper (#50526)

2026-08-05 15:10:17 +08:00

xpu_worker.py

[Bugfix][XPU] Skip oneCCL warm-up all_reduce when world_size == 1 (#52389)

2026-08-23 21:31:33 -07:00