3 people de7d70676d [Perf] Low-latency CPU all-reduce for DP metadata syncs (#61767) ...

Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: hjjq <50634613+hjjq@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-10-15 15:29:13 -07:00

..

__init__.py

[V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (#10681)

2024-12-26 19:02:58 +09:00

test_attn_utils.py

[Perf][HiSparse] Reuse pinned host pool after graph profiling (#61668)

2026-10-15 12:33:03 -07:00

test_cp_utils.py

[PCP][DCP] Support DCP target model with non-DCP Dspark (#56723)

2026-09-26 18:12:35 -04:00

test_cpu_worker_kv_budget.py

[Bugfix][CPU] Report the real cause when the worker outgrows its memory reservation (#59472)

2026-10-15 21:52:05 +08:00

test_dspark_utils.py

[EPLB] Add EPLB support to dspark draft models (DeepSeekV4) (#57549)

2026-10-12 10:03:09 -04:00

test_dsv4_packed_zeroer_geometry.py

[Core][Refactor] Move KV cache planning logic to kv_cache_planning.py (#61183)

2026-10-13 03:05:39 +00:00

test_eagle3_aux_hidden_states_pp.py

[Bugfix][MRV2][Spec Decode] Honor EAGLE3 single-state draft configuration (#61563)

2026-10-14 12:09:06 -07:00

test_encoder_runner.py

[Perf][Multimodal] Sort multimodal encoder inputs by modality (#60196)

2026-10-09 10:55:36 +08:00

test_gpu_bad_words.py

[Model Runner V2] Support custom logits processors (#56497)

2026-09-19 17:46:49 +00:00

test_gpu_batch_ordering.py

[Perf][PP] Skip sampled-token broadcasts whose requests leave the engine (#58542)

2026-09-30 19:18:55 -07:00

test_gpu_batch_shard.py

[Perf][MRV2] Allow FULL decode graphs for one-token prompt tails (#58400)

2026-09-28 12:48:52 -07:00

test_gpu_block_table.py

[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169)

2026-10-09 15:48:37 -04:00

test_gpu_draft_lm_head.py

[MRV2][Spec Decode] Quantized draft lm_head for drafters that share the target head (#59973)

2026-10-15 22:06:40 +00:00

test_gpu_extract_hidden_states_speculator.py

[ModelRunner V2] Speculative Decoding NGram GPU Implementations (#40704)

2026-10-06 13:32:59 -07:00

test_gpu_gumbel_sample.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

test_gpu_input_batch_v2.py

[MRV2] Miscellaneous code cleanup (#57980)

2026-09-23 01:51:52 -07:00

test_gpu_input_batch.py

[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169)

2026-10-09 15:48:37 -04:00

test_gpu_kpool_tail_slot_mapping.py

[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169)

2026-10-09 15:48:37 -04:00

test_gpu_kv_connector.py

[MRV2] Miscellaneous code structure updates (#60754)

2026-10-09 17:58:09 -07:00

test_gpu_logit_bias.py

[Model Runner V2] Support custom logits processors (#56497)

2026-09-19 17:46:49 +00:00

test_gpu_logits_processors.py

[Model Runner V2] Support custom logits processors (#56497)

2026-09-19 17:46:49 +00:00

test_gpu_model_runner_mm_gather.py

[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)

2026-09-16 10:57:24 -07:00

test_gpu_model_runner_prefill_tail_classification.py

[Attention] Remove cascade attention (#61126)

2026-10-12 10:41:07 -07:00

test_gpu_model_runner_v2_cudagraph_profiling.py

[MRV2] Keep warmup out of the EPLB load stats (#56716)

2026-10-14 00:00:48 +00:00

test_gpu_model_runner_v2_eplb.py

[EPLB] Add EPLB support to dspark draft models (DeepSeekV4) (#57549)

2026-10-12 10:03:09 -04:00

test_gpu_model_runner_v2.py

[Bug] Honor spec decode draft model config (#61240)

2026-10-12 15:13:23 -04:00

test_gpu_model_runner.py

[Perf][GDN] Internal prefill checkpoints for the FlashInfer GDN prefill backend (#60050)

2026-10-14 14:49:46 +08:00

test_gpu_pcp_manager.py

[Bugfix][PCP] Align MoE routing padding with gathered tokens (#60946)

2026-10-14 21:04:00 -04:00

test_gpu_profiler.py

[Metrics] Expose cached prompt tokens by cache tier (#56318)

2026-10-06 14:52:05 -04:00

test_gpu_rejection_sampler_chunking.py

[Feature][Spec Decode] Support sampling mask replay for MRV2 MTP (#59359)

2026-10-01 13:51:39 +08:00

test_gpu_rejection_sampler_i64.py

[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)

2026-09-16 10:57:24 -07:00

test_gpu_row_offsets_i64.py

[Bugfix][MRV2] Use int64 index mappings to fix row-offset overflow (#60736)

2026-10-09 14:38:11 -07:00

test_gpu_sampler_flags.py

[Model Runner V2] Support custom logits processors (#56497)

2026-09-19 17:46:49 +00:00

test_gpu_target_dependent_ar_speculator.py

[Bugfix][PCP] Align MoE routing padding with gathered tokens (#60946)

2026-10-14 21:04:00 -04:00

test_gpu_thinking_budget.py

[Model Runner V2] Support custom logits processors (#56497)

2026-09-19 17:46:49 +00:00

test_gpu_trace_replay.py

[Core][V1] Support trace_decode_token_ids for deterministic decode replay (#46701)

2026-08-20 15:12:43 +08:00

test_gpu_ubatch_slicing.py

[Perf] Low-latency CPU all-reduce for DP metadata syncs (#61767)

2026-10-15 15:29:13 -07:00

test_gpu_warmup_blocks.py

[Spec Decode] Remove Medusa and MLP speculator support (#61701)

2026-10-14 17:29:23 -07:00

test_gpu_worker_weight_transfer.py

[MRV2][Spec Decode] Quantized draft lm_head for drafters that share the target head (#59973)

2026-10-15 22:06:40 +00:00

test_gpu_worker.py

[MRV2] Switch CI tests still running on MRV1 unnecessarily (#61014)

2026-10-11 09:57:07 -07:00

test_grammar_invalid_drafts.py

[Bugfix] Don't let a structured-output request sample from an unmasked row (#54442)

2026-10-01 02:11:44 +00:00

test_jit_warmup_migration.py

[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169)

2026-10-09 15:48:37 -04:00

test_kv_block_zeroer.py

[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)

2026-09-16 10:57:24 -07:00

test_kv_cache_allocation_scope.py

[Perf][HiSparse] Reuse pinned host pool after graph profiling (#61668)

2026-10-15 12:33:03 -07:00

test_late_interaction_runner.py

[Bugfix][Pooling] Handle multimodal cache misses without crashing (#58975)

2026-10-02 10:14:33 -04:00

test_logits_processor_loader.py

[Model Runner V2] Support custom logits processors (#56497)

2026-09-19 17:46:49 +00:00

test_mamba_hybrid_model_state.py

[Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries (#59962)

2026-10-12 02:08:14 +00:00

test_mamba_utils.py

[Attention] Remove cascade attention (#61126)

2026-10-12 10:41:07 -07:00

test_mixed_warmup_gate.py

[Spec Decode][MRV2] Warm up top-k/top-p and greedy rejection kernels for speculative decoding (#59260)

2026-10-14 14:59:21 -04:00

test_mrope_prompt_embeds.py

[Security] Fix DoS via prompt_embeds on M-RoPE models (#45252)

2026-06-13 10:17:38 +00:00

test_pp_utils.py

[Perf][PP] Skip sampled-token broadcasts whose requests leave the engine (#58542)

2026-09-30 19:18:55 -07:00

test_prompt_embeds_state.py

[XPU][CI] enable prompt embeds tests on XPU (#58283)

2026-09-25 06:56:59 +08:00

test_sleep_mode_backend.py

[CI] Split (H200 MIG/MI300) Basic Correctness into named jobs (#57054)

2026-09-25 17:05:33 -07:00

test_spec_decode_embed_sharing_pp.py

[Bugfix][MiniMax M3] Share target embeddings with MTP under PP (#58648)

2026-09-29 20:04:10 -07:00

test_ubatch_offsets.py

[ROCm][Perf] Enable DPA+ETP prefill compute/comm dual-batch-overlap for DeepSeek-V4 (#57773)

2026-10-15 11:16:48 +08:00

test_utils.py

[Perf][HiSparse] Reuse pinned host pool after graph profiling (#61668)

2026-10-15 12:33:03 -07:00

test_worker_base.py

[Core] Add lifecycle hooks for worker extensions (#59463)

2026-10-13 21:51:05 +08:00

test_worker_memory_snapshot.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

test_workspace.py

[Core][Kernel] Share persistent workspaces for Marlin and Humming (#57421)

2026-09-20 17:28:33 +08:00

test_xpu_model_runner.py

[XPU] Fix CUDA API shims breaking Torch Dynamo during AOT compile (#43092)

2026-07-06 08:29:20 +08:00