MOVZXandHinata 754fb3e29d [TopK] Exact restart of the medium path on candidate-buffer overflow ...

When the persistent top-k medium-path candidate buffer overflows, the row is
restarted exactly instead of being truncated. Capture-safe counters with atomic
publication and a coherent snapshot record every restart.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>

2026-10-09 19:43:58 +07:00

..

attention

[Kimi-K3] DCP support (#50484)

2026-08-10 12:54:26 -07:00

core

[12/n] final _C library kernel migration (#45415)

2026-06-19 06:57:26 -07:00

cutlass_extensions

New stable abi cleanup (#46656)

2026-07-03 14:02:26 +08:00

gdn

[Model] Support Qwen3.8-Flash-Next (#53896)

2026-08-31 05:57:56 +00:00

kimi_k3

[ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill reland (#54038)

2026-09-10 15:37:54 +08:00

mamba

[8/n] Migrate merge_attn_states, mamba, sampler to torch stable ABI (continued) (#43361)

2026-05-27 09:35:24 -07:00

moe

[Refactor] Remove dead kernel code (#57621)

2026-09-20 10:39:23 -07:00

quantization

[Perf][Kernel] Vectorized flat abs-max for dynamic per-tensor FP8 quantization (#58194)

2026-09-25 09:19:48 -06:00

activation_kernels.cu

[MoE] Generalize masked activation for padded layouts (#51217)

2026-09-01 11:34:13 -04:00

async_util.cuh

Migrate header files to torch stable abi (#44013)

2026-06-02 08:09:52 -07:00

cache_kernels_fused.cu

[9/n] Migrate attention and cache kernels to torch stable ABI (continued) (#43717)

2026-05-29 04:44:45 +00:00

cache_kernels.cu

[Bugfix][SM120][MLA] Support NoPE sparse MLA (GLM-5.3-Flash) on the FlashInfer SM120 backend (#55277)

2026-09-23 02:03:20 -07:00

concat_mla_q.cuh

[ROCm][MLA] Replace torch.cat in sparse-MLA forward_mqa with fused concat_mla_q (#42838)

2026-06-06 18:20:50 -05:00

cooperative_topk.cu

[Perf][Kernel] Tune cooperative topk for medium batch-sizes (#53382)

2026-08-31 13:44:38 -07:00

cooperative_topk.cuh

[Perf][Kernel] Tune cooperative topk for medium batch-sizes (#53382)

2026-08-31 13:44:38 -07:00

cub_helpers.h

New stable abi cleanup (#46656)

2026-07-03 14:02:26 +08:00

cuda_utils_kernels.cu

[10/n] Migrate cuda_view and silu_and_mul_per_block_quant kernels to torch stale ABI. (#44334)

2026-06-04 20:14:43 -07:00

cuda_vec_utils.cuh

[Refactor] Remove dead kernel code (#57621)

2026-09-20 10:39:23 -07:00

cuda_view.cu

[10/n] Migrate cuda_view and silu_and_mul_per_block_quant kernels to torch stale ABI. (#44334)

2026-06-04 20:14:43 -07:00

custom_all_gather_reduce_scatter_ops.cpp

[Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 (#55072)

2026-09-25 03:27:21 +00:00

custom_all_gather_reduce_scatter.cu

[Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 (#55072)

2026-09-25 03:27:21 +00:00

custom_all_reduce.cu

[Model] Add Kimi K3 support: model files and kernels [1/N] (#50089)

2026-07-29 14:10:58 +08:00

dispatch_utils.h

add pad-aware reduce path (#48385)

2026-07-14 18:05:50 -07:00

dsv3_fused_a_gemm.cu

[K3 Perf] Enable DSV3 GEMM for inner-contiguous and row-strided tensors, 12%~81% kernel performance improvement (#54565)

2026-09-02 12:22:40 -07:00

fp32_router_gemm_entry.cu

[Kernel] Retire the DSv3 router GEMM CUDA kernel (#54040)

2026-08-31 07:15:31 +00:00

fp32_router_gemm.cu

[Kernel] Retire the DSv3 router GEMM CUDA kernel (#54040)

2026-08-31 07:15:31 +00:00

fused_deepseek_v4_qnorm_rope_kv_insert_kernel.cu

[Model][DSv4.1] FlashMLA mega attention and the NVFP4 compressed KV cache (#56935)

2026-09-16 03:07:50 -07:00

fused_kimi_k3_mla_key_concat_kv_cache_kernel.cu

[Bugfix] Fix SM100 fp8_ds_mla cache scales (#49435)

2026-09-19 17:15:07 -07:00

fused_minimax_m3_qknorm_rope_kv_insert_kernel.cu

[Attention][MiniMax-M3] NVFP4 KV cache on the MSA sparse attention path (#59300)

2026-10-01 14:39:55 -07:00

fused_qknorm_rope_kernel.cu

[Kernel] PDL enablement for fusedQKNormRopeKernel (#55755)

2026-09-07 20:49:03 -07:00

hisparse_kernels.cu

[Bugfix][HiSparse] Resolve MTP verification rows with a union residency kernel (#59235)

2026-09-30 03:01:45 +00:00

launch_bounds_utils.h

Migrate header files to torch stable abi (#44013)

2026-06-02 08:09:52 -07:00

layernorm_kernels.cu

[Perf] RMSNorm uncontiguous support, 1.2~3.1x kernel performance improvement (#49750)

2026-07-29 13:45:38 -07:00

layernorm_quant_kernels.cu

[Bugfix][Kernel] Fix batch invariance in RMSNorm kernels by pinning block size (#48391)

2026-07-28 22:24:02 +08:00

minimax_reduce_rms_kernel.cu

[Misc] Use functions instead of PTX for the PDL instruction (#46984)

2026-07-01 19:38:35 -07:00

minimax_reduce_rms_kernel.h

New stable abi cleanup (#46656)

2026-07-03 14:02:26 +08:00

ngram_embedding_kernels.cu

[Model] Add LongCat-Flash-Lite (n-gram embedding) (#47857)

2026-07-10 07:17:50 -07:00

nvfp4_ds_mla_cache_kernels.cu

[Bugfix][DSA] Write nvfp4_ds_mla from the fused norm+rope kernel (#55538)

2026-09-15 00:25:33 +04:00

nvfp4_ds_mla_cache.h

[Attention][DSA] Enable W4A16 DSA (#51724)

2026-09-01 12:37:47 +08:00

nvfp4_kv_cache_kernels.cu

Add NVFP4 KV 4-over-6 scale search (#45187)

2026-08-07 13:22:27 -07:00

ops.h

[TopK] Exact restart of the medium path on candidate-buffer overflow

2026-10-09 19:43:58 +07:00

persistent_topk.cuh

[TopK] Exact restart of the medium path on candidate-buffer overflow

2026-10-09 19:43:58 +07:00

pos_encoding_kernels.cu

[7/n] Migrate pos_encoding and norm kernels to libtorch stable ABI (continued) (#43209)

2026-05-23 13:20:00 +08:00

sampled_topk.cuh

[Perf][Kernel] Add sampled filtering for persistent top-k (#56346)

2026-09-15 14:15:59 -07:00

sampler.cu

[Refactor] Remove dead kernel code (#57621)

2026-09-20 10:39:23 -07:00

topk_histogram_4096.cuh

[CI/Build] Fix topk histogram build on SM75 (#46550)

2026-06-24 00:51:11 -07:00

topk.cu

[TopK] Exact restart of the medium path on candidate-buffer overflow

2026-10-09 19:43:58 +07:00

torch_bindings.cpp

[TopK] Exact restart of the medium path on candidate-buffer overflow

2026-10-09 19:43:58 +07:00

torch_utils.h

Enable perf_token_group_quant/_C_stable_libtorch for ROCm (#42758)

2026-06-02 23:23:28 -07:00

type_convert.cuh

New stable abi cleanup (#46656)

2026-07-03 14:02:26 +08:00

vocab_parallel_embedding_kernels.cu

[kernel] Fused embedding kernel (#53677)

2026-08-31 11:34:14 +00:00