[Mamba] Flashinfer selective_state_update (#36162)

Signed-off-by: Roi Koren <roik@nvidia.com>
This commit is contained in:
roikoren755
2026-04-14 15:10:58 -04:00
committed by GitHub
parent 30679319e8
commit ecf5ff7ce3
15 changed files with 516 additions and 71 deletions
@@ -8,4 +8,5 @@ server_args: >-
--max-model-len 4096
--tensor-parallel-size 8
--enable-expert-parallel
--mamba-backend flashinfer
--speculative-config '{"method":"mtp","num_speculative_tokens":5}'
@@ -8,4 +8,5 @@ server_args: >-
--max-model-len 4096
--tensor-parallel-size 2
--enable-expert-parallel
--mamba-backend flashinfer
--speculative-config '{"method":"mtp","num_speculative_tokens":5}'