[Mamba] Flashinfer selective_state_update (#36162)
Signed-off-by: Roi Koren <roik@nvidia.com>
This commit is contained in:
@@ -8,4 +8,5 @@ server_args: >-
|
||||
--max-model-len 4096
|
||||
--tensor-parallel-size 8
|
||||
--enable-expert-parallel
|
||||
--mamba-backend flashinfer
|
||||
--speculative-config '{"method":"mtp","num_speculative_tokens":5}'
|
||||
|
||||
@@ -8,4 +8,5 @@ server_args: >-
|
||||
--max-model-len 4096
|
||||
--tensor-parallel-size 2
|
||||
--enable-expert-parallel
|
||||
--mamba-backend flashinfer
|
||||
--speculative-config '{"method":"mtp","num_speculative_tokens":5}'
|
||||
|
||||
Reference in New Issue
Block a user