[Docs] Fix stale W4A16 NVFP4 default kernel in ModelOpt docs (#59135)
Signed-off-by: Rajeev Warrier <warrier.rajeev.95@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
d648847fe4
commit
30e956fad3
@@ -33,9 +33,7 @@ following `quantization.quant_algo` values:
|
||||
`flashinfer_cutlass`, `flashinfer_cutedsl`, `flashinfer_trtllm`,
|
||||
`flashinfer_cudnn`, and `marlin`; see `KernelConfig` on the
|
||||
[Engine Arguments](../../configuration/engine_args.md) page and shown by
|
||||
`vllm serve --help=KernelConfig`. For `W4A16_NVFP4`, `auto` currently
|
||||
selects Marlin. Models with BF16 activations can explicitly select the
|
||||
FlashInfer CuTe-DSL backend with `--linear-backend flashinfer_cutedsl`.
|
||||
`vllm serve --help=KernelConfig`.
|
||||
|
||||
!!! note
|
||||
For models quantized to MXFP8 with BF16 activations on SM100-family GPUs,
|
||||
|
||||
Reference in New Issue
Block a user