MOVZX
ac305c44cb
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-09 17:41:13 +00:00
Yifan Qiao and Claude Opus 5.5
240785b82c
[Bugfix] Keep the GLM-5.3 kpool tail and Qwen4 QSA ring out of the null block ( #59528 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 20:44:04 +00:00
Stefano Castagnetta and Thien Tran
4ff028d77e
[Bugfix][Qwen4Exp] Honor --kv-cache-dtype-skip-layers in QSA attention ( #60023 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-05 12:13:30 +00:00
MOVZX
2877121b2d
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-03 21:59:03 +07:00
Shuolei Wang and Thien
c9578f10ab
[Qwen3.8-Flash-Next] Fuse main QK-norm/RoPE/gate and KV-cache write into the QSA pre-indexer launch ( #57097 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Thien <thien@inferact.ai >
Co-authored-by: Thien <thien@inferact.ai >
2026-10-01 14:21:06 +08:00
MOVZX and Hinata
a1d3c2cf1e
[Spec Decode] Fused draft metadata updates for the QSA side cache (PR #58449 )
...
Intake of open upstream PR vllm-project/vllm#58449 .
With MTP num_speculative_tokens > 1, the V2 autoregressive speculator
reuses one attention-metadata build across draft steps only when every
group's builder sets supports_draft_decode_metadata_update. The QSA
side-cache builder (QWEN4_EXP_EXP_QSA_STATE) did not, so every draft
step rebuilt attention metadata for the whole model.
QSAMetadataBuilder now implements update_draft_decode_metadata: it
re-launches the kernel with in-place seq_lens updates, so one metadata
build serves every draft step.
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 13:36:09 +07:00
a2be4d3cc7
[Bugfix][Qwen4Exp] Release the profiling KV cache held by QSA key views ( #58961 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 08:11:46 +00:00
bilikas and Claude Opus 5.5
31d78a21b9
[Spec Decode][Qwen3.8] Enable fused draft metadata updates for the QSA side cache
...
QSAMetadataBuilder did not set supports_draft_decode_metadata_update, so
MTP with num_speculative_tokens > 1 fell back to rebuilding attention
metadata between draft steps for the whole model.
Opt the builder in. update_draft_decode_metadata re-launches the builder's
own _build_qsa_metadata_kernel in place on the metadata's persistent
tensors, so draft-step token_to_req, logical_positions, visible_blocks,
slot_mapping and k_work_metadata come from the same code path as build().
The kernel launch is factored into _launch_qsa_metadata_kernel, and
QSAForwardMetadata carries common_slot_mapping and num_mapped_tokens so
the update does not read query_start_loc_cpu. The PyTorch fallback builder
keeps the rebuild path.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Signed-off-by: bilikas <valdas@vbtech.eu >
2026-09-23 20:58:12 +03:00
dff1bde84d
[Model] Qwen4Exp: fp8_e4m3 main KV cache on the QSA path ( #55557 )
...
Signed-off-by: Andrea Semeraro <andrea.semeraro@sezione1.it >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-09-16 15:07:52 +00:00
3f55ad2f07
[Qwen3.8-Flash-Next] Fuse PLE residual and QSA output gate ( #55309 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-09-14 12:16:25 +00:00
Thien Tran
94e26dd3dd
[Qwen3.8-Flash-Next] Support FP8 indexer cache for QSA ( #54890 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-07 12:15:45 +00:00
Thien Tran
d9105ea800
[Qwen3.8-Flash-Next] Remove torch.compile for NVIDIA implementation ( #55272 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-06 21:57:56 -07:00
Thien Tran and Kimi
31a8a26662
[Qwen3.8-Flash-Next] Improve QSA sparse GQA for prefill and short-ctx decode ( #54873 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-09-04 06:08:02 -07:00
Thien Tran
a5c9179e73
[Qwen3.8-Flash-Next] Compact indexer logits workspace to improve prefill efficiency ( #54915 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-04 00:43:20 -07:00
Thien Tran
003e34341a
[Qwen3.8-Flash-Next] Separate prefill and decode paths for QSA indexer ( #54513 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-02 09:08:10 +08:00
+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00