57 Commits
Author SHA1 Message Date
MOVZXandHinata 2066d3143f [Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
  rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
  select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
  quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
  requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
  fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
  dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
  mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
  cut deleted while keeping the model tree

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 17:10:56 +07:00
MOVZXandHinata 2b544b7ba0 [Mini-VLLM] Cut dead subsystems across serving, connectors, parsers
- Remove P/D transfer connectors: nixl, nixl_chat, mooncake x2, hf3fs,
  moriio, hisparse, example and hidden-states connectors
- Reduce weight_transfer to the base API used by model loading
- Remove kv_offload tiering, simple_kv_offload, ec cpu/example,
  snapshot command, engine snapshot server
- Remove transformers model fallback, dsv4 tokenizers, dead platforms
  cpu/tpu/xpu, flashmla third_party, dsa, megamoe/flydsl, dead attn
  ops, dead layers (short_conv, fused_embed_norm, indexer_topk,
  relu2 gemma_ar, lightning_attn, minimax_rms_norm), gemm_rs_ar
- Keep MoE LoRA op (imported eagerly by punica_gpu); drop fp8 variant
- Rebuild parser hubs around Qwen3 adapters only (36 dead adapters)
- Remove dead tests with the code they covered

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 02:19:03 +07:00
MOVZXandHinata 54a396586e [Mini-VLLM] Repair test suite after SM86 cut
- Restore example connector, test helpers, and dropped patch tests
- Guard MLA metadata import in the example connector
- Point the PLE test at common.ops.ple after upstream #60021 move
- Cut tests for removed features: tools, DRY, epd proxy, STT batch,
  diffusion scheduler, INC/Quark PLE cases, mamba-2 mixer

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 01:06:30 +07:00
MOVZXandHinata 81074e94a3 [Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 00:20:48 +07:00
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-18 19:19:56 +07:00
MOVZX be95edb7f6 Merge branch 'main' into patches
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-16 17:35:42 +07:00
Andreas KaratzasandClaude 29345f42a6 [ROCm][Qwen4Exp] Support HC and MoE sequence parallelism on ROCm (#61471)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-10-15 20:11:17 -05:00
MOVZX 76df77c04e Merge branch 'main' into patches
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-15 21:16:39 +07:00
Stefano CastagnettaandKartikey Vyas 4e0681ed56 [Quantization] Support packed NVFP4 Qwen4Exp PLE embeddings (#56273)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Kartikey Vyas <kartikey@artificialanalysis.ai>
2026-10-14 08:24:55 +08:00
Harry Huangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> c75cb63cd2 [Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next (#56322)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-12 20:56:44 +08:00
MOVZX ac305c44cb Merge branch 'main' into patches
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-09 17:41:13 +00:00
Mikko TukiainenandCursor 81c993658e [ROCm][Perf] Fuse main QK-norm/RoPE/gate and KV-cache write into the AMD QSA prepare launch (#59732)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-09 21:14:49 +08:00
Yifan QiaoandClaude Opus 5.5 240785b82c [Bugfix] Keep the GLM-5.3 kpool tail and Qwen4 QSA ring out of the null block (#59528)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 20:44:04 +00:00
Mikko TukiainenandCursor e2414ec1db [ROCm][Qwen4Exp] Use fused PLE Triton kernels on the AMD backend (#60021)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-08 15:47:56 +00:00
Mikko TukiainenandCursor d23af12cb1 [ROCm][Perf] Reach the fused QSA pre-indexer from the AMD path (#57947)
Signed-off-by: mjkvaak-amd <mjkvaak-amd@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-05 16:22:29 +00:00
Stefano CastagnettaandThien Tran 4ff028d77e [Bugfix][Qwen4Exp] Honor --kv-cache-dtype-skip-layers in QSA attention (#60023)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-10-05 12:13:30 +00:00
Mikko TukiainenCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>TJian
edca360f13 [Bugfix][Qwen4Exp] Load PLE tables unquantized under Quark checkpoints (#59443)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
2026-10-05 10:40:02 +00:00
MOVZX 4a3ebc6b5b Merge branch 'main' into patches 2026-10-05 08:21:46 +00:00
042ab0305c [Perf][Qwen4Exp] Merge QSA QKVG and indexer QK projections (#59533)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: namgyu-youn <namgyu.dev@gmail.com>
2026-10-05 07:03:29 +00:00
MOVZX f83ecfac16 Merge remote-tracking branch 'upstream/main' into patches 2026-10-05 08:52:31 +07:00
Vadim GimpelsonandClaude 4f52fa35ef [Bugfix] Fix Qwen4Exp PLE embedding rejecting INC (AutoRound) checkpoints (#59990)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-10-05 01:52:06 +00:00
MOVZX 2877121b2d Merge remote-tracking branch 'upstream/main' into patches
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-03 21:59:03 +07:00
Harry MellorandClaude Opus 5 e3cae8d2ac [Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor (#57387)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-02 02:25:18 +00:00
Shuolei WangandThien c9578f10ab [Qwen3.8-Flash-Next] Fuse main QK-norm/RoPE/gate and KV-cache write into the QSA pre-indexer launch (#57097)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Thien <thien@inferact.ai>
Co-authored-by: Thien <thien@inferact.ai>
2026-10-01 14:21:06 +08:00
Mathew Oddenandopencode+glm-5.3+vllm 91dab0eb7f [ROCm][Perf] Allocate the pinned PLE prefetch buffer lazily (#58797)
Signed-off-by: Mathew Odden <modden@redhat.com>
Co-authored-by: opencode+glm-5.3+vllm <opencode+glm-5.3+vllm@example.com>
2026-09-30 17:43:16 +08:00
MOVZXandHinata a1d3c2cf1e [Spec Decode] Fused draft metadata updates for the QSA side cache (PR #58449)
Intake of open upstream PR vllm-project/vllm#58449.

With MTP num_speculative_tokens > 1, the V2 autoregressive speculator
reuses one attention-metadata build across draft steps only when every
group's builder sets supports_draft_decode_metadata_update. The QSA
side-cache builder (QWEN4_EXP_EXP_QSA_STATE) did not, so every draft
step rebuilt attention metadata for the whole model.

QSAMetadataBuilder now implements update_draft_decode_metadata: it
re-launches the kernel with in-place seq_lens updates, so one metadata
build serves every draft step.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-09-29 13:36:09 +07:00
MOVZXandHinata cb275fdf20 [Qwen4Exp] Fix MTP buffer placement, config parsing, Triton warmup (PR #56742)
Intake of open upstream PR vllm-project/vllm#56742. Every rig here runs
MTP on both Qwen3.8 models, so these three bugs sit on our hot path.

The MTP hidden buffer was allocated with torch.empty and no device, so
it followed the ambient default instead of device_config.device; on a
CPU-default process the draft buffer lands on the wrong device. Both
NVIDIA and AMD backends now call make_mtp_hidden_buffer() in
vllm/models/qwen4_exp/common/mtp.py, which pins it to the configured
device.

Config parsing now accepts configs carrying M-RoPE keys and the
Transformers spelling of layer_types; both previously failed at load.

Triton warmup: Qwen4Exp was missing from the Qwen warmup dispatch, so
its Triton kernels compiled during the first served requests instead of
at startup.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-09-29 13:36:05 +07:00
MOVZX a63dca4afb Merge remote-tracking branch 'upstream/main' into patches
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-09-29 12:54:18 +07:00
Jiangyun ZhuandOpenAI Codex 20b52e9f5b [Perf][Qwen3.8] Reduce PLE metadata construction overhead (#58114)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-28 20:46:22 +08:00
Thien Tran d6cce94fd4 [Perf][Qwen4Exp] Fuse HC down projection and SiLU on NVIDIA (#58957)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-09-28 18:51:38 +08:00
Gabriel WuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a2be4d3cc7 [Bugfix][Qwen4Exp] Release the profiling KV cache held by QSA key views (#58961)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 08:11:46 +00:00
lesj0610 521235d6df Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
2026-09-27 18:25:21 +09:00
MOVZXandHinata 43fbd6510b [Qwen4Exp] Serve PLE rows from the loader's mapped checkpoint shards (PR #58815)
Adds an opt-in backend, --engram-config '{"host_file_gather": true}', that
serves Qwen4Exp PLE rows from the checkpoint files through the page cache
instead of allocating the 47.7 GiB table on GPU or in pinned host memory.

- Qwen4ExpPLEFileGatherEmbedding allocates no table rows; load_weights keeps
  the default loader's zero-copy safetensors views of each shard. At bind
  (post-load hook) every view must be file-backed per /proc/self/maps (HF
  cache symlinks work; heap/pinned/anonymous memory is rejected) and the
  shards must cover the whole table.
- Each step, Qwen4ExpModelState.prepare_inputs computes the PLE n-gram ids,
  copies them to pinned host memory, syncs the stream once, issues
  POSIX_FADV_WILLNEED over the coalesced page ranges of the step's rows,
  gathers the rows on the host, and copies them into a fixed device staging
  buffer. The forward reads only the staging buffer, so FULL CUDA graph
  capture is unaffected.
- Rows stay in the checkpoint dtype (FP8 table stays FP8, dequantized on the
  GPU); dummy loads serve zeros; weight reload is rejected; takes precedence
  over cpu_offload; rejected on ROCm and with elastic EP.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-09-27 09:54:42 +07:00
267eee54bf [Qwen4Exp][ROCm] PLE n-gram table CPU offload (#57497)
Signed-off-by: Mathew Odden <modden@redhat.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: opencode+deepseek-v4-flash+vllm <opencode+deepseek-v4-flash+vllm@example.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-24 23:28:20 -05:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
48d8880d09 [Bugfix][Qwen4Exp] Keep pinned PLE prefetch ids out of the CUDA graph pool (#58489)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 10:51:50 +08:00
fxmarty-amdmgoinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
90a9515006 [Cleanup] Remove online quantization support in fp8.py in favor of online shorthands (#53585)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-24 12:48:14 -07:00
bilikasandClaude Opus 5.5 31d78a21b9 [Spec Decode][Qwen3.8] Enable fused draft metadata updates for the QSA side cache
QSAMetadataBuilder did not set supports_draft_decode_metadata_update, so
MTP with num_speculative_tokens > 1 fell back to rebuilding attention
metadata between draft steps for the whole model.

Opt the builder in. update_draft_decode_metadata re-launches the builder's
own _build_qsa_metadata_kernel in place on the metadata's persistent
tensors, so draft-step token_to_req, logical_positions, visible_blocks,
slot_mapping and k_work_metadata come from the same code path as build().
The kernel launch is factored into _launch_qsa_metadata_kernel, and
QSAForwardMetadata carries common_slot_mapping and num_mapped_tokens so
the update does not read query_start_loc_cpu. The PyTorch fallback builder
keeps the rebuild path.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: bilikas <valdas@vbtech.eu>
2026-09-23 20:58:12 +03:00
lesj0610 6c5552776b Merge remote-tracking branch 'origin/main' into lesj/qwen4exp-bugfixes
Both sides edited the same import block in the Qwen4Exp NVIDIA and AMD
models and nothing else collided. This branch adds
`from ..common.mtp import make_mtp_hidden_buffer`, the helper the MTP
buffer extraction calls; upstream ec4a3a5370 widened the `..config`
import to `ATTENTION_LAYER_TYPES, QSA_LAYER_TYPE, Qwen4ExpConfig` for the
layer-type constants it introduced. The two changes are independent, so
the resolution keeps both lines. Every symbol still has a use outside the
import in both files.

Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
2026-09-17 08:16:59 +09:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
semerandreClaudemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Jiangyun Zhu
dff1bde84d [Model] Qwen4Exp: fp8_e4m3 main KV cache on the QSA path (#55557)
Signed-off-by: Andrea Semeraro <andrea.semeraro@sezione1.it>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
2026-09-16 15:07:52 +00:00
lesj0610 21d2dd56d8 Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
2026-09-16 13:41:12 +09:00
3f55ad2f07 [Qwen3.8-Flash-Next] Fuse PLE residual and QSA output gate (#55309)
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
2026-09-14 12:16:25 +00:00
lesj0610andClaude Opus 5 3b1911c0ca [Bugfix][Model] Fix Qwen4Exp MTP buffer placement, config parsing, and Triton warmup
Three initialization and config defects found serving Qwen4Exp, each with a
regression test.

**The MTP hidden buffer lands on the ambient device.** torch.empty for the
spec-decode hidden buffer takes no device, so it follows whatever device is
current rather than vllm_config.device_config.device. The allocation was
duplicated verbatim in the NVIDIA and AMD backends; it moves to
common/mtp.py so the two cannot drift, and the caller passes is_last_rank
rather than reaching for the process group from shared code.

**M-RoPE keys are reported as unrecognized.** Qwen4Exp drives M-RoPE through
mrope_section and mrope_interleaved, which transformers' rope validator does
not know for rope_type="default", so every load warns about a configuration
the model genuinely uses. Declare them at rope validation on the text config
and on the outer config that mirrors it; every other key still warns.

**transformers' layer_types spelling is unread.** Its Qwen4Exp config
canonicalizes "full_attention" entries to "qwen_sparse_attention" in memory,
so checkpoints re-exported through it serialize that name. vLLM keys QSA off
"full_attention" plus indexer_n_heads, so map the spelling back.

**Qwen4Exp is missing from the Triton warmup set.** Its GDN prefill and QSA
run on other backends, but the kernels this warmup covers -- the gated RMS
norm, the causal conv, the post-conv preparation, the decode state update --
are Triton whichever prefill kernel is chosen, so they compile on the first
prompt instead. qwen4_exp_mtp stays out: kernel_warmup passes the worker's
own model_config, which is the target model's.

tests/models/qwen4_exp/test_config.py: 15 passed.
tests/model_executor/test_qwen_triton_warmup.py: 6 passed.
ruff check, ruff format --check and mypy (Python 3.10) report what they
already report on the base commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
2026-09-14 07:25:01 +09:00
Harry Huangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 3116c5d06b [Qwen4Exp] Support UVA PLE-offload and Engram tensor parallelism (#54371)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-09 07:32:34 -07:00
sychen52mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Isotr0py
60ad959b6f Fix block FP8 MTP in ModelOpt mixed checkpoints (#55513)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-09-08 22:47:21 +00:00
Thien Tran 94e26dd3dd [Qwen3.8-Flash-Next] Support FP8 indexer cache for QSA (#54890)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-09-07 12:15:45 +00:00
Thien Tran d9105ea800 [Qwen3.8-Flash-Next] Remove torch.compile for NVIDIA implementation (#55272)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-09-06 21:57:56 -07:00
Harry Huang 28e605fb33 [Bugfix][Qwen4Exp] fix state index strides in fused PLE conv (#55375)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
2026-09-05 14:02:20 +00:00
Thien TranandKimi 31a8a26662 [Qwen3.8-Flash-Next] Improve QSA sparse GQA for prefill and short-ctx decode (#54873)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi <noreply@moonshot.cn>
2026-09-04 06:08:02 -07:00