vr8vr8
848dbf89ec
Merge branch 'main' into qwen4exp-qsa-fused-draft-metadata
2026-09-23 23:37:13 +03:00
bilikas and Claude Opus 5.5
31d78a21b9
[Spec Decode][Qwen3.8] Enable fused draft metadata updates for the QSA side cache
...
QSAMetadataBuilder did not set supports_draft_decode_metadata_update, so
MTP with num_speculative_tokens > 1 fell back to rebuilding attention
metadata between draft steps for the whole model.
Opt the builder in. update_draft_decode_metadata re-launches the builder's
own _build_qsa_metadata_kernel in place on the metadata's persistent
tensors, so draft-step token_to_req, logical_positions, visible_blocks,
slot_mapping and k_work_metadata come from the same code path as build().
The kernel launch is factored into _launch_qsa_metadata_kernel, and
QSAForwardMetadata carries common_slot_mapping and num_mapped_tokens so
the update does not read query_start_loc_cpu. The PyTorch fallback builder
keeps the rebuild path.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Signed-off-by: bilikas <valdas@vbtech.eu >
2026-09-23 20:58:12 +03:00
c4cfd7007b
[CI][ROCm] Add an MI355 Kimi-K3 unit test group ( #58012 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 11:25:51 -05:00
Tianyu Guo and Cyrus Leung
3ee2906f47
[EPD] Support metadata-only audio inputs ( #57887 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-23 09:03:50 +08:00
c9b34fdb2d
[Bugfix][GLM-5.3-Flash] Run the dense MLP layers on the sequence-parallel shard ( #58061 )
...
Signed-off-by: Jared Wen <w13431838023@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-22 21:58:21 +08:00
Ting SUN and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a33b3bac5d
[Bugfix][Qwen2.5-VL] Honor video fps for temporal M-RoPE ( #47736 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-22 05:56:12 +00:00
Shanshan Shen
42a85a4976
[ROCm][Bugfix] Explicitly reject FSE=1 with DPA+ETP deployment for DeepSeek-V4 ( #57919 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-22 04:45:53 +00:00
13058faa49
[Frontend] Add reusable TP1 initialized-engine snapshots ( #51360 )
...
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com >
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-21 23:18:49 +00:00
382970ee6c
[Kimi-K3][AMD] Return KDA and MLA projection outputs directly ( #50592 )
...
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com >
Co-authored-by: Liuyinfeng01 <199041580+LiuYinfeng01@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-21 15:44:41 -05:00
djramic and Andreas Karatzas
4f14516790
[ROCm][CI] Use ROCm backend for DeepSeek V4.1 ViT test ( #57931 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-21 16:39:31 +00:00
Cyrus Leung
7853700d4c
[MM] Move get_dummy_processor_inputs into MM processor ( #57967 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-21 09:39:20 -07:00
Sahil Patel and Cyrus Leung
4e61ad8ee1
[Bugfix] unskip InternViT test for transformers v5 compatibility ( #55767 )
...
Signed-off-by: sahil <sahil@example.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-21 09:38:25 -07:00
Tuukka Sarvi and larryli2-amd
e34685dfc0
[ROCm][DSv4] Enable DSpark adaptive verification ( #52362 )
...
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com >
Signed-off-by: larryli2-amd <larryli2@amd.com >
Co-authored-by: larryli2-amd <larryli2@amd.com >
2026-09-21 10:03:41 -05:00
Cyrus Leung
c8824a1e42
[MM] Move supports_multimodal_inputs and cache out of registry ( #57913 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-21 07:43:07 -07:00
Andreas Karatzas and OpenAI Codex
8902dbb177
[CI][ROCm] Mirror remaining portable suites without duplicate coverage ( #57877 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-21 04:16:26 -05:00
akii96
0ff0477ff9
[Perf][ROCm] Add a ROCm path for Hy4 and compile the backbone ( #57526 )
...
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com >
2026-09-21 08:20:43 +00:00
Oliver Holworthy and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
9598487bc1
[Bugfix][Model] Mistral3: align placeholder grid with public processor ( #53758 )
...
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-21 06:36:59 +00:00
Tianyao Wu
59f1527280
[Bugfix] Fix Whisper engine crash on audio clips longer than 30s ( #57769 )
...
Signed-off-by: Tianyao Wu <rayroy31@gmail.com >
2026-09-21 05:44:40 +00:00
Andreas Karatzas and OpenAI Codex
8ec00ec2bc
[Tests] Release HF embedding weights after prompt extraction ( #57868 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-21 05:27:29 +00:00
Andreas Karatzas and OpenAI Codex
76ee20c1ec
[ROCm][Tests] Avoid pooling cleanup waits against live shared engines ( #57864 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-21 05:21:35 +00:00
Chauncey and zjy0516
3df4ae153e
[Core][KDA] Generalize Mamba prefill checkpoint builder and exporter ( #57783 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-09-21 04:46:11 +00:00
Isotr0py
27757dde02
[DSV4.1] Add encoder cuda graph support for deepseek-v4.1-flash ( #56625 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-20 21:51:13 +08:00
Chauncey
bf01fc4a31
[GLM-5.3-Flash] Route kpool indexer top-k through the shared SparseIndexerTopk dispatcher ( #57546 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-09-20 11:17:59 +00:00
yinfengLiu and Shanshan Shen
0e110f696d
[ROCm][DSV4][Perf] Use FP8 WO_A output projection ( #54894 )
...
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-20 09:13:55 +00:00
Cyrus Leung
751f6807d9
[Refactor] Use dynamic MM cache in processor ( #57674 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-19 15:38:00 +00:00
vllmellm
211e252d0b
[Bugfix][Model] Fix MiMo-V2.5 fused fp8 qkv_proj sharding (pre-shard count is num_key_value_heads; MTP path too) ( #57508 )
...
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com >
2026-09-19 06:05:00 -07:00
Jiang Wu and Jiang Wu
01c7bf8813
[Bugfix][Model] Fix Aria expert weight names and layout ( #57487 )
...
Signed-off-by: Jiang Wu <245355730+JiangLLM@users.noreply.github.com >
Co-authored-by: Jiang Wu <245355730+JiangLLM@users.noreply.github.com >
2026-09-18 21:06:30 -07:00
Kevin H. Luu and Kimi
9d75dd4544
[CI] Deflake pooling shards with GPU teardown fixtures between tests ( #57371 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-09-19 11:30:02 +08:00
Kevin H. Luu
2fd6268e6d
[CI] Deflake MTEB score tests with per-model mteb_tol and reruns ( #57364 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-09-19 11:29:00 +08:00
Kevin H. Luu
729ebac498
[CI] Reclaim GPU memory between model initialization tests ( #57362 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-09-18 17:04:14 -07:00
1dc2d854c1
[Feat][Model] Support encoder-side SWA-bounded replay for DeepSeek-V4.1-Flash ( #56227 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
2026-09-18 15:01:24 -07:00
Woosuk Kwon and Codex
50d812b66a
[Perf][DSV4.1] Optimize MegaMoE staging and NVFP4 cache gathers ( #57604 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-09-18 12:57:19 -07:00
Harry Mellor and Claude Opus 5
8a5cf54387
[Multimodal] Type dummy options per modality ( #57576 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-18 17:29:11 +01:00
Cyrus Leung
dbf4b89cf2
[Misc] Remove unnecessary Transformers version guards ( #57556 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-18 15:01:35 +00:00
44dd18fe0b
[Model][Gemma4] Load Weights with AutoWeightsLoader ( #55911 )
...
Signed-off-by: Josephasafg <ajgard7@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-18 12:07:30 +00:00
Kevin H. Luu and Kimi
f2b6152ec2
[CI] Add flaky rerun markers / timeout skips to sibling tests lacking them ( #57359 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-09-18 03:40:32 -07:00
Sun and Codex
42a33039eb
[Perf] Fuse CohereASR relative attention score accumulation ( #55190 )
...
Signed-off-by: levius <2114377220@qq.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-18 06:18:41 +00:00
Chauncey and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2bbdfcfcfe
[Perf][GLM5.3-Flash] Use cooperative top-k for small GLM decode batches ( #57327 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-18 10:31:35 +08:00
Rohan Potdar and Claude
213c379fca
[ROCm][Bugfix] Gate AITER MXFP8 MoE on the aiter enable flag ( #57426 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-18 00:06:05 +00:00
rasmith and Andreas Karatzas
d3074acdda
[AMD][CI][The Rock] Fix language models standard for The Rock on mi355 ( #53837 )
...
Signed-off-by: Randall Smith <Randall.Smith@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-17 22:13:13 +00:00
41f9104fa6
[DSv4.1] Integrate Mega-Gate from DeepGEMM ( #56266 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-09-17 12:06:36 -07:00
Taneem Ibrahim and Harry Mellor
e960ead0aa
[Mypy] Fix mypy typing for Transformers models ( #54320 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-17 12:11:31 -04:00
jiangkuaixue123 and Codex
de24e51908
[MM][V2] Enable encoder-only ViT CUDA graph capture ( #56922 )
...
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-17 16:12:17 +08:00
leo and OpenAI Codex
b3079e6e46
[Perf][DSpark] Stack DeepSeek V4 context WKV projections ( #54674 )
...
Signed-off-by: liuyao0322 <yaoliu548926@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-17 00:04:53 -07:00
Woosuk Kwon
30b847c1fa
[Perf][DSV4.1] Remove MegaMoE padding and shared padding workaround ( #57204 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-09-16 19:09:30 +00:00
Shanshan Shen
bc0f47cd03
[ROCm][Bugfix] Revert #56433 + #51692 to fix accuracy breakdown for DeepSeek-V4 ( #57132 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-16 18:58:23 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
dff1bde84d
[Model] Qwen4Exp: fp8_e4m3 main KV cache on the QSA path ( #55557 )
...
Signed-off-by: Andrea Semeraro <andrea.semeraro@sezione1.it >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-09-16 15:07:52 +00:00
sashko-zakharchuk and zjy0516
f8b5c11468
[Bugfix][Kimi-K3] Do not classify a stateless first chunk as a decode ( #51483 )
...
Signed-off-by: Oleksandr Zakharchuk <oleksandr.zakharchuk@gmail.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-09-16 21:24:48 +08:00