73a78e6f1f
[SpecDecode] Add LiLiCorr drafter ( #57934 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Signed-off-by: Andrii Skliar <andreyws96@gmail.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
2026-09-25 10:08:28 -04:00
Yongye Zhu and Claude Opus 5.5
378504a544
[MoE] Defer the TRTLLM-Gen top-k finalize on the modular path ( #58635 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-25 15:38:29 +08:00
8a269919f4
[MM] Add Triton kernel for mm_input_normal. ( #56798 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <2037008807@qq.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-25 04:31:20 +00:00
267eee54bf
[Qwen4Exp][ROCm] PLE n-gram table CPU offload ( #57497 )
...
Signed-off-by: Mathew Odden <modden@redhat.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: opencode+deepseek-v4-flash+vllm <opencode+deepseek-v4-flash+vllm@example.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-24 23:28:20 -05:00
4013f3ad6c
[ROCm] Fix CI runtime and tests for MI355 DPX ( #58244 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Mahesh Kunreddi <mahesh.kunreddi@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-24 23:15:11 -05:00
48d8880d09
[Bugfix][Qwen4Exp] Keep pinned PLE prefetch ids out of the CUDA graph pool ( #58489 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 10:51:50 +08:00
90a9515006
[Cleanup] Remove online quantization support in fp8.py in favor of online shorthands ( #53585 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-24 12:48:14 -07:00
5b3f280a6d
[Bugfix] Count unsplit Idefics3 image patches ( #48760 )
...
Signed-off-by: nightcityblade <nightcityblade@gmail.com >
Co-authored-by: nightcityblade <nightcityblade@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-24 11:36:26 -07:00
Thomas Ortner and Harry Mellor
8b84e15066
[transformer] RMSNorm matching for alternative rsqrt ( #54461 )
...
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-24 16:33:06 +00:00
Harry Mellor and Claude Opus 5.5
d881812500
[Chore] Use Transformers v5 names and drop redundant processor use_fast ( #58550 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-24 14:14:10 +00:00
Linze Shi
484c211fe8
[Bugfix][Pooling] Fix JinaVL label configuration and restore multimodal tests ( #57347 )
...
Signed-off-by: Linze-Shi <linzeshi0@gmail.com >
2026-09-24 13:51:25 +00:00
Aarushi Jain and Andreas Karatzas
e0f03eb4f4
[CI] Share BF16 baselines across quantization comparison tests ( #58469 )
...
Signed-off-by: Aarushi Jain <Aarushi.Jain2@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 19:26:09 -05:00
c4cfd7007b
[CI][ROCm] Add an MI355 Kimi-K3 unit test group ( #58012 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 11:25:51 -05:00
Tianyu Guo and Cyrus Leung
3ee2906f47
[EPD] Support metadata-only audio inputs ( #57887 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-23 09:03:50 +08:00
c9b34fdb2d
[Bugfix][GLM-5.3-Flash] Run the dense MLP layers on the sequence-parallel shard ( #58061 )
...
Signed-off-by: Jared Wen <w13431838023@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-22 21:58:21 +08:00
Ting SUN and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a33b3bac5d
[Bugfix][Qwen2.5-VL] Honor video fps for temporal M-RoPE ( #47736 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-22 05:56:12 +00:00
Shanshan Shen
42a85a4976
[ROCm][Bugfix] Explicitly reject FSE=1 with DPA+ETP deployment for DeepSeek-V4 ( #57919 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-22 04:45:53 +00:00
13058faa49
[Frontend] Add reusable TP1 initialized-engine snapshots ( #51360 )
...
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com >
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-21 23:18:49 +00:00
382970ee6c
[Kimi-K3][AMD] Return KDA and MLA projection outputs directly ( #50592 )
...
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com >
Co-authored-by: Liuyinfeng01 <199041580+LiuYinfeng01@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-21 15:44:41 -05:00
djramic and Andreas Karatzas
4f14516790
[ROCm][CI] Use ROCm backend for DeepSeek V4.1 ViT test ( #57931 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-21 16:39:31 +00:00
Cyrus Leung
7853700d4c
[MM] Move get_dummy_processor_inputs into MM processor ( #57967 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-21 09:39:20 -07:00
Sahil Patel and Cyrus Leung
4e61ad8ee1
[Bugfix] unskip InternViT test for transformers v5 compatibility ( #55767 )
...
Signed-off-by: sahil <sahil@example.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-21 09:38:25 -07:00
Tuukka Sarvi and larryli2-amd
e34685dfc0
[ROCm][DSv4] Enable DSpark adaptive verification ( #52362 )
...
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com >
Signed-off-by: larryli2-amd <larryli2@amd.com >
Co-authored-by: larryli2-amd <larryli2@amd.com >
2026-09-21 10:03:41 -05:00
Cyrus Leung
c8824a1e42
[MM] Move supports_multimodal_inputs and cache out of registry ( #57913 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-21 07:43:07 -07:00
Andreas Karatzas and OpenAI Codex
8902dbb177
[CI][ROCm] Mirror remaining portable suites without duplicate coverage ( #57877 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-21 04:16:26 -05:00
akii96
0ff0477ff9
[Perf][ROCm] Add a ROCm path for Hy4 and compile the backbone ( #57526 )
...
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com >
2026-09-21 08:20:43 +00:00
Oliver Holworthy and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
9598487bc1
[Bugfix][Model] Mistral3: align placeholder grid with public processor ( #53758 )
...
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-21 06:36:59 +00:00
Tianyao Wu
59f1527280
[Bugfix] Fix Whisper engine crash on audio clips longer than 30s ( #57769 )
...
Signed-off-by: Tianyao Wu <rayroy31@gmail.com >
2026-09-21 05:44:40 +00:00
Andreas Karatzas and OpenAI Codex
8ec00ec2bc
[Tests] Release HF embedding weights after prompt extraction ( #57868 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-21 05:27:29 +00:00
Andreas Karatzas and OpenAI Codex
76ee20c1ec
[ROCm][Tests] Avoid pooling cleanup waits against live shared engines ( #57864 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-21 05:21:35 +00:00
Chauncey and zjy0516
3df4ae153e
[Core][KDA] Generalize Mamba prefill checkpoint builder and exporter ( #57783 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-09-21 04:46:11 +00:00
Isotr0py
27757dde02
[DSV4.1] Add encoder cuda graph support for deepseek-v4.1-flash ( #56625 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-20 21:51:13 +08:00
Chauncey
bf01fc4a31
[GLM-5.3-Flash] Route kpool indexer top-k through the shared SparseIndexerTopk dispatcher ( #57546 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-09-20 11:17:59 +00:00
yinfengLiu and Shanshan Shen
0e110f696d
[ROCm][DSV4][Perf] Use FP8 WO_A output projection ( #54894 )
...
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-20 09:13:55 +00:00
Cyrus Leung
751f6807d9
[Refactor] Use dynamic MM cache in processor ( #57674 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-19 15:38:00 +00:00
vllmellm
211e252d0b
[Bugfix][Model] Fix MiMo-V2.5 fused fp8 qkv_proj sharding (pre-shard count is num_key_value_heads; MTP path too) ( #57508 )
...
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com >
2026-09-19 06:05:00 -07:00
Jiang Wu and Jiang Wu
01c7bf8813
[Bugfix][Model] Fix Aria expert weight names and layout ( #57487 )
...
Signed-off-by: Jiang Wu <245355730+JiangLLM@users.noreply.github.com >
Co-authored-by: Jiang Wu <245355730+JiangLLM@users.noreply.github.com >
2026-09-18 21:06:30 -07:00
Kevin H. Luu and Kimi
9d75dd4544
[CI] Deflake pooling shards with GPU teardown fixtures between tests ( #57371 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-09-19 11:30:02 +08:00
Kevin H. Luu
2fd6268e6d
[CI] Deflake MTEB score tests with per-model mteb_tol and reruns ( #57364 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-09-19 11:29:00 +08:00
Kevin H. Luu
729ebac498
[CI] Reclaim GPU memory between model initialization tests ( #57362 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-09-18 17:04:14 -07:00
1dc2d854c1
[Feat][Model] Support encoder-side SWA-bounded replay for DeepSeek-V4.1-Flash ( #56227 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
2026-09-18 15:01:24 -07:00
Woosuk Kwon and Codex
50d812b66a
[Perf][DSV4.1] Optimize MegaMoE staging and NVFP4 cache gathers ( #57604 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-09-18 12:57:19 -07:00
Harry Mellor and Claude Opus 5
8a5cf54387
[Multimodal] Type dummy options per modality ( #57576 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-18 17:29:11 +01:00
Cyrus Leung
dbf4b89cf2
[Misc] Remove unnecessary Transformers version guards ( #57556 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-18 15:01:35 +00:00
44dd18fe0b
[Model][Gemma4] Load Weights with AutoWeightsLoader ( #55911 )
...
Signed-off-by: Josephasafg <ajgard7@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-18 12:07:30 +00:00
Kevin H. Luu and Kimi
f2b6152ec2
[CI] Add flaky rerun markers / timeout skips to sibling tests lacking them ( #57359 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-09-18 03:40:32 -07:00
Sun and Codex
42a33039eb
[Perf] Fuse CohereASR relative attention score accumulation ( #55190 )
...
Signed-off-by: levius <2114377220@qq.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-18 06:18:41 +00:00
Chauncey and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2bbdfcfcfe
[Perf][GLM5.3-Flash] Use cooperative top-k for small GLM decode batches ( #57327 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-18 10:31:35 +08:00
Rohan Potdar and Claude
213c379fca
[ROCm][Bugfix] Gate AITER MXFP8 MoE on the aiter enable flag ( #57426 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-18 00:06:05 +00:00
rasmith and Andreas Karatzas
d3074acdda
[AMD][CI][The Rock] Fix language models standard for The Rock on mi355 ( #53837 )
...
Signed-off-by: Randall Smith <Randall.Smith@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-17 22:13:13 +00:00
