Commit Graph
21848 Commits
Author SHA1 Message Date
Andreas KaratzasandOpenAI Codex 82daf9f575 [CI][ROCm] Add ten AMD parity groups (#57876)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-21 11:08:06 +00:00
CaiJohnandjohn 15859bb3a1 [Test] Organize structured output utility tests (#57545)
Signed-off-by: john <6712432+CaiJohn@users.noreply.github.com>
Co-authored-by: john <6712432+CaiJohn@users.noreply.github.com>
2026-09-21 13:56:28 +03:00
Fangzhou AiandCursor 04c1f4a407 [Bugfix][ROCm][DSv4.1] Disable SWA bounded replay on ROCm (#57906)
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-21 10:29:14 +00:00
0b7f11a1ee [AuxOutput] Add block-keyed storage for routed-expert outputs (#45635)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-21 10:25:05 +00:00
djramic 0aee727ff6 [CI][ROCm] Increase timeout for MI300 Distributed DP Extended (#57903)
Signed-off-by: Djordje Ramic <djoramic@amd.com>
2026-09-21 04:18:59 -05:00
Andreas KaratzasandOpenAI Codex 8902dbb177 [CI][ROCm] Mirror remaining portable suites without duplicate coverage (#57877)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-21 04:16:26 -05:00
b8cf275382 [AMD][Minimax-M3][perf] Enable packed LBHNC AITER QK-norm fusion for MiniMax-M3 on ROCm (#54535)
Signed-off-by: Wei-Ting Liao <wei-ting.liao@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
2026-09-21 16:38:30 +08:00
Andreas KaratzasandOpenAI Codex f05b88751e [CI][ROCm] Make Python-only installation failures blocking (#57870)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-21 08:29:02 +00:00
76ffb0d374 [Model][Engram] Share host tables across co-located DP replicas by default (#57651)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
2026-09-21 08:26:56 +00:00
akii96 0ff0477ff9 [Perf][ROCm] Add a ROCm path for Hy4 and compile the backbone (#57526)
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
2026-09-21 08:20:43 +00:00
Woosuk KwonandOpenAI Codex 67513c8b67 [Bugfix][DSV4.1] Restrict mHC overlap to full CUDA graphs (#57874)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-21 01:16:59 -07:00
Woosuk KwonandOpenAI Codex 97dc6b19d2 [Perf][Attention] Avoid redundant sparse attention metadata operations (#57885)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-21 00:08:43 -07:00
3c84ad13ed [Bugfix] Fix xgrammar feature gate bypass when JSON Schema type is a list (#48416)
Signed-off-by: john <6712432+CaiJohn@users.noreply.github.com>
Co-authored-by: john <6712432+CaiJohn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-21 09:59:20 +03:00
Oliver Holworthyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 9598487bc1 [Bugfix][Model] Mistral3: align placeholder grid with public processor (#53758)
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-21 06:36:59 +00:00
Tianyao Wu 59f1527280 [Bugfix] Fix Whisper engine crash on audio clips longer than 30s (#57769)
Signed-off-by: Tianyao Wu <rayroy31@gmail.com>
2026-09-21 05:44:40 +00:00
Andreas KaratzasandOpenAI Codex 8ec00ec2bc [Tests] Release HF embedding weights after prompt extraction (#57868)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-21 05:27:29 +00:00
liuzhenweiandKunshang Ji 6654bcbfca [MRV2][XPU] use xpu sample kernel in mrv2 sampler (#57277)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-09-21 13:24:41 +08:00
Andreas KaratzasandOpenAI Codex 76ee20c1ec [ROCm][Tests] Avoid pooling cleanup waits against live shared engines (#57864)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-21 05:21:35 +00:00
liuzhenweiandKunshang Ji a15dbbd63d [XPU][CI] fallback to 7-args of moe_align_block_size (#57855)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-09-21 13:15:29 +08:00
Chaunceyandzjy0516 3df4ae153e [Core][KDA] Generalize Mamba prefill checkpoint builder and exporter (#57783)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
2026-09-21 04:46:11 +00:00
Matt MastracciClaude Fable 5.1Lucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ff0c885494 [Bugfix] DiffusionGemma: fix multimodal support (#57589)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-21 00:05:24 -04:00
Woosuk KwonandCodex 9a70c233cd [Perf][DSV4.1] Fuse TP all-reduce with mHC input preparation (#57643)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
2026-09-20 20:28:06 -07:00
eb87980585 [Build] Fix DeepGEMM CUDA 12.9 release builds (#57554)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
2026-09-20 20:14:00 -07:00
Jee Jee Li 86ce4d10e2 [BUGFIX][HY4] Record indexer completion event for full CUDA graph capture (#57811)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-09-21 10:59:11 +08:00
czhu-cohere 3918f3c5a3 [Profiler] Unify platform-aware torch profiling (#57460) 2026-09-20 19:17:23 -07:00
JohnQinAMD 6dfd87f596 [ROCm][DSv4.1] Keep the Engram tables in host memory on ROCm (#57491) 2026-09-21 10:17:11 +08:00
Zhou ZihangandJee Jee Li 3e67044972 [Doc] Refresh Kthena integration guide (#57314)
Signed-off-by: Zhou Zihang <z@mcac.cc>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
2026-09-21 01:42:55 +00:00
1185254851 [Bugfix][KV Connector] Propagate cache reset failure during sleep (#54581)
Signed-off-by: CrazyCoder <18235787078@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
2026-09-21 09:25:02 +08:00
jiangkuaixue123andCodex 7ddd461a28 [Docs] Add ECMooncakeConnector usage example for EPD (#57742)
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-21 09:15:02 +08:00
Yifan QiaoandClaude Fable 5.1 d2983f2f16 [Bugfix][Spec Decode] Stop dummy draft decode steps from writing KV through stale block-table rows (#56734)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 01:06:25 +00:00
Woosuk KwonandCodex 9b49f92344 [Perf][DSV4.1] Overlap mHC coefficients for small TP batches (#57603)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
2026-09-20 17:33:17 -07:00
YiSheng5andKunshang Ji 17e50b9b76 [XPU][UT]Bugfix when the process can't see all the world_size meet accuracy issue. (#57779)
Signed-off-by: yisheng <yi.sheng@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-09-21 06:50:25 +08:00
YukioZzz 9679173788 [KVConnector][MoRIIO] Transfer hybrid mamba/KDA recurrent state in READ mode (#51052)
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
2026-09-20 17:30:34 -04:00
01f1f58f10 [ROCm][CI] Query HIP device memory for test GPU teardown waits. (#57450)
Signed-off-by: Sheral Kumar <shekumar@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-20 15:20:39 -05:00
Wentao Ye 4868312128 [Refactor] Remove dead kernel code (#57621)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-09-20 10:39:23 -07:00
roikoren755andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 49ee12d742 [Misc] Rename --enable-mamba-fine-grained-prefix-cache (#53945 follow-up) (#57382)
Signed-off-by: Roi Koren <roik@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-20 09:50:34 -07:00
7d99c2c4fd [Feature][Humming] Humming feature integration (#56685)
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
2026-09-20 08:11:01 -07:00
Isotr0py 27757dde02 [DSV4.1] Add encoder cuda graph support for deepseek-v4.1-flash (#56625)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-20 21:51:13 +08:00
ZyannandAbatom 9b2f34cad4 [Feature] support bf16 MoE router and mxfp4 MoE for MiMo V2 (#57784)
Signed-off-by: Zyann7 <62597503+Zyann7@users.noreply.github.com>
Co-authored-by: Abatom <182586866+Abatom@users.noreply.github.com>
2026-09-20 13:25:56 +00:00
10e6a7f210 [Frontend] Expose per-request spec decode metrics in generate API (#43310)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
2026-09-20 12:21:33 +00:00
Chauncey bf01fc4a31 [GLM-5.3-Flash] Route kpool indexer top-k through the shared SparseIndexerTopk dispatcher (#57546)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-09-20 11:17:59 +00:00
f648eed23d [Bugfix][KV Offload] Skip non-prefix-cacheable groups in SimpleCPUOffload (GLM-5.3-Flash kpool tail and QSA) (#56810)
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
2026-09-20 10:58:55 +00:00
Jared WenClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
1b9fa3eaa8 [Perf][GLM] Fuse the kpool tail slot mapping into one Triton kernel (#57534)
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-20 18:20:55 +08:00
Jared WenClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
db1bfdd4fb [Bugfix][GLM-5.3-Flash] Address kpool tail blocks by the padded indexer stride in the NVIDIA prefill seed kernel (#57477)
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-20 18:15:04 +08:00
mahird3 0748d3bd57 [Model][LoRA] Enable LoRA support for VoyageQwen3BidirectionalEmbedModel (#57708)
Signed-off-by: Mahir Dursunoglu <142054011+mahird3@users.noreply.github.com>
2026-09-20 09:50:00 +00:00
Misha GoinandErinYin e5fce7b56b [Core][Kernel] Share persistent workspaces for Marlin and Humming (#57421)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: ErinYin <liang.yin@daocloud.io>
2026-09-20 17:28:33 +08:00
yinfengLiuandShanshan Shen 0e110f696d [ROCm][DSV4][Perf] Use FP8 WO_A output projection (#54894)
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
2026-09-20 09:13:55 +00:00
Dilber 27b7757f63 [Bugfix][Frontend] Bound the prompt after multimodal expansion (#57076)
Signed-off-by: Dilber P Shakir <dilbersha@gmail.com>
2026-09-20 08:52:27 +00:00
Mi Jiazhiandjiazhimi e378275a8f [Model] Pass intermediate_tensors to the model when capturing CUDA gr… (#57745)
Co-authored-by: jiazhimi <jiazhimi@tencent.com>
2026-09-19 23:23:29 -07:00
Yan Ma d05da62e9c [XPU] fix incorrect gdn kernel log for XPU path (#54871)
Signed-off-by: Yan Ma <yan.ma@intel.com>
2026-09-20 14:00:31 +08:00