aoshen02
4eef91c03c
[Model] Support R3 capture with DeepGEMM MegaMoE ( #51831 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
2026-08-12 01:07:07 -07:00
Matej Sirovatka and Jee Jee Li
3ac9525507
[Bugfix][Quantization] Reuse online NVFP4 MoE kernel across reloads ( #50074 )
...
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-08-12 14:06:53 +08:00
Lucas Wilkinson and OpenAI Codex
02ac17851d
[Refactor][MRV2] Unify uniform decode token count helper ( #51917 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-12 05:58:43 +00:00
aoshen02 and Codex
f2906b492e
[Bugfix] Avoid repeated layerwise reload warning scans ( #49505 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-08-12 13:53:50 +08:00
Vinay R Damodaran and Claude Opus 5
3ee2df3033
[Frontend] Make model optional on all /derender request classes ( #51463 )
...
Signed-off-by: Vinay Damodaran <vrdn@hey.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-12 04:48:58 +00:00
3962042304
[Bugfix][V1][Multimodal] Recover from P0/P1 processor cache drift ( #46747 ) ( #46747 )
...
Signed-off-by: William Zhao <williamzh@meta.com >
Signed-off-by: WillZZZy <93454470+WillZZZy@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-12 12:06:57 +08:00
7f9173dfa2
[ROCm][Perf] Kimi-K3 Fused kernel for KDA decode ( #50654 )
...
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-08-11 20:39:34 -07:00
Stellar鱼
3fa89225bc
[CI/Build] Use file rendezvous for local distributed tests ( #51652 )
...
Signed-off-by: yu-xin-c <2182712990@qq.com >
2026-08-12 10:48:12 +08:00
Matvei Pashkovskii and Claude
20727e2841
[Hardware][AMD] Enable fused bf16→fp32 router GEMM on ROCm ( #50268 )
...
Signed-off-by: Matvei Pashkovskii <mpashkov@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-12 02:46:13 +00:00
Andreas Karatzas
23f25aac14
[CI] Stabilize tensor IPC multiprocessing tests ( #50804 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-08-12 10:16:43 +08:00
Varun Sundar Rabindranath and Varun Sundar Rabindranath
f067737b2a
[Bugfix][TieredOffloading] : Return HIT_PENDING when KV promotion is triggered ( #51840 )
...
Signed-off-by: <>
Co-authored-by: Varun Sundar Rabindranath <varun-sundar-rabindranath@h100-01.nemg-001.lab.rdu2.dc.redhat.com >
2026-08-12 05:07:16 +03:00
Aaron Hao and Claude Opus 5
793ca6998a
[Docs][RL] Rewrite weight-transfer docs; standardize examples ( #51729 )
...
Signed-off-by: haoaaron <ahao@anyscale.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-11 18:53:15 -07:00
Micah Williamson and Andreas Karatzas
045a986f3b
[ROCm][CI] Speed Up ROCm Skinny GEMM Tests (reduced parameterizations, ( #51877 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-12 01:40:13 +00:00
0a94d85a66
[Bugfix][MRV2] Require all requests to be decoding for uniform-decode dispatch ( #51865 )
...
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com >
Co-authored-by: Janelle Cai <janelle.cai@modal.com >
2026-08-11 17:15:06 -07:00
1d2d83a07f
[Attention] Add FlashInfer XQA decode support on SM12x ( #49718 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Signed-off-by: Andrii Skliar <andreyws96@gmail.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-11 16:32:49 -07:00
鐘天楽
a367cdbca0
[Profiler] Add minimal Triton Proton profiling backend ( #48789 )
...
Signed-off-by: Luossu <zhluosuu@outlook.com >
2026-08-11 14:51:57 -07:00
Richard Zou
cb30f6f8dd
[Testing] Fix test_sharded_state_loader ( #51736 )
...
Signed-off-by: Richard Zou <zou3519@gmail.com >
2026-08-11 21:16:09 +00:00
stefankoncarevic
3e372c5ff2
[Bugfix][ROCm] Give KV-first attention blocks their own page in hybrid models ( #51837 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
2026-08-11 16:52:30 -04:00
Rohan Potdar and Claude Opus 4.8
0f0cb918b7
[ROCm][MoE] Fix expert_map vs AITER expert_mask for non-AITER experts under EP ( #49758 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-11 15:33:02 -05:00
Andreas Karatzas and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
36f4630d89
[Bugfix][MoE] Fix fused block-scale orientation ( #50727 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 16:05:38 -04:00
Andreas Karatzas
cc668c5b72
[CI][Bugfix][V1] Remove stale FlashAttention metadata arguments ( #51854 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-08-11 18:54:56 +00:00
Andreas Karatzas and OpenAI Codex
1ab2801dde
[ROCm] Remove stale SDPA and skinny GEMM workarounds ( #50907 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-11 14:19:16 -04:00
Harry Mellor
9cc347ae42
Enable MiniCPMV for vLLM in CI ( #45042 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-11 16:30:39 +00:00
Michael Goin and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
e3fe212eaf
[Bugfix] Generalize KV block zeroing to AttentionSpec ( #51749 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 12:13:09 -04:00
Nick Hill
513f83e7ec
[Bugfix] Take the sliding window from the layer, not the KV cache group ( #51756 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-11 09:10:46 -07:00
Harry Mellor
c65aa2ee6c
Bump Transformers version to 5.15.0 ( #51668 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-11 09:00:51 -07:00
Guan-Ming Chiu and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
457a5f34db
[Bugfix][MRV2] Support encoder timing stats in model runner V2 ( #50020 )
...
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-12 00:00:49 +08:00
5426311d91
[Kernel][ROCm][Perf] FlyDSL decode-attention kernel for 4-bit TurboQuant KV cache ( #47896 )
...
Signed-off-by: root <root@smci355-ccs-aus-m02-09.cs-aus.dcgpu >
Co-authored-by: root <root@smci355-ccs-aus-m02-09.cs-aus.dcgpu >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 08:45:39 -07:00
liuzhenwei and Kunshang Ji
d648236199
[XPU][CI] Fix ExampleConnector KV cache device selection ( #51806 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-11 23:40:29 +08:00
pmanczak and Kunshang Ji
5b5eae2394
[Misc] Enable test_silu_mul_fp8_quant_deep_gemm on XPU ( #49444 )
...
Signed-off-by: pmanczak <pawel.manczak@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-11 23:19:14 +08:00
Wentao Ye
4988df2eb8
[Config] Update default _max_num_batched_tokens from 8192 to 16384 ( #51726 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-11 11:09:34 -04:00
Woosuk Kwon and OpenAI Codex
87668ab69b
[Bugfix] Guard DeepSeek V4 MRV1 piecewise CUDA graphs ( #51768 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-11 07:34:44 -07:00
b64a2708b0
Bound generation inputs before expensive work ( #51447 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 22:03:50 +08:00
6c95a641e9
[2/N][Feat][Perf] Add new warmup infrastructure for JITs. Add predicate filtering for JIT warmup, and migrate Inkling FA4 ( #49315 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 07:46:07 -04:00
AlexHuang
0fec3d652b
[Bugfix][KV Offload] Centralize shared mmap cleanup in CPU worker ( #51622 )
...
Signed-off-by: Alex <jihui.huang@daocloud.io >
2026-08-11 13:22:15 +03:00
Dao007forever and OpenAI Codex
ce07118669
[Bugfix][Core] Preserve Mamba running CoW after external hits ( #51766 )
...
Signed-off-by: Dao Le <Dao007forever@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-11 18:15:24 +08:00
Tianyu Guo and OpenAI Codex
3fb7bb46d6
[Perf] Avoid repeated multimodal prompt update scans ( #51774 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-11 09:23:24 +00:00
Yizhou
a311916a29
[MRV2][Spec] Fuse AR speculator multi-step decodes back into one CUDA graph ( #46849 )
...
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com >
2026-08-11 15:06:22 +08:00
Fangzhou Ai
1044d19998
[ROCm][DSV4] Preserve native MXFP4 TP8 shard allocation ( #51473 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
2026-08-10 23:57:27 -07:00
Priyansh Jain and Andreas Karatzas
2acb055ed7
[CPU][Zen] Route BF16 MoE inference through zentorch on AMD ( #44201 )
...
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-10 23:20:05 -07:00
Wei Zhao
608c12473f
[Attention] Fix MLA prefill workspace allocation size ( #51733 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-08-11 05:34:30 +00:00
c76a425278
[Kernel] Optimize long-context MLA cache gathers ( #51739 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-10 22:23:43 -07:00
cherry77-cloud and OpenAI Codex
65b7662d3f
[Bugfix][Frontend] Report Cohere stop sequences correctly ( #51556 )
...
Signed-off-by: cherry77-cloud <1615405@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-11 04:49:49 +00:00
Khushali Desai and wang.yuqi
dc5101fb1b
replace batch_norm to numerically identical without cudnn ( #51734 )
...
Signed-off-by: khushali9 <khushali.desai9@gmail.com >
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-08-11 12:43:20 +08:00
Clinton Thomas and Lucas Bourtoule
90fd4a333f
Preserve revision pins in secondary artifact loaders ( #51446 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
2026-08-11 04:33:10 +00:00
Frank
adc1200597
[Misc] Use VLLMValidationError in scoring input validation ( #51753 )
...
Signed-off-by: frank-suwen <suwenw2@outlook.com >
2026-08-11 04:12:22 +00:00
Wentao Ye
0914ed2e81
[Perf] Adaptive budget for spec scheduled input tokens, ~60% better Kimi K3 DSpark TTFT ( #51725 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-10 20:52:21 -07:00
Itay Etelis and Itay Etelis
0f2ea973be
[KV Connector][Offloading] Keep per-layer KV registration when canonical_layout is requested ( #51688 )
...
Signed-off-by: Itay Etelis <itay.etelis@ibm.com >
Co-authored-by: Itay Etelis <itay.etelis@ibm.com >
2026-08-11 06:03:59 +03:00
aoshen02 and Roger Wang
99e62b802c
[Bugfix][Model Loader] Defer post-load attention weight processing ( #49519 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-08-10 19:16:57 -07:00
33584901ba
[Frontend] Add content_parts to /inference/v1/generate for raw multim… ( #51478 )
...
Signed-off-by: Allen Shen <aoshen@inferact.ai >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 10:00:12 +08:00
aoshen02