Kevin H. Luu and Kimi
99ea5f525d
[CI] Raise DSv4-Flash disaggregated engine readiness timeout to 1800s ( #57334 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-09-17 03:34:00 -07:00
Kevin H. Luu and Kimi
683f6c8edf
[CI] Raise H200 LM Eval Large Models timeout to 120 min ( #57335 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-09-17 03:33:44 -07:00
Itay Etelis and Or Ozeri
b6e7c1f1f0
[kv_offload] Skip scratch groups ( #57145 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-09-17 13:22:49 +03:00
40b40d1d39
[Bugfix][KV Offload] Register the offload region in chunks ( #51081 )
...
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Colton Ottley <coltonottley@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-09-17 13:19:56 +03:00
08633cb5cd
[Qwen3.8-Flash-Next] Enable FP8 TP with FlashInfer TRTLLM MoE ( #55867 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-17 08:56:28 +00:00
f1c2f6ada8
[ROCm][CI] Stage H gating and MI355 test reallocation ( #57080 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-17 01:44:11 -07:00
23cd0346ce
[CPU] Align scheduler and NIXL CPU affinity per local rank ( #53636 )
...
Signed-off-by: Li, Tianmu <tianmu.li@intel.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-17 16:43:08 +08:00
coderfornow and Li, Jiang <jiang1.li@intel.com>
88afb77700
[CPU][s390x] Pin protobuf to 7.36.1 and drop C++ extension removal workaround ( #54978 )
...
Signed-off-by: coderfornow <Ritik.Dhiranan@ibm.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-17 16:35:17 +08:00
jiangkuaixue123 and Codex
de24e51908
[MM][V2] Enable encoder-only ViT CUDA graph capture ( #56922 )
...
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-17 16:12:17 +08:00
Jiahan Chang (Cyrus) and OpenAI Codex
4e5ffda19d
[Bugfix] Isolate supplemental FlashInfer BF16 autotuning ( #57285 )
...
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-17 07:38:58 +00:00
528fa835fd
[ROCm] Stage large pageable H2D copies instead of registering them ( #56343 )
...
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-17 07:14:43 +00:00
leo and OpenAI Codex
b3079e6e46
[Perf][DSpark] Stack DeepSeek V4 context WKV projections ( #54674 )
...
Signed-off-by: liuyao0322 <yaoliu548926@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-17 00:04:53 -07:00
Shuolei Wang and Roger Wang
7bbce752b8
[Perf][Model] Qwen4Exp QSA: sm_90 tuning table for _select_config ( #57273 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Shuolei Wang <948904026@qq.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-09-17 07:02:24 +00:00
Lucas Wilkinson
bd2d7ae7c1
[Perf] Reuse MoE workspace for DeepGEMM warmup ( #57268 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-09-17 03:01:19 -04:00
Shanshan Shen
fdcd42350b
[ROCm][Perf] Enable HCA dual-stream overlap for DeepSeek-V4 ( #56853 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-17 06:46:00 +00:00
aoshen02
b978df3299
Revert "[Frontend] Omit absent fields from /inference/v1/generate stream chunks" ( #57302 )
2026-09-16 23:31:59 -07:00
lesj0610
1eacb41e0f
Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-09-17 15:01:29 +09:00
aoshen02 and Claude Fable 5.1
8538017f4b
[Frontend] Omit absent fields from /inference/v1/generate stream chunks ( #57264 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-17 05:58:56 +00:00
Qiming Zhang
e6b1a5e5e3
[XPU] Enable XPU eplb ( #44987 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com >
2026-09-17 13:19:13 +08:00
2743dc4f92
[Feature][Multimodal] Add cross-encoder caching to Mooncake P2P ( #56242 )
...
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com >
Co-authored-by: Tianyu Guo <guoty@inferact.ai >
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-17 13:16:41 +08:00
Matt
2d8a5c5741
[ROCm][Bugfix] Reduce CUDA graph divergences ( #57229 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
2026-09-17 05:12:13 +00:00
Kevin H. Luu
0bfc7a15d0
[CI] Fix ruff docstring violations in EC connector files ( #57283 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-09-16 21:13:15 -07:00
Robert Shaw and Robert Shaw
c38f92aae7
[A2A] Allow EPLB + SharedExpert Overlap + DeepEPv2 ( #57236 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
2026-09-17 04:02:39 +00:00
Matt and Claude Code
68f2c7e74f
[ROCm][CI] Fix Nixl+Offloading PD edge cases on TheRock image ( #57249 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-09-16 22:56:13 -05:00
Alec
d2a2a82954
[Rust Frontend] Bump vllm-proto to 0.3.0 ( #57233 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
2026-09-17 03:41:55 +00:00
lesj0610
b8adc7710e
Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-09-17 12:32:30 +09:00
Itay Etelis and Or Ozeri
9854b580df
[Bugfix][KV Offload] Compact canonical MLA rows ( #56799 )
...
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-09-17 06:24:06 +03:00
Nick Hill
5efbf0886a
[CI] Support FORCE_COLOR env var; enable colors in CPU CI output ( #56913 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-09-16 22:40:21 -04:00
omerpaz95
fa3622a4a1
[EC Connector] Add Metrics Collection ( #54960 )
...
Signed-off-by: omerpaz95 <omerpaz95@gmail.com >
2026-09-17 10:36:23 +08:00
Oxana Korzh and Andreas Karatzas
2307a1d16c
[Core] Don't issue a blocking collective RPC during engine handshake ( #57226 )
...
Signed-off-by: okorzh <okorzh@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-16 21:30:59 -05:00
lesj0610
77b8783b25
Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-09-17 11:29:18 +09:00
Chuan (Richard) Li and Cursor
f730a93d2b
[Scheduler] Add --max-num-active-seqs to cap RUNNING admission ( #56758 )
...
Signed-off-by: Li <chuali@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-17 10:22:12 +08:00
Jiangyun Zhu and Codex
b1f9ebc0c3
[Test][Core] Add hybrid model prefix cache hit-rate coverage ( #57127 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-16 19:17:45 -07:00
062f81d8ef
Use skip-redaction option for crcr-report ( #56532 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-16 19:01:54 -07:00
Alec and Codex
28ec03bc6e
[Bugfix][Rust Frontend] Accept appended EngineCoreOutput fields ( #56533 )
...
Co-authored-by: Codex <noreply@openai.com >
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
2026-09-17 01:54:43 +00:00
dongluw and Andreas Karatzas
19b6ff62f2
[ROCm][Bugfix] Ignore descales for unquantized AITER caches ( #56726 )
...
Signed-off-by: donglu <donglu@cohere.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-17 01:29:50 +00:00
Robert Shaw and Robert Shaw
ed769b4a45
[A2A] Add DeepEPv2 to SP Supported List ( #57210 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
2026-09-16 21:29:10 -04:00
Bugen Zhao
e52da038d1
[Rust Frontend] Normalize native renderer reasoning controls ( #56998 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-16 18:15:05 -07:00
3e267bae70
[ROCm] triton+triton_kernels 3.8 mxfp4 MoE support (gpt-oss + DeepSeek-V4) ( #55934 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: Matt <156021403+mawong-amd@users.noreply.github.com >
2026-09-16 20:03:34 -05:00
Divakar Verma and Andreas Karatzas
95f4925c3a
[ROCm][CI] Enable AITER FP8/unquantized cases in modular-kernel sweep + fix MoRI per-tensor FP8 dispatch ( #57074 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-16 19:53:06 -05:00
Thillai Chithambaram
a615f53364
[Bugfix][Metrics][MFU] Size activation traffic from the model dtype ( #57070 )
...
Signed-off-by: Thillai Chithambaram <thillaichithambaram.a@gmail.com >
2026-09-17 00:23:28 +00:00
c58ff535f4
[Bugfix][Spec Decode] Fix EAGLE and dense draft startup with EP ( #56930 )
...
Signed-off-by: Kyle Liang <kyleliang-nv@users.noreply.github.com >
Co-authored-by: Kyle Liang <kyleliang-nv@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-16 17:03:36 -07:00
Aarushi Jain and Andreas Karatzas
91b96a533c
[ROCm][CI] Shard MI300 Multimodal Processor ( #57056 )
...
Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-16 18:55:14 -05:00
lesj0610
6c5552776b
Merge remote-tracking branch 'origin/main' into lesj/qwen4exp-bugfixes
...
Both sides edited the same import block in the Qwen4Exp NVIDIA and AMD
models and nothing else collided. This branch adds
`from ..common.mtp import make_mtp_hidden_buffer`, the helper the MTP
buffer extraction calls; upstream ec4a3a5370 widened the `..config`
import to `ATTENTION_LAYER_TYPES, QSA_LAYER_TYPE, Qwen4ExpConfig` for the
layer-type constants it introduced. The two changes are independent, so
the resolution keeps both lines. Every symbol still has a use outside the
import in both files.
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-09-17 08:16:59 +09:00
andrewor14
62f9482583
[Tests] Delete deprecate torchao tests for v1 configs ( #52956 )
...
Signed-off-by: andrewor14 <andrewor14@gmail.com >
2026-09-16 23:11:27 +00:00
xinnywinne and Benjamin Chislett
fbf2c5e8be
[Frontend] Add per-request metrics to Responses API ( #55084 )
...
Signed-off-by: wxin <wxin@nvidia.com >
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com >
2026-09-16 22:52:05 +00:00
aoshen02
0983aef8da
[Frontend] Return prompt metadata from /inference/v1/generate ( #53187 )
2026-09-16 15:29:07 -07:00
Robert Shaw and Claude
6c3d9816af
[BugFix][KV Connector] Fix Deadlock with KVConnector + MTP under KV Pressure ( #57104 )
...
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-16 22:22:07 +00:00
yzong-rh and Flora Feng
5a6ccc5892
[Bugfix] Add Responses cache_write_tokens for API compat and CC parity ( #57222 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-16 21:52:34 +00:00
Yongye Zhu
929aa2a7ed
[Build] Bump DeepGEMM pin to a6bbb80 ( #57218 )
2026-09-16 14:49:35 -07:00