Shijin Zhang and Chauncey
83773ecdf3
[Frontend] Support CustomToolCall in the non-Harmony Responses API ( #57322 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-12 17:22:55 +08:00
Thuongvu Ho and Nicolò Lucchesi
99b44c7a42
[Metrics] Add admission rejection metrics ( #55812 )
...
Signed-off-by: Thuongvu Ho <thuongvuho@gmail.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
2026-10-12 08:31:07 +00:00
Ting SUN
f4114e068e
[Bugfix][Frontend] Preserve token IDs in chat derender responses ( #61065 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-10-12 07:55:06 +00:00
AMD-yanfeiwang and aoshen02
e3bbaf2369
[ROCm][DSv4] Fuse the shared expert into AITER MegaMoE ( #61004 )
...
Signed-off-by: Yanfei Wang <yanfei.wang@amd.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-12 15:53:16 +08:00
Giancarlo Delfin and Rishi Puri
36f3972812
[Spec Decode] Skip metadata build for dflash/dspark when all model backends support it ( #60692 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Rishi Puri <20074092+puririshi98@users.noreply.github.com >
2026-10-12 00:09:10 -07:00
Zhenzhong Xu and Kunshang Ji
6a70913590
[XPU] bump up auto-round-lib to 0.16.0 ( #60976 )
...
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-12 14:47:59 +08:00
74f4404f58
[Bugfix][Distributed] Own SHM out-of-band buffers before releasing read slots ( #61000 )
...
Signed-off-by: Hubert Marek <163992334+hubert-marek@users.noreply.github.com >
Signed-off-by: Ao Shen <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Ao Shen <aoshen@inferact.ai >
2026-10-12 01:44:12 -04:00
Canlin Guo and Claude Opus 5.5
fe4df3df0e
[Bugfix][GLM-5.3-Flash] Restore MTP index sharing across draft iterations ( #61079 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 05:41:52 +00:00
d4e92ccc37
[Bugfix][Frontend] Honor endpoint plugin route overrides ( #58839 )
...
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-12 03:58:15 +00:00
liuzhenwei and Kunshang Ji
bfa7e5b7af
[XPU][CI] Enable XPU graph in graph tests ( #60784 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-12 10:39:40 +08:00
Yan Ma
f147949be8
[XPU] limit FA kernel block size to multipleOf 64 ( #57297 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-10-12 10:25:32 +08:00
dingdangmao and Codex
124b89d5b7
[Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries ( #59962 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-12 02:08:14 +00:00
Nick Hill
3fe917abb1
[Bugfix][MRV2] Fix FlashInfer attention and ReplaySSM warmups on MRV2 ( #61013 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-11 16:04:41 -07:00
f26d9f432c
[Bugfix][GLM-5.3-Flash] Honor --mamba-ssm-cache-dtype for the KDA recurrent state ( #59147 )
...
Signed-off-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-11 17:55:00 +00:00
Nick Hill
c059bee812
[MRV2] Switch CI tests still running on MRV1 unnecessarily ( #61014 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-11 09:57:07 -07:00
Ting SUN and Cyrus Leung
b99fdbe1f8
[Bugfix] Respect the configured offline chat template ( #61061 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-11 16:27:21 +00:00
93d2cab35e
[Security] Fix P2P ZMQ socket exhaustion via unbounded peer sessions ( #51504 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
2026-10-11 11:39:11 -04:00
aoshen02 and Codex
239a3fa3a8
[Sleep] Retain draft-private weights with existing parameter selector ( #61046 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 23:30:56 +08:00
3e8e1364ff
[Perf][GLM-5.3-Flash] Shard long-context indexer prefill rows across TP ( #54951 )
...
Signed-off-by: Zheng Cai <8370601+zigzagcai@users.noreply.github.com >
Signed-off-by: Zheng Cai <caizheng1993@gmail.com >
Signed-off-by: Codex <noreply@openai.com >
Co-authored-by: root <root@rjob-fb29567b88fec0a9-88c9a4c797eb2ec1-0.caizheng.ailab-sys.svc.pjlab.local >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-11 10:43:37 -04:00
aoshen02
bd5bbce6e4
[RL] Release weight-transfer communicators without stopping inference ( #61060 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: fh3 <a@b>
2026-10-11 21:15:14 +08:00
tc-mb and Cyrus Leung
c85616efc6
[Model] Support MiniCPM-V 4.7 ( #58674 )
...
Signed-off-by: tc-mb <tianchi_cai@icloud.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-11 10:21:10 +00:00
Subhash Polisetti
f425a21208
[Bugfix][Frontend] Support scalar priority and wire priority into LLM.chat() ( #57243 )
...
Signed-off-by: subhash polisetti <subhashr161347@gmail.com >
2026-10-11 10:03:33 +00:00
2339bd39d6
[Bugfix][Mooncake] Use the EAGLE replay boundary when retaining sliding-window tails ( #61012 )
...
Signed-off-by: Stu Cao <stu@primeintellect.ai >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 09:15:18 +00:00
aoshen02 and OpenAI Codex
1789f09588
[Frontend] Explain disabled scale-out endpoints with opt-in hints ( #61042 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-11 08:58:07 +00:00
Juan Pérez de Algaba and Cyrus Leung
fcae5c9887
[Security] Restrict default CORS origins ( #61009 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-11 08:26:02 +00:00
Bruno Volpato and Codex
61f084b5f3
[Bugfix][Multimodal] Fix per-video cache option reuse ( #55203 )
...
Signed-off-by: bvolpato <brunocvcunha@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 06:21:55 +00:00
Yongye Zhu and Claude Opus 5.5
2eb491e18b
[Perf][MiniMax-M3] MSA prefill attends with the E4M3 query; bump fmha_sm100 to 968061a ( #60684 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 13:40:45 +08:00
Nick Hill
f8471164bf
[MRV2] Detatch weight-reload logic from MRV1 ( #60937 )
2026-10-10 21:15:05 -07:00
bzsuni
01abe975b5
[Bugfix][KV Connector] Fix speculative block reservations across KV cache groups ( #59880 )
...
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io >
2026-10-10 20:55:41 -07:00
Shuolei Wang and aoshen02
b89852aad0
[RL][Frontend] Return token-level weight-version spans on final outputs ( #59582 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Shuolei Wang <948904026@qq.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-11 11:33:29 +08:00
3c062d391d
[Bugfix] Support 0-dim tensors in packed weight transfer ( #52235 )
...
Signed-off-by: subhash polisetti <subhashr161347@gmail.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 01:41:06 +00:00
f2376cdc4d
[Fast Start] Preload the FlashInfer autotune table on the weight cache daemon ( #60085 )
...
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-10 15:48:06 +00:00
Mook
bf16c9c139
[Bugfix][Model] K-EXAONE: restore loading of pre-2.0 configs and apply RoPE to the MTP layer ( #60187 )
...
Signed-off-by: godmook <cmoh4135@naver.com >
Signed-off-by: Mook <68294499+Godmook@users.noreply.github.com >
2026-10-10 15:05:55 +00:00
Tony Lin
2bea22002d
[BUG] Fix KV offload shared-memory slot selection using worker rank ( #58497 )
...
Signed-off-by: Tony Lin <tony.lin@intel.com >
2026-10-10 20:25:11 +08:00
Kay Yan and Codex
c41b2639e2
[Bugfix][Responses API] Complete cache-write token accounting ( #60973 )
...
Signed-off-by: Kay Yan <kay.yan@daocloud.io >
Co-authored-by: Codex <noreply@openai.com >
2026-10-10 09:08:36 +00:00
wang.yuqi
c1692c85f1
[Misc] mv chat_utils.py to renderers. ( #60522 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-10 06:50:34 +00:00
SeongJun Lee and Isotr0py
699c892dd5
[Perf] Skip the M-RoPE warmup when every holder of the RoPE fuses it ( #60246 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-10 06:47:39 +00:00
Chauncey
fa5cfd48d5
[Refactor] Deduplicate trace header handling in BaseServing ( #60828 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-10-10 05:27:58 +00:00
Nick Hill
7d0b4e57aa
[ROCm] Remove MRV1 model exceptions ( #56360 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-10 05:09:19 +00:00
linitra24
364bc81a9e
[Bugfix][LoRA] Fix GPT-OSS gate/up interleaving with mixed MoE LoRA formats ( #60812 )
...
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-10 12:52:51 +08:00
Matthew Bonanni and Claude Opus 5.5
d5f356dae0
[HiSparse] Own GPU residency instead of editing core block-table rows ( #60647 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 21:25:44 -07:00
SeongJun Lee and Misha Goin
9bd074a996
[Attention] Decode an NVFP4 KV cache with XQA on SM12x ( #60452 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-09 22:24:00 -06:00
Misha Goin and Claude Opus 5.5
29e4d2d42e
[Frontend][Build] Add vllm download-kernels for FlashInfer precompiled kernels ( #58765 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 21:22:35 -07:00
Miłosz Grunwald and Kunshang Ji
be0823357f
[TEST][XPU][CI] Fix ci uva kernel test instability 2 ( #60404 )
...
Signed-off-by: microslaw <milosz.grunwald@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-10 12:09:13 +08:00
Giulio De Pasquale
f1a3c60558
[Feature][Attention] Cover a whole spec-verify in one DiffKV split-KV program ( #59085 )
...
Signed-off-by: Giulio De Pasquale <git@depasquale.giugl.io >
2026-10-10 11:55:45 +08:00
2d69083c4a
[ROCm][MXFP4] Align the AITER W4A4 MoE intermediate size to 128 ( #55104 )
...
Signed-off-by: xiaofei.zheng <xiaofei.zheng@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
2026-10-10 11:14:01 +08:00
DimensionSTP and OpenAI Codex
10cc2f6ae2
[Bugfix] Bound GDN speculative decode query widths ( #55504 )
...
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-10 11:00:16 +08:00
02aa10539f
[Bugfix][ModelLoader] Make enable_weights_track check quantized layers ( #57108 )
...
Signed-off-by: siliangchen-amd <SiLiang.Chen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-10 10:44:06 +08:00
aoshen02 and Claude Opus 5.5
1dfe9fbdae
[Core] Release FlashInfer allreduce workspace with --enable-nccl-comm-suspend ( #59360 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 02:26:40 +00:00
Fangzhou Ai and Claude Opus 5.5
193922d6ff
[ROCm][Perf][DSv4.1] Mono decode layer for DeepSeek-V4.1-Flash on MI355X ( #60397 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 20:53:18 -05:00
