9308 Commits
Author SHA1 Message Date
Zhenzhong XuandKunshang Ji 6a70913590 [XPU] bump up auto-round-lib to 0.16.0 (#60976)
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-10-12 14:47:59 +08:00
74f4404f58 [Bugfix][Distributed] Own SHM out-of-band buffers before releasing read slots (#61000)
Signed-off-by: Hubert Marek <163992334+hubert-marek@users.noreply.github.com>
Signed-off-by: Ao Shen <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Ao Shen <aoshen@inferact.ai>
2026-10-12 01:44:12 -04:00
Canlin GuoandClaude Opus 5.5 fe4df3df0e [Bugfix][GLM-5.3-Flash] Restore MTP index sharing across draft iterations (#61079)
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 05:41:52 +00:00
d4e92ccc37 [Bugfix][Frontend] Honor endpoint plugin route overrides (#58839)
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-10-12 03:58:15 +00:00
liuzhenweiandKunshang Ji bfa7e5b7af [XPU][CI] Enable XPU graph in graph tests (#60784)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-10-12 10:39:40 +08:00
Yan Ma f147949be8 [XPU] limit FA kernel block size to multipleOf 64 (#57297)
Signed-off-by: Yan Ma <yan.ma@intel.com>
2026-10-12 10:25:32 +08:00
dingdangmaoandCodex 124b89d5b7 [Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries (#59962)
Signed-off-by: apex-mochen <2756823972@qq.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-12 02:08:14 +00:00
Nick Hill 3fe917abb1 [Bugfix][MRV2] Fix FlashInfer attention and ReplaySSM warmups on MRV2 (#61013)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-10-11 16:04:41 -07:00
Jared WenClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f26d9f432c [Bugfix][GLM-5.3-Flash] Honor --mamba-ssm-cache-dtype for the KDA recurrent state (#59147)
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-11 17:55:00 +00:00
Nick Hill c059bee812 [MRV2] Switch CI tests still running on MRV1 unnecessarily (#61014)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-10-11 09:57:07 -07:00
Ting SUNandCyrus Leung b99fdbe1f8 [Bugfix] Respect the configured offline chat template (#61061)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-10-11 16:27:21 +00:00
93d2cab35e [Security] Fix P2P ZMQ socket exhaustion via unbounded peer sessions (#51504)
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
2026-10-11 11:39:11 -04:00
aoshen02andCodex 239a3fa3a8 [Sleep] Retain draft-private weights with existing parameter selector (#61046)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
2026-10-11 23:30:56 +08:00
3e8e1364ff [Perf][GLM-5.3-Flash] Shard long-context indexer prefill rows across TP (#54951)
Signed-off-by: Zheng Cai <8370601+zigzagcai@users.noreply.github.com>
Signed-off-by: Zheng Cai <caizheng1993@gmail.com>
Signed-off-by: Codex <noreply@openai.com>
Co-authored-by: root <root@rjob-fb29567b88fec0a9-88c9a4c797eb2ec1-0.caizheng.ailab-sys.svc.pjlab.local>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
2026-10-11 10:43:37 -04:00
aoshen02 bd5bbce6e4 [RL] Release weight-transfer communicators without stopping inference (#61060)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
2026-10-11 21:15:14 +08:00
tc-mbandCyrus Leung c85616efc6 [Model] Support MiniCPM-V 4.7 (#58674)
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-10-11 10:21:10 +00:00
Subhash Polisetti f425a21208 [Bugfix][Frontend] Support scalar priority and wire priority into LLM.chat() (#57243)
Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
2026-10-11 10:03:33 +00:00
2339bd39d6 [Bugfix][Mooncake] Use the EAGLE replay boundary when retaining sliding-window tails (#61012)
Signed-off-by: Stu Cao <stu@primeintellect.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
2026-10-11 09:15:18 +00:00
aoshen02andOpenAI Codex 1789f09588 [Frontend] Explain disabled scale-out endpoints with opt-in hints (#61042)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-10-11 08:58:07 +00:00
Juan Pérez de AlgabaandCyrus Leung fcae5c9887 [Security] Restrict default CORS origins (#61009)
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-10-11 08:26:02 +00:00
Bruno VolpatoandCodex 61f084b5f3 [Bugfix][Multimodal] Fix per-video cache option reuse (#55203)
Signed-off-by: bvolpato <brunocvcunha@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-11 06:21:55 +00:00
Yongye ZhuandClaude Opus 5.5 2eb491e18b [Perf][MiniMax-M3] MSA prefill attends with the E4M3 query; bump fmha_sm100 to 968061a (#60684)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 13:40:45 +08:00
Nick Hill f8471164bf [MRV2] Detatch weight-reload logic from MRV1 (#60937) 2026-10-10 21:15:05 -07:00
bzsuni 01abe975b5 [Bugfix][KV Connector] Fix speculative block reservations across KV cache groups (#59880)
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
2026-10-10 20:55:41 -07:00
Shuolei Wangandaoshen02 b89852aad0 [RL][Frontend] Return token-level weight-version spans on final outputs (#59582)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
2026-10-11 11:33:29 +08:00
3c062d391d [Bugfix] Support 0-dim tensors in packed weight transfer (#52235)
Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
2026-10-11 01:41:06 +00:00
f2376cdc4d [Fast Start] Preload the FlashInfer autotune table on the weight cache daemon (#60085)
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-10-10 15:48:06 +00:00
Mook bf16c9c139 [Bugfix][Model] K-EXAONE: restore loading of pre-2.0 configs and apply RoPE to the MTP layer (#60187)
Signed-off-by: godmook <cmoh4135@naver.com>
Signed-off-by: Mook <68294499+Godmook@users.noreply.github.com>
2026-10-10 15:05:55 +00:00
Tony Lin 2bea22002d [BUG] Fix KV offload shared-memory slot selection using worker rank (#58497)
Signed-off-by: Tony Lin <tony.lin@intel.com>
2026-10-10 20:25:11 +08:00
Kay YanandCodex c41b2639e2 [Bugfix][Responses API] Complete cache-write token accounting (#60973)
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
Co-authored-by: Codex <noreply@openai.com>
2026-10-10 09:08:36 +00:00
wang.yuqi c1692c85f1 [Misc] mv chat_utils.py to renderers. (#60522)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
2026-10-10 06:50:34 +00:00
SeongJun LeeandIsotr0py 699c892dd5 [Perf] Skip the M-RoPE warmup when every holder of the RoPE fuses it (#60246)
Signed-off-by: lesj0610 <lesj0610@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-10-10 06:47:39 +00:00
Chauncey fa5cfd48d5 [Refactor] Deduplicate trace header handling in BaseServing (#60828)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-10-10 05:27:58 +00:00
Nick Hill 7d0b4e57aa [ROCm] Remove MRV1 model exceptions (#56360)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-10-10 05:09:19 +00:00
linitra24 364bc81a9e [Bugfix][LoRA] Fix GPT-OSS gate/up interleaving with mixed MoE LoRA formats (#60812)
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
2026-10-10 12:52:51 +08:00
Matthew BonanniandClaude Opus 5.5 d5f356dae0 [HiSparse] Own GPU residency instead of editing core block-table rows (#60647)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:25:44 -07:00
SeongJun LeeandMisha Goin 9bd074a996 [Attention] Decode an NVFP4 KV cache with XQA on SM12x (#60452)
Signed-off-by: lesj0610 <lesj0610@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
2026-10-09 22:24:00 -06:00
Misha GoinandClaude Opus 5.5 29e4d2d42e [Frontend][Build] Add vllm download-kernels for FlashInfer precompiled kernels (#58765)
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:22:35 -07:00
Miłosz GrunwaldandKunshang Ji be0823357f [TEST][XPU][CI] Fix ci uva kernel test instability 2 (#60404)
Signed-off-by: microslaw <milosz.grunwald@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-10-10 12:09:13 +08:00
Giulio De Pasquale f1a3c60558 [Feature][Attention] Cover a whole spec-verify in one DiffKV split-KV program (#59085)
Signed-off-by: Giulio De Pasquale <git@depasquale.giugl.io>
2026-10-10 11:55:45 +08:00
2d69083c4a [ROCm][MXFP4] Align the AITER W4A4 MoE intermediate size to 128 (#55104)
Signed-off-by: xiaofei.zheng <xiaofei.zheng@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
2026-10-10 11:14:01 +08:00
DimensionSTPandOpenAI Codex 10cc2f6ae2 [Bugfix] Bound GDN speculative decode query widths (#55504)
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-10-10 11:00:16 +08:00
02aa10539f [Bugfix][ModelLoader] Make enable_weights_track check quantized layers (#57108)
Signed-off-by: siliangchen-amd <SiLiang.Chen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
2026-10-10 10:44:06 +08:00
aoshen02andClaude Opus 5.5 1dfe9fbdae [Core] Release FlashInfer allreduce workspace with --enable-nccl-comm-suspend (#59360)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 02:26:40 +00:00
Fangzhou AiandClaude Opus 5.5 193922d6ff [ROCm][Perf][DSv4.1] Mono decode layer for DeepSeek-V4.1-Flash on MI355X (#60397)
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 20:53:18 -05:00
3c802af3cf [Feat][Frontend] structured decisions: DiffusionGemma read strategy (#60466)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
2026-10-10 09:33:22 +08:00
Nick Hill 7d1d8660d6 [MRV2] Miscellaneous code structure updates (#60754)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-10-09 17:58:09 -07:00
9f1c783886 [Test] Allow one fp8 step on K bytes in the kpool decode fuzz test (#60725)
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-10-10 00:26:53 +00:00
f8e6e365ae [Bugfix][Mooncake] Save the EAGLE attention proof with hybrid Mamba partial tails (#60695)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Signed-off-by: Dao007forever <daole@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
2026-10-10 00:08:08 +00:00
Wei ZhaoandCodex 03a7d3776f [Perf] Support minimum KV splits configuration for TokenSpeed MLA decode (#60345)
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 23:33:47 +00:00