1bc6e072b0
[Rust Frontend] Add per-request timing metrics to chat and completions ( #59314 )
...
Signed-off-by: wxin <wxin@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 15:39:10 -07:00
802feb6770
[Bugfix][KVConnector] Fix MoRI-IO WRITE-mode requests stranded in WAITING_FOR_REMOTE_KVS ( #57705 )
...
Signed-off-by: Akshay Viswanathan <4953343+akshayv@users.noreply.github.com >
Co-authored-by: Akshay Viswanathan <4953343+akshayv@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com >
2026-10-12 22:29:31 +00:00
Maroon Ayoub and Flora Feng
44f0ddfd7b
[Bugfix][Parser] Stop re-parsing the whole GLM tool argument per chunk ( #61253 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-12 21:28:35 +00:00
Mikko Tukiainen
abad32a718
[ROCm][Perf] Fix Gemma AR+RMS prefill fallback ( #58547 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Signed-off-by: Mikko Tukiainen <mikko.tukiainen@amd.com >
2026-10-12 19:53:08 +00:00
Martin Hickey
7039c7f17b
[Bugfix][Rust Frontend] Omit unset sampling params in render output ( #61220 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-12 19:31:03 +00:00
ztysth and OpenAI Codex
1bdd3fa2d5
[Bugfix][Core] Check multimodal chunk boundaries before encoder cache reuse ( #59721 )
...
Signed-off-by: ztysth <rnk.liulw@gmail.com >
Signed-off-by: ztysth <166926442+ztysth@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-12 19:25:35 +00:00
Martin Hickey
08582942d8
[Bugfix][Frontend] Honor request_id and X-Request-Id in chat and completions render ( #61236 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-12 19:17:12 +00:00
Wentao Ye
d0948f754d
[Bug] Honor spec decode draft model config ( #61240 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-12 15:13:23 -04:00
8055293353
[ROCm][MLA] Cut ~21x host dispatch from the MLA metadata build ( #58381 )
...
Signed-off-by: lifulu <fululi12@amd.com >
Signed-off-by: lifulu <lifulu@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: lifulu <lifulu@users.noreply.github.com >
2026-10-12 13:11:20 -05:00
Woosuk Kwon and Claude Opus 5.5
e1a5086256
[Attention] Remove cascade attention ( #61126 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 10:41:07 -07:00
5b0b8f251e
[Bugfix][HiSparse] Fix TP group hang on steps without scheduled tokens ( #60941 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 17:38:12 +00:00
a7b7bc1fbe
[Metrics] Count finished requests by watermark status ( #58715 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-12 17:21:22 +00:00
Misha Goin and Claude Opus 5.5
45bb710eca
[CI] Add nightly Python 3.14 smoke test ( #60456 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 10:11:26 -07:00
Chester
c599b5971f
[Bugfix][Frontend] Return all logprobs when completions requests logprobs=-1 ( #55717 )
...
Signed-off-by: Chester <42577861+superdiaodiao@users.noreply.github.com >
2026-10-12 17:05:55 +00:00
haotiaa
6e6e86078a
[Bugfix][Platform] Don't resolve current_platform during import vllm ( #60766 )
...
Signed-off-by: Haotian An <haotiaa@amazon.com >
2026-10-12 17:05:34 +00:00
fa39582756
[Frontend] Serve LiquidAI/d1-3B on /v1/systemone ( #60676 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 13:00:47 -04:00
Misha Goin and Claude
d79b52b182
[Misc] Consolidate csrc shared headers and prune dead code ( #60721 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 09:46:22 -07:00
e0317b28ce
[Feature] Support batch invariance on ROCm ( #52231 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-12 10:53:26 -05:00
TJian
50b21a2667
[Perf] [Feat] [ROCm] Add densemha support to ROCm AITER Sparse MLA ( #49263 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-10-12 15:50:28 +00:00
Shang Wang and Claude Opus 5.5
ffec221276
[Bugfix][DSv4.1] Match decoder-replay input_ids dtype for FlashInfer MegaMoE ( #61041 )
...
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 14:23:00 +00:00
Shang Wang and Claude Opus 5.5
716961cf1f
[Bugfix] Drop enable_in_kernel_fc2_reduce from FlashInfer CuteDSL MegaMoE config ( #59900 )
...
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 09:05:09 -05:00
Ilya Markov
a281ad32f0
[EPLB] Add EPLB support to dspark draft models (DeepSeekV4) ( #57549 )
...
Signed-off-by: Markov Ilya <markovilya197@gmail.com >
2026-10-12 10:03:09 -04:00
Ilya Markov
b3ee560ae8
[Bugfix][MoE] Apply expert_map remapping for CUTLASS FP4 MOE expert parallel ( #59042 )
...
Signed-off-by: Markov Ilya <markovilya197@gmail.com >
2026-10-12 10:02:56 -04:00
bzsuni
f022975964
[Bugfix] Reject beam search in completion render ( #61212 )
...
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io >
2026-10-12 13:49:46 +00:00
4c36d21280
[Bugfix][LoRA] Apply PEFT rank_pattern / alpha_pattern scaling per module ( #59801 )
...
Signed-off-by: suraj <surajsharan85@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-12 21:49:30 +08:00
3facf822b7
[Performance] Load HF tokenizers and processors from a local snapshot ( #60825 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-12 13:45:54 +00:00
092c35dd15
[Model] Support Cloudflare/clef-omni on /v1/systemone ( #61064 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-12 21:27:34 +08:00
DimensionSTP and Artem Perevedentsev
314ab8cffd
[Bugfix][Structured Output] Drain compilation tasks before releasing the backend ( #61047 )
...
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-12 16:25:36 +03:00
stefankoncarevic
8c175f403a
[CI][ROCm] Compare greedy output only in the AITER attn+quant test ( #61185 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-12 08:20:35 -05:00
Rohan Potdar and Claude Opus 5.5
7900d2eabc
[Test] Accept tied tokens at the k-th value in the Triton top-k test ( #60726 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 08:18:44 -05:00
Harry Huang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
c75cb63cd2
[Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next ( #56322 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-12 20:56:44 +08:00
Tianyu Guo
258b97ff6b
[Bugfix][EC Connector] Size transfers by measured encoder output widths ( #57904 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-10-12 12:32:39 +00:00
8d7b93fec2
[Frontend] Report stop_reason on /inference/v1/generate and pass it through derender ( #60251 )
...
Signed-off-by: Hyeonki Hong <hhk7734@gmail.com >
Signed-off-by: Shimi Bandiel <shimib@google.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Hyeonki Hong <hhk7734@gmail.com >
Co-authored-by: Mimir <mimir@nousresearch.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-12 19:37:53 +08:00
MatthewCYM
f7a3a7265f
[Bugfix][Multimodal] Fix Qwen3-ASR audio preprocessing parity ( #57146 )
...
Signed-off-by: MatthewCYM <chenyiming0v0@gmail.com >
2026-10-12 10:42:51 +00:00
Shijin Zhang and Chauncey
83773ecdf3
[Frontend] Support CustomToolCall in the non-Harmony Responses API ( #57322 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-12 17:22:55 +08:00
Thuongvu Ho and Nicolò Lucchesi
99b44c7a42
[Metrics] Add admission rejection metrics ( #55812 )
...
Signed-off-by: Thuongvu Ho <thuongvuho@gmail.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
2026-10-12 08:31:07 +00:00
Ting SUN
f4114e068e
[Bugfix][Frontend] Preserve token IDs in chat derender responses ( #61065 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-10-12 07:55:06 +00:00
AMD-yanfeiwang and aoshen02
e3bbaf2369
[ROCm][DSv4] Fuse the shared expert into AITER MegaMoE ( #61004 )
...
Signed-off-by: Yanfei Wang <yanfei.wang@amd.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-12 15:53:16 +08:00
Giancarlo Delfin and Rishi Puri
36f3972812
[Spec Decode] Skip metadata build for dflash/dspark when all model backends support it ( #60692 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Rishi Puri <20074092+puririshi98@users.noreply.github.com >
2026-10-12 00:09:10 -07:00
Zhenzhong Xu and Kunshang Ji
6a70913590
[XPU] bump up auto-round-lib to 0.16.0 ( #60976 )
...
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-12 14:47:59 +08:00
74f4404f58
[Bugfix][Distributed] Own SHM out-of-band buffers before releasing read slots ( #61000 )
...
Signed-off-by: Hubert Marek <163992334+hubert-marek@users.noreply.github.com >
Signed-off-by: Ao Shen <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Ao Shen <aoshen@inferact.ai >
2026-10-12 01:44:12 -04:00
Canlin Guo and Claude Opus 5.5
fe4df3df0e
[Bugfix][GLM-5.3-Flash] Restore MTP index sharing across draft iterations ( #61079 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 05:41:52 +00:00
d4e92ccc37
[Bugfix][Frontend] Honor endpoint plugin route overrides ( #58839 )
...
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-12 03:58:15 +00:00
liuzhenwei and Kunshang Ji
bfa7e5b7af
[XPU][CI] Enable XPU graph in graph tests ( #60784 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-12 10:39:40 +08:00
Yan Ma
f147949be8
[XPU] limit FA kernel block size to multipleOf 64 ( #57297 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-10-12 10:25:32 +08:00
dingdangmao and Codex
124b89d5b7
[Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries ( #59962 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-12 02:08:14 +00:00
Nick Hill
3fe917abb1
[Bugfix][MRV2] Fix FlashInfer attention and ReplaySSM warmups on MRV2 ( #61013 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-11 16:04:41 -07:00
f26d9f432c
[Bugfix][GLM-5.3-Flash] Honor --mamba-ssm-cache-dtype for the KDA recurrent state ( #59147 )
...
Signed-off-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-11 17:55:00 +00:00
Nick Hill
c059bee812
[MRV2] Switch CI tests still running on MRV1 unnecessarily ( #61014 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-11 09:57:07 -07:00
Ting SUN and Cyrus Leung
b99fdbe1f8
[Bugfix] Respect the configured offline chat template ( #61061 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-11 16:27:21 +00:00
