5cadabfafd
[Docs] Add Qwen2.5-Coder-7B-Instruct to batch invariance tested models ( #46368 )
...
Signed-off-by: Yuval Luria <yluria@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-09 10:41:29 +00:00
Yan Ma
4e318ac563
[XPU] fix online fp8_per_channel quantization ( #56027 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-10-09 18:38:01 +08:00
Thang Nguyen and Claude Opus 5.5
efd141047d
[CI] Enroll a third batch of five steps in automatic sharding ( #60757 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 03:26:47 -07:00
Thang Nguyen and Claude Opus 5.5
b8bd335754
[CI] Enroll five more steps in automatic sharding ( #60662 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 03:26:27 -07:00
Warren Deng
59ffd73e85
[Bugfix][Kernel] Restore FP fusion for indexer Q quantization ( #60716 )
...
Signed-off-by: Warren Deng <warrdeng@meta.com >
2026-10-09 09:46:41 +00:00
Damien Laine and Cyrus Leung
2186b89a9a
[Bugfix][Model] Voxtral realtime: fix boot OOM and engine crash at max_model_len ( #45022 )
...
Signed-off-by: Damien Laine <damien.laine@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-09 17:40:29 +08:00
Harry Mellor and Claude Opus 5.5
2c99ee9333
[Chore] Remove vendored code made redundant by Transformers 5.16.1 ( #60623 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 09:29:03 +00:00
Chao Lei
bc713cf8cc
[Bugfix]MooncakeConnect add query timeout and retry ( #55923 )
2026-10-09 09:19:33 +00:00
Taewoong Kim and Chauncey
18c19eaf23
[Frontend] Add cache salt and tracing to structured decisions ( #60786 )
...
Signed-off-by: Taewoong Kim <ktw2172@gmail.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-09 08:53:50 +00:00
a6f43e57d6
[ROCm] Enable the cuMem CUDA-graph pool offload on ROCm ( #59523 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: indianspeedster <cspandey016@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: indianspeedster <cspandey016@gmail.com >
2026-10-09 08:29:55 +00:00
The Anh Nguyen
bacbbe1878
[Bugfix][NIXL] Guard record_transfer against None telemetry ( #57734 )
...
Signed-off-by: The Anh Nguyen <ntheanh201@gmail.com >
2026-10-09 08:22:18 +00:00
Sriram Kumar
560fae64cd
[ROCm][Bugfix] Pass per-sequence context lengths to the AITER paged MQA-logits kernels ( #59420 )
...
Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com >
2026-10-09 08:16:20 +00:00
Maroon Ayoub and Claude
8a03ccf7b3
[Frontend] Add return_mm_kwargs to render requests ( #60728 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-09 08:14:13 +00:00
f43e4c39e3
[Structured Output] Allow propertyNames with additionalProperties on xgrammar ( #57594 )
...
Signed-off-by: errmakov <ide404@gmail.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-09 10:49:47 +03:00
efaca9547d
[Perf][DSv4.1] Exchange packed FP8 Engram rows across DP and feed wkv MXFP8 ( #59923 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 15:30:18 +08:00
82e89e9944
[Perf][Qwen4Exp] Retune H200 M=4 merged QSA LL-GEMM plan ( #60776 )
...
Signed-off-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-10-09 07:18:50 +00:00
19c0bed584
[Fusion] Support 3D activation quantization fusion and enable it for Pixtral ( #59612 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-09 15:12:03 +08:00
dd30786fef
[CI] Enable Zen CPU image build on AMD CI ( #60016 )
...
Signed-off-by: Chinmay Kulkarni <Chinmay.Kulkarni@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
2026-10-09 01:40:21 -05:00
Paweł Mańczak and Kunshang Ji
f0a5f111f2
[XPU] Add tuned Mamba SSU configs for Intel Arc Pro B60 ( #56765 )
...
Signed-off-by: pmanczak <pawel.manczak@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-09 14:29:29 +08:00
ad73a4740d
[Perf][DSv4.1] Trim the last KV-source layer in decoder replay ( #59894 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Juntian Liu <Juntianl777@gmail.com >
2026-10-09 06:07:49 +00:00
8cbd5d0300
[Frontend] structured decisions: noul and score question types ( #60696 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Co-authored-by: DeepSeek <noreply@deepseek.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-09 04:48:16 +00:00
Rohan Potdar and Claude Opus 5.5
88a419649e
[ROCm][Kimi-K3] Use vLLM's FP8 max in the fused AttnRes quantization ( #60717 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 04:45:13 +00:00
fa312090cb
[Frontend] Accept each connection on the least loaded API worker ( #60723 )
...
Signed-off-by: hao-aaron <aaronhao@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 04:40:13 +00:00
Rajeev Warrier
81d9c45e4b
[Docs] [Multimodal] Note that per-request media_io_kwargs needs --trust-request-mm… ( #60598 )
...
Signed-off-by: Rajeev Warrier <warrier.rajeev.95@gmail.com >
2026-10-09 04:23:07 +00:00
e31b823bf5
[Bugfix][Tool Parser] Use LongCat tags in the LongCat structural tag ( #60747 )
...
Signed-off-by: hpdkhoa <hpdkhoa2311@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 21:20:54 -07:00
9824159d94
[Bugfix] Fix InternLM2 tool parser dropping characters from streamed arguments ( #60709 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 21:20:49 -07:00
Vineeth Sai Varikuntla and Flora Feng
112a282915
[Bugfix][Frontend] Keep streamed completion text_offset in step with tokens ( #60668 )
...
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 21:20:43 -07:00
51ef239768
[Rust Frontend] Carry reasoning controls over gRPC ( #59836 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 11:11:19 +08:00
Rémi Delhaye
a1cd0ec650
[Model] Add zerank-2 reranker ( #60415 )
...
Signed-off-by: rdlhio <contact@rdlh.io >
2026-10-09 03:01:51 +00:00
zzaebok and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
702ce313f4
[Perf][Multimodal] Sort multimodal encoder inputs by modality ( #60196 )
...
Signed-off-by: Jaebok Lee <jaebok9541@naver.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 10:55:36 +08:00
Tiezhen WANG and Kevin H. Luu
295c062db8
Remove sudo from dockerfile command ( #60488 )
...
Signed-off-by: Tiezhen WANG <38108242+xianbaoqian@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-08 19:24:56 -07:00
Mia Ojeda and Lucas Wilkinson
4611c2c3d2
[BugFix][DiffusionGemma][CPU] Copy async output snapshots instead of aliasing reused buffers ( #59829 )
...
Signed-off-by: ojeda-e <eojeda@league.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-08 19:16:51 -07:00
201e537d98
[Bugfix][LoRA] Validate incompatible MoE adapter formats ( #59924 )
...
Signed-off-by: Brian Weston <brianweston@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-09 09:49:13 +08:00
Nishant Revur and Claude
581692cc0f
[Bugfix][Spec Decode] Fix multimodal embedding merge in Llama 4 and Mistral Large 3 EAGLE drafters ( #57633 )
...
Signed-off-by: Nishant Revur <kumrfni@amazon.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-09 01:44:50 +00:00
DGlan
4274ae956c
[Bugfix][EC Connector] Release shared memory on NIXL setup failure ( #59875 )
...
Signed-off-by: DGlan <162426117+DGlan@users.noreply.github.com >
2026-10-08 17:38:16 -07:00
ab905a885d
[CI] Run GH200 test on the shared arm64 CI image ( #60253 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-08 15:22:20 -07:00
Kamran and Codex
8d10c38b73
[Perf] Reduce fallback decode CPU by 69% at 32K ( #60497 )
...
Signed-off-by: spa5k <79936503+spa5k@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
2026-10-08 21:42:56 +00:00
Yifan Qiao and Claude Opus 5.5
240785b82c
[Bugfix] Keep the GLM-5.3 kpool tail and Qwen4 QSA ring out of the null block ( #59528 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 20:44:04 +00:00
sungbin1015 and Flora Feng
2ddde19e8e
[Frontend] Support min_p in the Responses API ( #48084 )
...
Signed-off-by: sungbin1015 <sbin@solbox.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 20:38:17 +00:00
Divakar Verma
c5c1161382
[ROCm][Build] Fail fast when setup.py develop deps aren't pip-preinstalled ( #59447 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
2026-10-08 15:28:38 -05:00
ae9adec905
[Frontend] Expose /v1/systemone without an opt-in flag ( #60677 )
...
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-08 20:21:05 +00:00
72d59adc2c
[Bugfix][KV Connector][NIXL] Transfer the whole PLE short-conv page in disaggregated P/D ( #59997 )
...
Signed-off-by: Alberto Massidda <amassidda@nvidia.com >
Co-authored-by: Alberto Massidda <amassidda@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-08 23:58:13 +04:00
Matthew Bonanni and Claude Opus 5.5
1afb029839
[Bugfix][HiSparse] Route by swap-row capacity, not the index-group workspace ( #60682 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 19:45:44 +00:00
c23ca06b6d
[Bugfix][Structured Output] Feed the token that implicitly ends reasoning to the grammar ( #59627 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Yifan Zong <263738792+yzong-rh@users.noreply.github.com >
2026-10-08 19:38:33 +00:00
dingdangmao and Flora Feng
bab0ac61f8
[Bugfix][Frontend] Forward the reasoning wiring to the engine for batch chat completions ( #59263 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 19:32:46 +00:00
Misha Goin and Claude
e3a29aa9b3
[Bugfix][MoE] Size the permute scratch from its inputs ( #60447 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-08 15:32:07 -04:00
SeongJun Lee and Misha Goin
fff10d2072
[Attention] Allow an NVFP4 KV cache on SM90 with fa2 ( #60620 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 12:16:22 -07:00
Kevin Li and Misha Goin
fc494822ea
[Test][Quantization] Gate CUTLASS W4A8 kernel tests to SM90 ( #58878 )
...
Signed-off-by: Kevin Li <kevinbean99@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 19:14:35 +00:00
Divakar Verma and Andreas Karatzas
18751bc6a7
[ROCm][CI] Add coverage for AiterExperts token padding: inf/nan garbage rows ( #59334 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-08 19:11:45 +00:00
Aarushi Jain and Andreas Karatzas
9bbc581560
[Bugfix] Close MessageQueue zmq resources via finalizer to avoid exit ( #60472 )
...
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-08 13:53:00 -05:00
