 Rémi Delhaye
|
a1cd0ec650
|
[Model] Add zerank-2 reranker (#60415)
Signed-off-by: rdlhio <contact@rdlh.io>
|
2026-10-09 03:01:51 +00:00 |
|
|
zzaebokandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
702ce313f4
|
[Perf][Multimodal] Sort multimodal encoder inputs by modality (#60196)
Signed-off-by: Jaebok Lee <jaebok9541@naver.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-09 10:55:36 +08:00 |
|
|
Tiezhen WANGandKevin H. Luu
|
295c062db8
|
Remove sudo from dockerfile command (#60488)
Signed-off-by: Tiezhen WANG <38108242+xianbaoqian@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-08 19:24:56 -07:00 |
|
|
Mia OjedaandLucas Wilkinson
|
4611c2c3d2
|
[BugFix][DiffusionGemma][CPU] Copy async output snapshots instead of aliasing reused buffers (#59829)
Signed-off-by: ojeda-e <eojeda@league.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-10-08 19:16:51 -07:00 |
|
|
|
201e537d98
|
[Bugfix][LoRA] Validate incompatible MoE adapter formats (#59924)
Signed-off-by: Brian Weston <brianweston@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io>
|
2026-10-09 09:49:13 +08:00 |
|
|
Nishant RevurandClaude
|
581692cc0f
|
[Bugfix][Spec Decode] Fix multimodal embedding merge in Llama 4 and Mistral Large 3 EAGLE drafters (#57633)
Signed-off-by: Nishant Revur <kumrfni@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-09 01:44:50 +00:00 |
|
|
DGlan
|
4274ae956c
|
[Bugfix][EC Connector] Release shared memory on NIXL setup failure (#59875)
Signed-off-by: DGlan <162426117+DGlan@users.noreply.github.com>
|
2026-10-08 17:38:16 -07:00 |
|
|
|
ab905a885d
|
[CI] Run GH200 test on the shared arm64 CI image (#60253)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-08 15:22:20 -07:00 |
|
|
KamranandCodex
|
8d10c38b73
|
[Perf] Reduce fallback decode CPU by 69% at 32K (#60497)
Signed-off-by: spa5k <79936503+spa5k@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-10-08 21:42:56 +00:00 |
|
|
Yifan QiaoandClaude Opus 5.5
|
240785b82c
|
[Bugfix] Keep the GLM-5.3 kpool tail and Qwen4 QSA ring out of the null block (#59528)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-08 20:44:04 +00:00 |
|
|
sungbin1015andFlora Feng
|
2ddde19e8e
|
[Frontend] Support min_p in the Responses API (#48084)
Signed-off-by: sungbin1015 <sbin@solbox.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-08 20:38:17 +00:00 |
|
|
Divakar Verma
|
c5c1161382
|
[ROCm][Build] Fail fast when setup.py develop deps aren't pip-preinstalled (#59447)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
|
2026-10-08 15:28:38 -05:00 |
|
|
|
ae9adec905
|
[Frontend] Expose /v1/systemone without an opt-in flag (#60677)
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-08 20:21:05 +00:00 |
|
|
|
72d59adc2c
|
[Bugfix][KV Connector][NIXL] Transfer the whole PLE short-conv page in disaggregated P/D (#59997)
Signed-off-by: Alberto Massidda <amassidda@nvidia.com>
Co-authored-by: Alberto Massidda <amassidda@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
|
2026-10-08 23:58:13 +04:00 |
|
|
Matthew BonanniandClaude Opus 5.5
|
1afb029839
|
[Bugfix][HiSparse] Route by swap-row capacity, not the index-group workspace (#60682)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-08 19:45:44 +00:00 |
|
|
|
c23ca06b6d
|
[Bugfix][Structured Output] Feed the token that implicitly ends reasoning to the grammar (#59627)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Yifan Zong <263738792+yzong-rh@users.noreply.github.com>
|
2026-10-08 19:38:33 +00:00 |
|
|
dingdangmaoandFlora Feng
|
bab0ac61f8
|
[Bugfix][Frontend] Forward the reasoning wiring to the engine for batch chat completions (#59263)
Signed-off-by: apex-mochen <2756823972@qq.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-08 19:32:46 +00:00 |
|
|
Misha GoinandClaude
|
e3a29aa9b3
|
[Bugfix][MoE] Size the permute scratch from its inputs (#60447)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-08 15:32:07 -04:00 |
|
|
SeongJun LeeandMisha Goin
|
fff10d2072
|
[Attention] Allow an NVFP4 KV cache on SM90 with fa2 (#60620)
Signed-off-by: lesj0610 <lesj0610@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-08 12:16:22 -07:00 |
|
|
Kevin LiandMisha Goin
|
fc494822ea
|
[Test][Quantization] Gate CUTLASS W4A8 kernel tests to SM90 (#58878)
Signed-off-by: Kevin Li <kevinbean99@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-08 19:14:35 +00:00 |
|
|
Divakar VermaandAndreas Karatzas
|
18751bc6a7
|
[ROCm][CI] Add coverage for AiterExperts token padding: inf/nan garbage rows (#59334)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-10-08 19:11:45 +00:00 |
|
|
Aarushi JainandAndreas Karatzas
|
9bbc581560
|
[Bugfix] Close MessageQueue zmq resources via finalizer to avoid exit (#60472)
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-10-08 13:53:00 -05:00 |
|
|
Yeonwoo SungandFlora Feng
|
56e6da2c47
|
[Bugfix][Frontend] Honor stop_token_ids on the Responses API (#56592)
Signed-off-by: YeonwooSung <neos960518@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-08 18:47:30 +00:00 |
|
|
Ning XieandSimon Mo
|
c4a5ff5c18
|
[Frontend] shutdown engine core and its sub processes asap when initializing (#52299)
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
|
2026-10-08 11:46:01 -07:00 |
|
|
Shijin Zhang
|
2274e86842
|
[Spec Decode][Model] Support DFlash2 draft models with GLM-5.3-Flash (#56983)
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
|
2026-10-08 14:41:10 -04:00 |
|
|
Misha GoinandClaude
|
30c7405890
|
[MoE] MXFP4 oracle: add CUTLASS W4A4 backend and BF16 activation fallback (#60079)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-08 11:39:09 -07:00 |
|
|
|
d8fa463e2b
|
[Bugfix][Prefix caching] Fix inconsistent hybrid Mamba prefix cache boundaries (#60533)
Signed-off-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-08 18:33:48 +00:00 |
|
|
|
1664c4f444
|
[Bugfix][MiniMax-M3] Guard sparse decode sentinel indices (#54597)
Signed-off-by: Kyle Liang <kylliang@nvidia.com>
Signed-off-by: Kyle Liang <kyleliang-nv@users.noreply.github.com>
Co-authored-by: Kyle Liang <kyleliang-nv@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-08 18:27:57 +00:00 |
|
|
TialoandMisha Goin
|
972195f4f5
|
[CI][Build] build macos wheels using abi3 (#60312)
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-08 14:19:20 -04:00 |
|
|
Francisco Javier ArceoandMisha Goin
|
705e124c05
|
[Bugfix] Followup to #59299: preserve Unicode in structured decision state prompts (#60651)
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-08 11:00:03 -07:00 |
|
|
Doug Smith
|
242e4213fc
|
[Doc] Require GCC 13 for CUDA source builds (#60259)
Signed-off-by: Doug Smith <dosmith@redhat.com>
|
2026-10-08 09:43:39 -07:00 |
|
|
Elvir CrnčevićandMisha Goin
|
8674a794c7
|
[Perf] Enable Kimi K3 shared expert sharding with DeepEP v2 (#56875)
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-08 16:42:15 +00:00 |
|
|
stefankoncarevic
|
f4dde3132c
|
[CI][ROCm] Reuse AITER's prebuilt MoE module in the modular-kernel harness (#60633)
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
|
2026-10-08 11:18:05 -05:00 |
|
|
|
9f3d70e081
|
[Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) (#55686)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-08 12:04:46 -04:00 |
|
|
Mikko TukiainenandCursor
|
e2414ec1db
|
[ROCm][Qwen4Exp] Use fused PLE Triton kernels on the AMD backend (#60021)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-10-08 15:47:56 +00:00 |
|
|
Justin BarlowandHarry Mellor
|
0fc0cefb4e
|
[Bugfix] Don't int-sort attention-type names in --kv-cache-dtype-skip-layers for packed KV dtypes (#60364)
Signed-off-by: lobo235 <7@netlobo.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-10-08 16:34:55 +01:00 |
|
|
|
e41d5d5543
|
[Bugfix] Don't warn about missing watermark config for default requests (#60412)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-10-08 15:29:20 +00:00 |
|
|
Haokai MaandClaude
|
0751f5272f
|
[Bugfix] Scope deprecated CLI argument warnings to the parser that defines them (#59917)
Signed-off-by: Tyler Ma <158780265+TylerMa-debugg@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-08 15:29:04 +00:00 |
|
|
Woosuk Kwon
|
194bd72be7
|
[Compile] Remove torch.compile-based sequence parallelism and AsyncTP (#60517)
|
2026-10-08 08:28:23 -07:00 |
|
|
wangxiyuanandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
072d49bb6e
|
[Platform] Support platform-specific MLA prefill backend selection (#56959)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-08 15:25:21 +00:00 |
|
|
|
fd928e5f3c
|
[Bugfix] Check batch-invariance support for every attention backend (#60271)
Signed-off-by: yifanFengg <evanfengff@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-10-08 11:19:57 -04:00 |
|
|
JartX
|
994b8b77be
|
[Bugfix][ROCm] RDNA3 W4A16 MoE: handle the N-first CT weight layout (#60388)
Signed-off-by: JartX <sagformas@epdcenter.es>
|
2026-10-08 15:06:39 +00:00 |
|
|
kliuae
|
d80ddaac73
|
[ROCm][Perf] Kimi-K3 Store only the current rank's shards in latent MoE up-proj (#59591)
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
|
2026-10-08 14:55:20 +00:00 |
|
|
|
d33872dfeb
|
[Feat] Structured decisions endpoint (/v1/systemone) (#59299)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-10-08 14:48:12 +00:00 |
|
|
yhcheong (vllmellm)
|
273e099693
|
[Bugfix][ROCm] Select page-aligned kernel blocks for pooled indexers so the block table addresses storage pages (GLM-5.3-Flash) (#59412)
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
|
2026-10-08 14:44:58 +00:00 |
|
|
zzaebok
|
2f7f05c256
|
[Bugfix][MoRIIO] Prevent remote KV reload after decoder preemption (#53250)
Signed-off-by: Jaebok Lee <jaebok9541@naver.com>
|
2026-10-08 22:28:16 +08:00 |
|
|
|
daa9085143
|
[KV Connector][NIXL] Register HiSparse host pool under single TP Rank (#60161)
Signed-off-by: Pravein Govindan Kannan <pravein.govindan.kannan@ibm.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-08 07:25:02 -07:00 |
|
|
Matthew BonanniandClaude Opus 5.5
|
4ed4c4433b
|
[Bugfix][HiSparse] Drop the unaccounted, write-only prefill mirror staging buffer (#60429)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-08 14:00:49 +00:00 |
|
|
Vineeth Sai VarikuntlaandHarry Mellor
|
194da61de1
|
[Bugfix] Reject negative --device-ids indices instead of selecting the wrong device (#49053)
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-10-08 13:01:25 +00:00 |
|
|
|
458ba2edf8
|
[ROCm][Kimi-K3] Fuse AttnRes output with per-token FP8 input quantization (#59069)
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
|
2026-10-08 12:01:47 +00:00 |
|