|
a9fe30e9f6
|
[Bugfix][Multimodal] Recover PyNvVideoCodec decoder slots after failures (#59597)
Signed-off-by: Daniel Afrimi <dafrimi@nvidia.com>
Signed-off-by: danielafrimi <45691845+danielafrimi@users.noreply.github.com>
Signed-off-by: Daniel Afrimi <dafrimi@oci-hsg-cs-001-login-02.cm.cluster>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Daniel Afrimi <dafrimi@oci-hsg-cs-001-login-02.cm.cluster>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
|
2026-10-15 09:17:19 +03:00 |
|
|
liranschourandKevin H. Luu
|
c5f8adc0b1
|
[KV Offload] Expose OffloadingManager.lock()/unlock() and service the tiering control plane off the scheduler thread (#58168)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 08:50:56 +03:00 |
|
|
HarshilandCyrus Leung
|
c54dca96bd
|
[Bugfix][Benchmark] Fix probe latency and duration in bench serve (#60511)
Signed-off-by: QHarshil <harshil_c@hotmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-15 05:50:30 +00:00 |
|
|
|
5341b5ba7d
|
[KV Offload] Pin only each worker's own offload slots (#59544)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-10-15 08:49:42 +03:00 |
|
|
|
6605802904
|
[Perf][KV Connector][Mooncake] Send the KV layout once per peer (#59825)
Signed-off-by: staryxchen <staryxchen@tencent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-10-15 05:36:31 +00:00 |
|
|
|
3f1bcfd82b
|
[Bugfix][KV Offload] Bound unanswered P2P lookup waits (#56836)
Signed-off-by: jacklin78911-collab <jacklin78911@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 08:23:34 +03:00 |
|
|
|
81e1c3f183
|
[XPU][MLA] Integrate XPU MLA support into the existing FLASH_ATTN_MLA backend (#58297)
Signed-off-by: baodii <di.bao@intel.com>
Signed-off-by: Zhouxiaoya25 <xiaoya.zhou@intel.com>
Signed-off-by: ZhouXiaoya25 <xiaoya.zhou@intel.com>
Co-authored-by: baodii <di.bao@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-10-15 12:47:37 +08:00 |
|
|
Vivaan SharmaandKevin H. Luu
|
96b5b0ee4b
|
[Model] Enable torch.compile and CUDA graphs for EmbeddingGemma2 (#61572)
Signed-off-by: Vivaan Sharma <99265792+Vivaansharma2003@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 12:32:39 +08:00 |
|
|
Haiyu ZuoandKevin H. Luu
|
ff5a91a456
|
[Feature][Model Loader] Apply model-owned MTP shard selection to RunAI ModelStreamer (#60759)
Signed-off-by: 浩韵 <zuohaiyu.zhy@alibaba-inc.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 12:23:11 +08:00 |
|
|
Chaitanya Sri Krishna LollaandCursor
|
833665cded
|
[ROCm][DI] Enable MoRI FP4 dispatch for mxfp4 w4a4 moe (#59596)
Signed-off-by: lcskrishna <lollachaitanya@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-10-14 23:20:14 -05:00 |
|
|
Tuukka SarviandKevin H. Luu
|
10d2c6083d
|
[ROCm][DSv4.1] Enable DSpark adaptive verification (#58151)
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 12:05:43 +08:00 |
|
|
Shanshan Shen
|
f46021a7fa
|
[ROCm][Perf] Enable DPA+ETP prefill compute/comm dual-batch-overlap for DeepSeek-V4 (#57773)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
Signed-off-by: shen-shanshan <467638484@qq.com>
|
2026-10-15 11:16:48 +08:00 |
|
|
FeanorandTaneem Ibrahim
|
5d5c3d7c43
|
[Bugfix] Preserve padding token types for cross encoders (#58143)
Signed-off-by: zhaoye <772971548@qq.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-10-15 03:09:24 +00:00 |
|
|
|
429a3da665
|
[Bugfix][ROCm][Disagg] Allow MoRIIO HMA per-layer block_len (#58585)
Signed-off-by: avininjamay8 <Avinash.Paul@amd.com>
Signed-off-by: avininjamay8 <avpaul@amd.com>
Co-authored-by: avininjamay8 <avpaul@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
|
2026-10-15 09:35:31 +08:00 |
|
|
KaiandKunshang Ji
|
de2b869ee9
|
[Bugfix][Kernel] Register int8 scaled_mm as a custom op (#57331)
Signed-off-by: Kai <limkc.reply@gmail.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-10-15 09:27:49 +08:00 |
|
|
Woosuk KwonandClaude Opus 5.5
|
e57bb4ace7
|
[Core] Remove the legacy Ray compiled-graph executor (#61712)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 18:13:27 -07:00 |
|
|
|
bfdbfa1632
|
[Bugfix][PCP] Align MoE routing padding with gathered tokens (#60946)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: wangyicong <wangyicong@bytedance.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-10-14 21:04:00 -04:00 |
|
|
Woosuk KwonandClaude Opus 5.5
|
8530aed2b7
|
[Spec Decode] Remove Medusa and MLP speculator support (#61701)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 17:29:23 -07:00 |
|
|
Rohan PotdarandClaude Opus 5.5
|
9a648b4649
|
[ROCm][Kimi-K3] Don't read W_UK_T before weight loading in the MLA wrapper (#61654)
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 21:44:55 +00:00 |
|
|
Ben BrowningandFlora Feng
|
4c5ea86f53
|
[Bugfix][Parser] Stream Gemma4 string tool arguments incrementally (#61494)
Signed-off-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-14 21:25:41 +00:00 |
|
|
|
984c242ede
|
[Quantization] Support compressed-tensors MXFP4 MoE with FP8 activations via Mxfp4MoEMethod (#59815)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-14 13:33:35 -07:00 |
|
|
|
fe98d90a9d
|
[Dependency] Update Humming and remove NVFP4 scale workaround (#61556)
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-14 13:32:36 -07:00 |
|
|
Woosuk KwonandClaude Opus 5.5
|
2e8a5930f6
|
[Benchmark] Remove the temporary bench forward dummy loader (#61676)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 13:30:18 -07:00 |
|
|
Juntian Liu
|
7c3f7e5500
|
[Perf][DSv4.1] Length-aware candidate block selection in the decode indexer (#61039)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
|
2026-10-14 20:15:58 +00:00 |
|
|
rasmithandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2622ee21aa
|
[ROCm][BugFix] Don't use AITER rms_norm as an IR OP on gfx90a when AITER always on. (#59290)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-14 12:48:37 -07:00 |
|
|
alpandmottopanikeiku
|
e73fa00981
|
[Bugfix][Frontend] Apply the JSON schema in Cohere Chat v2 response_format (#60586)
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
|
2026-10-14 19:42:20 +00:00 |
|
|
Juntian LiuandClaude Opus 5.5
|
388a610dfb
|
[Perf][DFlash] Capture context combine and anchor in the draft CUDA graph (#59511)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-14 15:40:52 -04:00 |
|
|
Wei ZhaoandJee Jee Li
|
4975fda882
|
[Dependency] Upgrade FlashInfer to 0.7.1 (#60948)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-10-14 12:31:08 -07:00 |
|
|
Ayushman Singh
|
9b5c6707a3
|
[Bugfix][MRV2] Capture DCP FULL cudagraphs with KV context (#61496)
Signed-off-by: Ayushman Singh <40520701+ayush1399@users.noreply.github.com>
|
2026-10-14 19:20:48 +00:00 |
|
|
Andreas KaratzasandClaude
|
e96c822660
|
[Bugfix][ROCm][MiniMax-M3] Reject unbuilt index-head counts in the AITER indexer (#60990)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-14 14:13:48 -05:00 |
|
|
Sandeep AgrawalandKevin H. Luu
|
21e207efa1
|
[Bugfix][MRV2][Spec Decode] Honor EAGLE3 single-state draft configuration (#61563)
Signed-off-by: Sandeep Agrawal <sandeepa@nvidia.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-14 12:09:06 -07:00 |
|
|
amd-xavierwangandJartX
|
06265beb25
|
[ROCm] Enable fp8 query for Triton attention (#60296)
Signed-off-by: xawang <xawang@amd.com>
Co-authored-by: JartX <sagformas@epdcenter.es>
|
2026-10-14 15:03:20 -04:00 |
|
|
|
857ddac4e5
|
[Perf][MRV2] Skip empty draft-token resets on request admission (#61640)
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-14 12:01:01 -07:00 |
|
|
|
c1407a7e86
|
[Spec Decode][MRV2] Warm up top-k/top-p and greedy rejection kernels for speculative decoding (#59260)
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-14 14:59:21 -04:00 |
|
|
simon-veitner-redhatandClaude
|
ae94e9c9e8
|
[Bugfix] Don't crash DeepGEMM MegaMoE warmup when get_bf16_mega_gate_config is missing (#61657)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-14 18:58:35 +00:00 |
|
|
Simon Danielsson
|
3f729dafcc
|
[Perf][ROCm][GLM-5.3-Flash] AITER prefill indexer topk (#60753)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
|
2026-10-14 13:48:56 -05:00 |
|
|
lucas-santos-amdandDouglas Lehr
|
e534024e49
|
[ROCm][Bugfix][MLA] Fix FP8 MLA prefill unpad for interleave-padded heads (#60907)
Signed-off-by: Lucas Santos <Lucas.Santos@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
|
2026-10-14 14:34:28 -04:00 |
|
|
|
d1d8e6c096
|
[Perf][Nemotron-Parse] Do image rescale/normalize on the device (#60500)
Signed-off-by: DiegoCao <DiegoCao@users.noreply.github.com>
Co-authored-by: DiegoCao <DiegoCao@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 18:33:22 +00:00 |
|
|
Robert ShawandClaude
|
d3f6b2a7b3
|
[Frontend] Clean Client Facing 500 Report (#58043)
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-14 14:27:58 -04:00 |
|
|
Woosuk KwonandClaude Opus 5.5
|
9d3dfa2886
|
[Benchmark] Add vllm bench forward: exact-shape forward timing (#61339)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 10:52:26 -07:00 |
|
|
TaimysandOpenAI Codex
|
2c74fd1873
|
[Model Runner V2] Enable batch-sharded sampling for GPT-OSS (#61257)
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-10-14 10:11:11 -07:00 |
|
|
Xunzhuo
|
89cab55086
|
[Model] Support Decision 3.0 decision models (vllm-sr/d3) (#61565)
Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com>
|
2026-10-14 17:10:55 +00:00 |
|
|
zexplorerhjandCodex
|
3895a8babe
|
[Bugfix] Fix TP loading of small block-FP8 MLA gates in BailingMoeV3 (#61642)
Signed-off-by: zexplorerhj <zhjoneson@163.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-14 10:09:23 -07:00 |
|
|
Harry MellorandClaude Opus 5.5
|
3e851a7803
|
[Bugfix][Attention] Make the query contiguous for TRTLLM ragged MLA prefill (#61634)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 16:38:52 +00:00 |
|
|
Zheng GongandCursor
|
355553ac22
|
[ROCm][Perf] Drop two dead per-step ops from the AITER sparse MLA metadata build (#58354)
Signed-off-by: ZhengGong-amd <zgong@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-10-14 11:15:51 -05:00 |
|
|
|
13dbf96093
|
[Model Loader] Apply EP weight filtering in InstantTensor (#61167)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-14 09:10:10 -07:00 |
|
|
Nick HillandIlia Yastrebov
|
eaf25f675a
|
[NIXL] Prepare KV transfer descriptors as NIXL 1.5 strided lists (#61032)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Ilia Yastrebov <iyastrebov@nvidia.com>
|
2026-10-14 18:07:24 +02:00 |
|
|
Nan ZhangandRobert Shaw
|
9e2b579aab
|
[Model] Support EAGLE3/DFlash aux hidden states for GraniteMoe and GraniteMoeShared (#61110)
Signed-off-by: ZN1010 <neilzhang1010@126.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-10-14 12:01:14 -04:00 |
|
|
|
e2aabc8b15
|
[Bugfix][KV Offload][P2P] Keep store pins and AbortAck until NIXL transfers stop (#59790)
Signed-off-by: apex-mochen <2756823972@qq.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-14 18:31:31 +03:00 |
|
|
Juan Pérez de AlgabaandOpenAI Codex
|
5d88c034f6
|
[Security] Reject local verifier references in Speculators metadata (#61587)
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-10-14 15:30:20 +00:00 |
|