Commit Graph
1080 Commits
Author SHA1 Message Date
Harry MellorandClaude Opus 5 8a5cf54387 [Multimodal] Type dummy options per modality (#57576)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 17:29:11 +01:00
Kevin Liandkevinli 911548247a [Distributed][MoonEP] BF16 integration of MoonEP balanced EP backend (#52101)
Signed-off-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com>
Co-authored-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com>
2026-09-18 10:34:01 -04:00
Andy LoandOpenAI Codex 09c379ab61 [Bugfix] Max-load throughput cliff when max_num_seqs is not a multiple of 8 (#57355)
Signed-off-by: Andy Lo <andy@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-17 18:09:18 +00:00
Qiming Zhang e6b1a5e5e3 [XPU] Enable XPU eplb (#44987)
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com>
2026-09-17 13:19:13 +08:00
2743dc4f92 [Feature][Multimodal] Add cross-encoder caching to Mooncake P2P (#56242)
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-17 13:16:41 +08:00
Chuan (Richard) LiandCursor f730a93d2b [Scheduler] Add --max-num-active-seqs to cap RUNNING admission (#56758)
Signed-off-by: Li <chuali@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 10:22:12 +08:00
Robert ShawandRobert Shaw ed769b4a45 [A2A] Add DeepEPv2 to SP Supported List (#57210)
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
2026-09-16 21:29:10 -04:00
c58ff535f4 [Bugfix][Spec Decode] Fix EAGLE and dense draft startup with EP (#56930)
Signed-off-by: Kyle Liang <kyleliang-nv@users.noreply.github.com>
Co-authored-by: Kyle Liang <kyleliang-nv@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-16 17:03:36 -07:00
jxjandHarry Mellor 6b6bf6b3c3 [Bugfix] Remove unsupported comma-separated detailed trace values (#55702)
Signed-off-by: git-jxj <65210887+git-jxj@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-09-16 18:44:48 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
f37c550bf6 [Feature][Frontend] Expose effective attention block size for DCP (#56538)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-09-16 08:29:43 +00:00
Giancarlo Delfin 85c1f58d50 [BugFix][Model Runner V2][Spec Decode] Fix decode instance's multi-layer MTP kv caches during P/D (#55055)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
2026-09-15 17:31:25 -07:00
Jared WenandClaude Fable 5.1 1257512305 [DSA] Wire DeepGEMM sparse MQA logits into the DeepSeek V4.1 indexer (#56254)
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-15 14:07:14 -07:00
Nicolò Lucchesi df42d112ee [Config] Infer HiSparse attention config from HiSparseConnector (#57041)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
2026-09-15 19:44:26 +00:00
Nick Hill ceaa9f95dd [Minor] Invert MRV1 specdec method fallback logic (#56899)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-09-14 18:56:04 -07:00
Giancarlo Delfin 3bb0a03f35 [Model Runner V2] Acceptance estimation for adaptive verification (#52228)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
2026-09-14 15:58:48 -07:00
Netanel Haber d4ee7fe7a9 [Quantization] Select linear backends per quantization (#51204)
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
2026-09-14 16:24:18 -04:00
鐘天楽andNick Hill 0a5747d410 [Profiler] Add Proton CUDA graph attribution for MRV2 (#51084)
Signed-off-by: Luossu <zhluosuu@outlook.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
2026-09-14 19:02:07 +00:00
78e84261ab [Performance] Add Triton kernel for Gemma3n sparse GELU (#48498)
Signed-off-by: BWAAEEEK <jooho414@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-09-14 07:42:15 +00:00
Anjie HouandCodex 73d2a8cf86 [2/2][Model Runner V2] FULL CUDA graph capture for microbatched steps (DBO) (#51700)
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-13 22:42:33 -07:00
Wentao Ye 9f03b510c3 [DSv4 Bug] fix dsv4 start up error NotImplementedError: DeepSeek V4 MegaMoE currently requires expert parallel (#55914)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-09-13 20:22:08 -04:00
Lucas WilkinsonandOpenAI Codex 7fe8fc803d [NIXL][PCP][DCP] Expose PCP producer KV shards as transfer ranks (#56645)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-13 20:23:15 +00:00
d2d649e674 [DS V4.1][Engram] Support async prefetch for offloaded engram lookups and engram DP sharding (#56512)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: ywang96 <>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
2026-09-13 02:44:23 -07:00
fa008bdccf [Perf][Kernel] Integrate DeepSelect TopK for the DSA sparse indexer (#56464)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
2026-09-13 00:44:37 -07:00
Markus HartikainenandJee Jee Li c711f740b7 [ROCm] Auto-enable breakable CUDA graphs for DeepseekV41ForCausalLM (#56349)
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
2026-09-12 20:47:07 -07:00
7ee8a6dd01 [watermarking] Dual-key gumbel-max watermarking for speculative decoding support (#56122)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
2026-09-12 18:42:01 +00:00
Itay Alroy 658c8131c7 [Elastic EP] Reuse CUDA graphs across reconfiguration (#54985) 2026-09-12 10:26:39 -04:00
JartX 986e2f870a [Refactor][ROCm] Migrate the RDNA3 W4A16 MoE to the oracle/experts pa… (#55522)
Signed-off-by: JartX <sagformas@epdcenter.es>
2026-09-12 21:20:26 +08:00
d43bb2f37f [3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
2026-09-12 00:36:47 -04:00
Wentao Ye 5fe77aecfc [Deprecation] Deprecate items scheduled for 0.29 (#55353)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-09-11 08:17:02 -07:00
Isotr0py 22258a26bc [Multimodal] Use GPU NVDEC for EPD encoder-only instance video media IO (#53675)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-11 22:20:11 +08:00
Harry MellorandClaude Opus 5 c191787a68 [Bugfix] Align vLLM YaRN with Transformers and stop re-scaling max_model_len (#56446)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 13:01:38 +00:00
e77daef89e [Model] Support DeepSeek-V4.1-Flash (#56214)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 02:11:19 -07:00
d0dfe587d5 [Bugfix] Fall back to full decode graphs for noncompiled models (#55095)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
2026-09-11 01:56:05 -07:00
Tianyu Guo e7edf17cea [EC] Automatically enable embedding inputs on EC/KV consumers (#56379)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
2026-09-11 04:53:37 +00:00
Wei ZhaoWei ZhaoOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
7cdd9304ae [KV Connector] Support symmetric DCP disagg for hybrid mamba models (#55531)
Signed-off-by: Wei Zhao <weizha@oci-aga-slurm-1-vscode-02.cm.cluster>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Wei Zhao <weizha@oci-aga-slurm-1-vscode-02.cm.cluster>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-10 11:12:15 -07:00
Nick Hill e6cb56337b [Core] MRV2 support for fast-prefill (#56145)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-09-10 10:36:04 -07:00
YoavMironandBenjamin Chislett 40e6042ec8 [Feature][Spec Decode] MTP with separate (possibly quantized) lm head for nemotron (#54574)
Signed-off-by: Yoav Miron <yomiron@nvidia.com>
Signed-off-by: YoavMiron <yomiron@nvidia.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
2026-09-10 08:26:31 -04:00
zexplorerhjCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
fe26c705f7 [Model] Add Bailing V3 VL support (#55921)
Signed-off-by: zexplorerhj <zhjoneson@163.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-10 02:46:56 -07:00
Bugen ZhaoandIsotr0py b47b01cf33 [Model][Frontend] Support DeepSeek-V4.1-Flash in Rust and Python frontends (#56208)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-09-10 01:15:06 -07:00
Raphaël RiallandOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ea40bb9e90 [feature] Watermarked generation and detection (Gumbel-max algorithm) (#54053)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-09 21:00:15 -07:00
Harry MellorandClaude Opus 5 719284fe15 [Core][Model] Unify XD-RoPE into M-RoPE and derive the channel count (#56078)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 09:23:30 -07:00
cc4210f671 [Perf] Improve BF16x3 router GEMM accuracy and make it default on sm100 (#55899)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
2026-09-09 07:33:12 -07:00
Harry Huangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 3116c5d06b [Qwen4Exp] Support UVA PLE-offload and Engram tensor parallelism (#54371)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-09 07:32:34 -07:00
Andreas Karatzas 18615ad1ee [Bugfix] Handle null RoPE parameters for NoPE layers (#55949)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-09-08 20:20:05 +00:00
Qiu Chunshuo 7c2f1ff495 [Core] Scope PCP-DP validation to GPU manager (#54523)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
2026-09-08 08:18:01 -07:00
263c4ff95f [Bugfix][Spec Decode] Cache the Mamba state at the block-grid position of EAGLE resume (#53945)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Adam Shaver <ashaver@nvidia.com>
Signed-off-by: akshaver <168006157+akshaver@users.noreply.github.com>
Co-authored-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: roikoren755 <26850796+roikoren755@users.noreply.github.com>
2026-09-08 09:59:58 -04:00
Nicolò Lucchesi cd64c2dea9 [Docs] Clarify admission control limits apply server-wide, not per DP rank (#55124)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
2026-09-07 14:08:48 +00:00
Soumyajit GhoshSOUMYAJIT GHOSHmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
b339d75a41 [Bugfix][Spec Decode] Resolve n_predict from text_config for Qwen3.5 multimodal MTP (#55369)
Signed-off-by: SOUMYAJIT GHOSH <23051387@kiit.ac.in>
Signed-off-by: Soumyajit Ghosh <jobsoumyajit6124@gmail.com>
Co-authored-by: SOUMYAJIT GHOSH <23051387@kiit.ac.in>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 13:28:26 +00:00