|
ffe3bb3c72
|
[Hardware][XPU] Register matmul and linear batch-invariant kernels for XPU (#49209)
Signed-off-by: tzielinski-habana <tomasz.zielinski@intel.com>
Signed-off-by: Tomasz Zielinski <85164140+tzielinski-habana@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-09-02 13:31:38 +00:00 |
|
|
fxmarty-amd
|
7894394b07
|
[Online quantization] Add targeted online quantization configuration based on user patterns (#51285)
|
2026-09-02 08:55:15 -04:00 |
|
|
周峥andGuanyi Chen
|
c866ba9d11
|
[KV Connector] Support heterogeneous TP sharing in Mooncake Store Connector (#53129)
Signed-off-by: Guanyi Chen <939416532@qq.com>
Signed-off-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: Zheng Zhou <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: zane <zhouzh93@mail2.sysu.edu.cn>
Co-authored-by: Guanyi Chen <939416532@qq.com>
|
2026-09-01 10:18:06 -07:00 |
|
|
|
40824284bc
|
[Doc] Document FP8 GEMM kernel selection and Blackwell support (#49936)
Signed-off-by: harjoth <harjoth.khara@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-09-01 13:22:40 +01:00 |
|
|
Jan Kwieciński
|
ec32f669bb
|
[Feature][MM_UUIDs] Allow empty video URLs when using multi-modal UUIDs (#54220)
Signed-off-by: Jan Kwiecinski <jkwiecinski@nvidia.com>
|
2026-09-01 05:55:27 +00:00 |
|
|
Lain
|
882ca8d696
|
[Kernel] add Flashinfer cutedsl w4a16 linear (#53014)
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
|
2026-09-01 02:49:14 +00:00 |
|
|
macheroandClaude
|
87b9b5b8d9
|
[Test][VLM] Add batch-invariance tests for Qwen3-VL (#53531)
Signed-off-by: machero <sr1123640465@126.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-30 09:49:41 -04:00 |
|
|
Isotr0py
|
4fc943b867
|
[Multimodal] Deprecate PyAV video decoder backend (#54231)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-08-29 15:13:28 +00:00 |
|
|
hangy-amd
|
31c579503d
|
update quark docs to include online quantization (#52736)
Signed-off-by: Hang Yang <hangy@amd.com>
|
2026-08-28 18:23:06 +00:00 |
|
|
Paarth SharmaandWentao Ye
|
a18dbe49ad
|
[Doc] Add EXAONE-4.0-1.2B to batch invariance tested models (#53839)
Signed-off-by: Cogniera <paarths376@gmail.com>
Signed-off-by: Paarth Sharma <paarths376@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-08-27 18:38:03 +00:00 |
|
|
|
f18d0ba90d
|
[Doc] Add Granite 3.1 series to batch invariance tested models (#53650)
Signed-off-by: Shenglei Fu <117230642+ShengleiFu@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-26 18:29:10 -04:00 |
|
|
Naveenraj KamalakannanandBenjamin Chislett
|
d5cadcee86
|
[Feature][DSpark]: Logprobs adaptive verification (#52242)
Signed-off-by: Naveenraj Kamalakannan <therealnaveenkamal@gmail.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-08-25 10:22:33 -04:00 |
|
|
Lucas Wilkinson
|
8bdc70ec7b
|
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-08-21 17:47:49 -07:00 |
|
|
Maroon Ayoub
|
a556f3fccb
|
Forward Anthropic vllm_xargs to sampling params (#53308)
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
|
2026-08-21 15:23:21 -07:00 |
|
|
Luke Alonsoandmgoin
|
2740c817ff
|
[Kernel] Add b12x FP4 MoE backend (#52018)
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-08-21 08:04:36 -07:00 |
|
|
|
7cfb97e337
|
[Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915)
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-20 15:42:49 -07:00 |
|
|
SeonjinandMisha Goin
|
bd8865a299
|
[Kernel] Add FlashInfer TRTLLM MXFP8 linear backend (#52204)
Signed-off-by: seonjinn <sna@nvidia.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-08-20 10:07:22 -04:00 |
|
|
Guanyi Chenandz-zanez
|
bf2866f8bf
|
[KV Connector] Add decode offloading to Mooncake Store consumers (#52466)
Signed-off-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: Guanyi Chen <939416532@qq.com>
Co-authored-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
|
2026-08-19 17:40:35 -07:00 |
|
|
Russell BryantandClaude Opus 4.8
|
ef47a897e2
|
[Core] Make prefix-cache NONE_HASH deterministic by default (#51875)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-18 23:14:39 +00:00 |
|
|
Tzu-Ling KanandClaude Opus 4.8
|
653cc6faca
|
[Bugfix][NIXL] Include transfer mode (push/pull) in the compatibility hash (#50620)
Signed-off-by: Tzu-Ling <tzulingk@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-14 12:11:22 +08:00 |
|
|
Luke Alonso
|
3c79b1a8bf
|
[Kernel] Add B12X dense linear backends (#52016)
Signed-off-by: Luke Alonso <lalonso@gmail.com>
|
2026-08-13 19:48:33 -07:00 |
|
|
QWERQWERQWE86
|
443fa5aca0
|
[Doc] Fix stale rejection_sample_method and synthetic_acceptance_rate (#51611)
Signed-off-by: qwerqwerqwe8688-jpg <xuuuuu2021@163.com>
|
2026-08-13 10:56:27 +00:00 |
|
|
|
7f7a32cfec
|
[Spec Decode] DSpark confidence-scheduled verification (#47808)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-12 12:27:48 -07:00 |
|
|
Kyungmin LeeandHarry Mellor
|
529d010351
|
[Doc] Fix typos in speculative decoding docs (#51500)
Signed-off-by: lkm2835 <lkm2835@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-08-11 09:17:05 +01:00 |
|
|
ChaunceyandOr Ozeri
|
c423998642
|
[KV Offload] Emit self-describing events for partial recurrent blocks (#51243)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-09 11:28:44 +03:00 |
|
|
Isotr0py
|
073c510c91
|
[Migration] Migrate bitsandbytes support to OOT plugin (#43529)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-08-09 10:27:31 +08:00 |
|
|
fxmarty-amd
|
3518110f2f
|
[Online quantization] Add online MXFP4 quantization support (#49347)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
|
2026-08-07 12:39:46 -07:00 |
|
|
|
b50fdebce0
|
[Bugfix] Fix level-2 sleep/wake/reload with enable_lora=True (#39935)
Signed-off-by: Silen Naihin <silen.naihin@gmail.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-08-06 09:13:10 +08:00 |
|
|
LZWandRoger Wang
|
d36f24b66a
|
[KV Connector][Mooncake] Add tenant ID support to MooncakeStoreConnector (#48069)
Signed-off-by: lzw <2994587066@qq.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-08-05 17:31:18 -07:00 |
|
|
|
14e57ad47d
|
[Docker][KVConnector] Install mooncake from official wheels instead of a custom build (#51067)
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-08-05 11:58:44 -07:00 |
|
|
Ronen SchafferandOr Ozeri
|
8543522ca7
|
[KV Offload] Support out-of-tree secondary tier managers via module_path (#51007)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-05 20:30:53 +03:00 |
|
|
|
952694e384
|
[Bugfix] Validate NIXL speculative config compatibility (#49230)
Signed-off-by: Tzu-Ling <tzulingk@nvidia.com>
Co-authored-by: GPT-5.6 Sol <noreply@openai.com>
Co-authored-by: Cursor Grok 4.5 <noreply@cursor.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-04 02:51:04 +08:00 |
|
|
wangxiyuan
|
dd11df04f3
|
[Misc] Remove deprecated calculate_kv_scales runtime KV scale calculation (#49389)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
|
2026-08-03 11:15:13 -04:00 |
|
|
Nicolò Lucchesi
|
8ba87e0181
|
[CI] Mooncake PD integration tests (#46844)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-08-03 06:55:51 -07:00 |
|
|
Fede KamelharandHarry Mellor
|
005fa01756
|
docs: document reasoning_content output removal as a breaking client change (#50624)
Signed-off-by: fede-kamel <fkamelhar@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-08-03 13:45:23 +00:00 |
|
|
zofiaandKunshang Ji
|
e481da9508
|
[XPU] [Linear] add torch as xpu linear backend (#49664)
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-08-03 14:30:21 +08:00 |
|
|
Nicolò Lucchesi
|
17beffd55f
|
[Misc] Clarify mono audio requirement (#50141)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-07-31 10:46:53 +00:00 |
|
|
yinfengLiu
|
f1e8fd27ae
|
[ROCm] Add AITER FP8 ViT encoder attention (#49937)
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
|
2026-07-30 06:22:10 +00:00 |
|
|
Brandon PelfreyandCyrus Leung
|
5c7a7f9462
|
[docs] Add documentation for pynvvideocodec video decoding backend (#49066)
Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-07-29 17:23:04 +00:00 |
|
|
Seiji Eicher
|
6370e53f24
|
[Frontend] Reuse prefill token ids on the decode chat path for disaggregated serving (#48145)
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
|
2026-07-29 10:02:57 +02:00 |
|
|
Philip Pesic
|
dc1be79031
|
Add CachePolicyFactory for pluggable/external eviction policies (#49114)
Signed-off-by: Philip Pesic <philippesic06@gmail.com>
|
2026-07-29 07:34:55 +03:00 |
|
|
nvbfalk
|
60915c972c
|
[Feature] Add VidCom2 video token pruning (#47750)
Signed-off-by: Benedikt Falk <bfalk@nvidia.com>
|
2026-07-28 03:20:17 +00:00 |
|
|
Harjoth KharaandClaude Opus 5
|
8de50e46d4
|
[Docs] Document NVFP4 GEMM kernel selection and Marlin weight-only fallback (#49376)
Signed-off-by: harjoth <harjoth.khara@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-07-26 20:53:28 -07:00 |
|
|
|
da3a252fd1
|
[KVOffload][P2P] Generic P2P secondary tier: peer lookup and serving via ParentManager (#48021)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: liranschour <liranschour@users.noreply.github.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-26 11:45:47 +03:00 |
|
|
Brandon PelfreyandOpenAI Codex
|
94682b79f4
|
[multimodal] Make PyNvVideoCodec decoder concurrency configurable (#49753)
Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-07-24 23:31:41 -07:00 |
|
|
Andreas Karatzas
|
16aca639b7
|
[ROCm] Upgrade NIXL and UCX (#49251)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
|
2026-07-21 22:30:02 -05:00 |
|
|
Chang GuoandOpenAI Codex
|
f007cceb42
|
[KV Offload] Support self-describing KV events with TieringOffloadingSpec (#48679)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-20 16:41:58 +03:00 |
|
|
Itay EtelisandItay Etelis
|
5245c80564
|
[Doc] Document blocks_per_chunk in the KV offloading guide (#49100)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
|
2026-07-20 09:48:43 +03:00 |
|
|
|
426d48bfa1
|
[KV Offload] Add optional tier locality to FS/OBJ KV events (#48281)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-17 10:31:52 +03:00 |
|
|
|
ab0a20d151
|
[Docs] Add Phi-3.5-mini-instruct to batch invariance tested models (#46396)
Signed-off-by: Yuval Luria <yuvalluria@users.noreply.github.com>
Co-authored-by: Yuval Luria <yuvalluria@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
|
2026-07-16 18:05:58 -04:00 |
|