fa6ae31177
feat: rename logit_bias/logit_scale to logit_mean/logit_sigma for affine score calibration ( #39530 )
...
Signed-off-by: Jesus Federico <jefp@amazon.com >
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-13 04:43:44 +00:00
maobaolong
2a3c32ce67
fix(lmcache): correct store for cached requests and num_scheduled_tokens in lmcache_mp_connector.py ( #39655 )
...
Signed-off-by: baoloongmao <baoloongmao@tencent.com >
2026-04-13 03:29:19 +00:00
Kaicheng Yang and Claude Sonnet 4.6
4beeb0689c
fused qknorm+rope kernel optimization for SM9.0 ( #37376 )
...
Signed-off-by: EricccYang <yangyang4991@gmail.com >
Signed-off-by: Kaicheng Yang <53411596+EricccYang@users.noreply.github.com >
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-12 19:58:37 -07:00
Zhengxu Chen and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
cae984060f
[compile] Enable AOT compile with batch invariance mode. ( #39201 )
...
Signed-off-by: zhxchen17 <zhxchen17@fb.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-12 19:58:33 -07:00
Jee Jee Li
715681c127
[LoRA] Support dual CUDA streams-Linear Layer ( #35721 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
2026-04-13 10:57:07 +08:00
Animesh Jain
f00c5539d7
[compile] Bug fix for _decompose_size_nodes ( #38360 )
...
Signed-off-by: Animesh Jain <anijain@umich.edu >
2026-04-12 20:20:24 +00:00
Le Yang
21fab0a3db
fix(moe): fix RoutedExpertsCapturer assertion failure with DP>1 and MK path ( #37879 )
2026-04-12 10:28:17 -04:00
Nicolò Lucchesi
3244a2ebf2
[KVConnector][NIXL] Organize NIXL connector into its own directory ( #39354 )
...
The number of features supported by the connector has grown substantially
and the `nixl_connector.py` file has accumulated a lot of code. Creates a separate
directory and isolates connector/scheduler code in the hope of improving clarity
and maintainability.
Further refactor of components aimed at improving clarity and simplifying code
will follow soon.
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-12 13:10:50 +00:00
Mark McLoughlin
72ff142c37
[Core][Metrics] Remove vllm:prompt_tokens_recomputed metric ( #38709 )
...
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
2026-04-12 12:22:01 +03:00
Nick Hill
ee3c0c83db
[Pooling] Disable async scheduling by default for pooling models ( #39592 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-04-12 07:23:42 +00:00
Martin Hickey and Or Ozeri
cc07dad789
[HMA] [KVEvent] Enable GPU-side KV events for HMA ( #37688 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: Or Ozeri <or@ozery.com >
2026-04-12 10:01:02 +03:00
r266-tech and Roger Wang
17e787a779
fix(kimi_k25): resolve media_placeholder_token_id from tokenizer ( #39344 )
...
Signed-off-by: r266-tech <r266.tech@gmail.com >
Signed-off-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-04-11 21:10:24 -07:00
Xinyu Chen and Kunshang Ji
639402f5a2
Support FP8 KVCache on XPU ( #37731 )
...
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-04-12 03:53:32 +00:00
Yan Ma
394ff86965
[XPU][CT] support per-channel quantization in xpu fp8 linear method ( #38316 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-04-12 02:46:28 +00:00
EdalatiAli
df1e30e74b
[Quant] add CompressedTensorsW8A8Mxfp8 for linear and MoE layers ( #38815 )
...
Signed-off-by: EdalatiAli <aliedalati@cohere.com >
2026-04-11 17:21:36 -06:00
ShubyM
92feb9991d
[Gemma4][Bugfix]: Enable Gemma4ForCasualLM to load lora adapters correctly ( #38844 )
...
Signed-off-by: ShubyM <shubymishra20@gmail.com >
2026-04-11 09:06:49 +00:00
d4cb783c10
[Bugfix] Fix GDN FLA kernel crashes with NULL_BLOCK_ID=0 CUDA graph padding ( #39064 )
...
Signed-off-by: Vibhav Agarwal <vibhavagarwal5@gmail.com >
Co-authored-by: vibhav-agarwal <vibhav.agarwal@glance.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-04-11 08:35:19 +00:00
z1ying
a3e750c0a5
[Misc] Update deprecation warning for --model flag ( #39518 )
...
Signed-off-by: Ziying Tao <tzzying@outlook.com >
2026-04-10 23:25:20 -07:00
Lee Yongjun
da72daced2
[Bugfix] add SupportsMultiModal to Exaone4_5_MTP ( #39526 )
...
Signed-off-by: leeyongjun <jqueen.astro@gmail.com >
2026-04-10 22:57:26 -07:00
Tianyu Guo and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
8d0aabdde9
Fix the order of _free_encoder_inputs ( #38907 )
...
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-10 22:47:48 -07:00
Michael Goin
11e2375fe2
[Refactor] Move MXFP8 GEMM management into MxFp8LinearKernel ( #39205 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-04-10 14:02:03 -07:00
e7cfd7c5b9
Add Gemma4 Eagle3 support ( #39450 )
...
Signed-off-by: Rahul-Tuli <rtuli@redhat.com >
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com >
Co-authored-by: Rahul-Tuli <rtuli@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-04-10 12:35:35 -07:00
yzong-rh
e816a8811f
[Bugfix] Fix FlashInfer crash with kv_cache_dtype_skip_layers ( #39002 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
2026-04-10 18:50:47 +00:00
Manu
51cfc0e76c
perf(moe): add tuned fused_moe config for RTX PRO 6000 Blackwell Server Edition ( #39183 )
...
Signed-off-by: manu <fortin.emmanuel@gmail.com >
2026-04-10 11:32:42 -06:00
Jesus Federico and Claude Opus 4.6
b87575d24b
feat: add logit_scale to PoolerConfig for affine score calibration ( #39435 )
...
Signed-off-by: Jesus Federico <jefp@amazon.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-10 17:21:14 +00:00
Jee Jee Li
ecd1ea1363
[Kernel] Porting the TRTLLM minimax_allreduce_rms kernels ( #37045 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
2026-04-11 00:20:20 +08:00
zhrrr
8f121f7879
[Model Runner V2] support auto resolve cudagraph mode/sizes based on attn backend ( #32936 )
...
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com >
2026-04-10 08:27:15 -07:00
wang.yuqi
cb5f7501cb
[New Model]: jinaai/jina-reranker-v3 ( #38800 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-10 15:20:40 +00:00
Peter Nguyen
8d0f908b98
[Model] Implement LoRA support for Qwen3ASRForConditionalGeneration ( #37247 )
...
Signed-off-by: Peter Nguyen <petern0408@gmail.com >
2026-04-10 18:34:31 +04:00
Ravitez Dondeti and Claude
c1cc7344fb
[ROCm] Add RDNA 3.5/4 device IDs (gfx1150, gfx1151, gfx1201) ( #38455 )
...
Signed-off-by: rdondeti <ravitez.dondeti@gmail.com >
Signed-off-by: Ravitez Dondeti <ravitez.dondeti@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-04-10 11:35:07 +00:00
xaguilar-amd
f976e3b98b
[Performance] Remove unnecessary zero-fill of MLA decode output tensor in Aiter backend ( #37539 )
...
Signed-off-by: xaguilar-amd <xaguilar@amd.com >
2026-04-10 11:27:35 +00:00
d468322dc1
[Kernel][Hardware][AMD] Add TritonW4A16LinearKernel for ROCm ( #37352 )
...
Signed-off-by: jatseng-ai <jatseng@amd.com >
Signed-off-by: jatseng-ai <janet.tseng@amd.com >
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Matthias Gehre <matthias.gehre@amd.com >
2026-04-10 10:25:27 +00:00
PatchyTIS and PatchouliTaisa
967146e7bd
[model] support FireRedLID ( #39290 )
...
Signed-off-by: PatchouliTaisa <patchychen@tencent.com >
Co-authored-by: PatchouliTaisa <patchychen@tencent.com >
2026-04-10 08:43:58 +00:00
8e8a3becd1
[ZenCPU] Make PT Backport Patch Accessible to vLLM ( #38205 )
...
Signed-off-by: Lalithnarayan C <Lalithnarayan.C@amd.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
2026-04-10 08:29:35 +00:00
Hexiang Wang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
1dfd64c1cc
[PluggableLayer][3/N] Apply PluggableLayer to llm_head and vocab embedding layer ( #33465 )
...
Signed-off-by: whx-sjtu <2952154980@qq.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-10 16:12:59 +08:00
Elvir Crnčević and Claude Sonnet 4
ad720aefe9
[Bugfix] Fix V1 dummy run writing NaN to KV cache null block ( #39444 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com >
2026-04-10 10:09:46 +02:00
milesial
270e8a4102
Nemotron Nano VL: Streamline pixel shuffle ( #37580 )
...
Signed-off-by: milesial <milesial@users.noreply.github.com >
2026-04-10 07:31:19 +00:00
Richard Zou
f44afef6d6
[compile] Allow strings in custom ops without regressing compilation times ( #38123 )
...
Signed-off-by: Richard Zou <zou3519@gmail.com >
2026-04-10 07:26:37 +00:00
Injae Ryou and Isotr0py
447ce22212
[GGUF] Support non-standard quant types with prefix (e.g. UD-IQ1_S) ( #39471 )
...
Signed-off-by: Injae Ryou <injaeryou@gmail.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-04-10 07:22:53 +00:00
Chaofan Wang and Isotr0py
49d20346e4
[Perf] Reduce H2D pageable memory copies ( #38794 )
...
Signed-off-by: jackcfwang <jackcfwang@tencent.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-04-10 15:03:26 +08:00
Nick Hill
ef076c1b73
[Core] Change max_model_len in EngineCoreReadyResponse to be non-None ( #39442 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-04-10 14:34:57 +08:00
Yan Ma
ec68d53b2b
Add platform manual_seed_all API ( #38468 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-04-10 13:43:50 +08:00
Elham
13e6b1b908
[BugFix][CPU] Add CPU profiler summary file output ( #38366 )
...
Signed-off-by: Elham Harirpoush <elham.harirpoush@arm.com >
2026-04-10 13:41:15 +08:00
3dd60971de
[feat]: make DCP error msg clearer ( #28443 )
...
Signed-off-by: vnadathur <glvikramn@gmail.com >
Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com >
Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com >
Co-authored-by: vnadathur <glvikramn@gmail.com >
Co-authored-by: vnadathur <236933696+vnadathur@users.noreply.github.com >
2026-04-10 13:27:22 +08:00
Ronen Schaffer
a5b17fba8f
[KV Offload] Implement shutdown() in OffloadingConnector and related classes ( #39182 )
...
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com >
2026-04-10 08:06:22 +03:00
Kyungmin Lee and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
e7a1387e73
Add EXAONE-4.5 ( #39388 )
...
Signed-off-by: lkm2835 <lkm2835@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-09 20:53:26 -07:00
Ganesh R
445a2a4d1a
feat(cpu): add CPU support for draft model speculative decoding ( #32662 )
...
Signed-off-by: R <Ganesh.R@amd.com >
2026-04-10 11:49:52 +08:00
Kunshang Ji
55d037e2e5
[CT][FP8][Marlin] refactor CompressedTensorsW8A16Fp8 to use kernel abstraction ( #38244 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com >
2026-04-10 09:58:35 +08:00
Chauncey
ecbfbb8d61
[Feature] Add auto-detection for reasoning_config when only reasoning_parser is set ( #38214 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-04-10 01:36:26 +00:00
Chuan (Richard) Li
e0613702ad
[ROCm] Fix AITER ops fake impl and minor bugs ( #36092 )
...
Signed-off-by: Li <chuali@amd.com >
2026-04-09 17:56:17 -07:00
