+85
431cea3eea
[Bugfix] Fix tool_calls Iterable consumed when debug logging is enabled ( #34844 )
...
Signed-off-by: Wojciech Wais <wojciech.wais@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com >
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com >
Signed-off-by: Rishi Puri <riship@nvidia.com >
Signed-off-by: Jaebok Lee <jaebok9541@naver.com >
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
Signed-off-by: yuwei <yuwei@dev.local >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Signed-off-by: Ibrahim Arshad <38925737+ibrahim1023@users.noreply.github.com >
Signed-off-by: Li <chuali@amd.com >
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com >
Signed-off-by: R <Ganesh.R@amd.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: lkm2835 <lkm2835@gmail.com >
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com >
Signed-off-by: vnadathur <glvikramn@gmail.com >
Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com >
Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Elham Harirpoush <elham.harirpoush@arm.com >
Signed-off-by: Yan Ma <yan.ma@intel.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Signed-off-by: jackcfwang <jackcfwang@tencent.com >
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
Signed-off-by: Injae Ryou <injaeryou@gmail.com >
Signed-off-by: Richard Zou <zou3519@gmail.com >
Signed-off-by: milesial <milesial@users.noreply.github.com >
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
Signed-off-by: whx-sjtu <2952154980@qq.com >
Signed-off-by: Lalithnarayan C <Lalithnarayan.C@amd.com >
Signed-off-by: PatchouliTaisa <patchychen@tencent.com >
Signed-off-by: jatseng-ai <jatseng@amd.com >
Signed-off-by: jatseng-ai <janet.tseng@amd.com >
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
Signed-off-by: xaguilar-amd <xaguilar@amd.com >
Signed-off-by: rdondeti <ravitez.dondeti@gmail.com >
Signed-off-by: Ravitez Dondeti <ravitez.dondeti@gmail.com >
Signed-off-by: NickLucche <nlucches@redhat.com >
Signed-off-by: Peter Nguyen <petern0408@gmail.com >
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Signed-off-by: Jesus Federico <jefp@amazon.com >
Signed-off-by: manu <fortin.emmanuel@gmail.com >
Signed-off-by: ZhanqiuHu <zhu@redhat.com >
Signed-off-by: Yifan Zong <yzong@redhat.com >
Signed-off-by: Rahul-Tuli <rtuli@redhat.com >
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Michael Goin <mgoin64@gmail.com >
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn >
Signed-off-by: leeyongjun <jqueen.astro@gmail.com >
Signed-off-by: Ziying Tao <tzzying@outlook.com >
Signed-off-by: jiang1.li <jiang1.li@intel.com >
Signed-off-by: Vibhav Agarwal <vibhavagarwal5@gmail.com >
Signed-off-by: ShubyM <shubymishra20@gmail.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Signed-off-by: Itay Etelis <itay.etelis@ibm.com >
Signed-off-by: EdalatiAli <aliedalati@cohere.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: r266-tech <r266.tech@gmail.com >
Signed-off-by: Roger Wang <hey@rogerw.io >
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Signed-off-by: Animesh Jain <anijain@umich.edu >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: zhxchen17 <zhxchen17@fb.com >
Signed-off-by: EricccYang <yangyang4991@gmail.com >
Signed-off-by: Kaicheng Yang <53411596+EricccYang@users.noreply.github.com >
Signed-off-by: baoloongmao <baoloongmao@tencent.com >
Signed-off-by: sihao.li <sihao.li@intel.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Signed-off-by: Tihomir Elek <tiho.elek@gmail.com >
Signed-off-by: yiliu30 <yi4.liu@intel.com >
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Santino Ramos <santinor@inferact.ai >
Signed-off-by: haosdent <haosdent@gmail.com >
Signed-off-by: JartX <sagformas@epdcenter.es >
Signed-off-by: George-ao <yuyiao772@gmail.com >
Signed-off-by: Yuyi Ao <yuyiao772@gmail.com >
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Signed-off-by: Mukesh Baphna <mukesh@hippocraticai.com >
Signed-off-by: Pedram Razavi <pedram.razavi@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com >
Co-authored-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com >
Co-authored-by: Rishi Puri <riship@nvidia.com >
Co-authored-by: zzaebok <44357534+zzaebok@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com >
Co-authored-by: yuwei <yuwei@dev.local >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Ibrahim Arshad <38925737+ibrahim1023@users.noreply.github.com >
Co-authored-by: Chuan (Richard) Li <chuali@amd.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: Ganesh R <ganesh.r@amd.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Kyungmin Lee <30465912+lkm2835@users.noreply.github.com >
Co-authored-by: Ronen Schaffer <ronen.schaffer@ibm.com >
Co-authored-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com >
Co-authored-by: vnadathur <glvikramn@gmail.com >
Co-authored-by: vnadathur <236933696+vnadathur@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Elham <elham.harirpoush@arm.com >
Co-authored-by: Yan Ma <yan.ma@intel.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Chaofan Wang <jackcfwang@tencent.com >
Co-authored-by: Chendi.Xue <chendi.xue@intel.com >
Co-authored-by: Injae Ryou <injaeryou@gmail.com >
Co-authored-by: Richard Zou <zou3519@users.noreply.github.com >
Co-authored-by: milesial <milesial@users.noreply.github.com >
Co-authored-by: Elvir Crnčević <elvircrn@gmail.com >
Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com >
Co-authored-by: Hexiang Wang <56632993+whx-sjtu@users.noreply.github.com >
Co-authored-by: Lalithnarayan C <Lalithnarayan.C@amd.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
Co-authored-by: PatchyTIS <58251192+PatchouliTIS@users.noreply.github.com >
Co-authored-by: PatchouliTaisa <patchychen@tencent.com >
Co-authored-by: jatseng-ai <janet.tseng@amd.com >
Co-authored-by: Matthias Gehre <matthias.gehre@amd.com >
Co-authored-by: xaguilar-amd <xavier.aguilarfruto@amd.com >
Co-authored-by: Ravitez Dondeti <dondetir@users.noreply.github.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
Co-authored-by: Peter Nguyen <petern0408@gmail.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: zhrrr <43847754+izhuhaoran@users.noreply.github.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
Co-authored-by: Jesus Federico <14651+jefp@users.noreply.github.com >
Co-authored-by: Manu <efortin@users.noreply.github.com >
Co-authored-by: zhanqiuhu <49648934+ZhanqiuHu@users.noreply.github.com >
Co-authored-by: yzong-rh <yzong@redhat.com >
Co-authored-by: Fynn Schmitt-Ulms <fschmitt@redhat.com >
Co-authored-by: Rahul-Tuli <rtuli@redhat.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com >
Co-authored-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn >
Co-authored-by: Lee Yongjun <35302114+elwhyjay@users.noreply.github.com >
Co-authored-by: z1ying <55220715+z1ying@users.noreply.github.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
Co-authored-by: Vibhav Agarwal <vibhavagarwal5@gmail.com >
Co-authored-by: vibhav-agarwal <vibhav.agarwal@glance.com >
Co-authored-by: ShubyM <shubymishra20@gmail.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Itay Etelis <itay.etelis@ibm.com >
Co-authored-by: EdalatiAli <aliedalati@cohere.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: r266-tech <r2668940489@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: Or Ozeri <or@ozery.com >
Co-authored-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: Le Yang <562593859@qq.com >
Co-authored-by: Animesh Jain <anijain@umich.edu >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Zhengxu Chen <zhxchen17@fb.com >
Co-authored-by: Kaicheng Yang <53411596+EricccYang@users.noreply.github.com >
Co-authored-by: maobaolong <baoloongmao@tencent.com >
Co-authored-by: sihao_li <165983188+1643661061leo@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Co-authored-by: zofia <110436990+zufangzhu@users.noreply.github.com >
Co-authored-by: Tihomir Elek <tiho.elek@gmail.com >
Co-authored-by: Yi Liu <yi4.liu@intel.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Santino Ramos <51103228+santiramos27@users.noreply.github.com >
Co-authored-by: haosdent <haosdent@gmail.com >
Co-authored-by: JartX <sagformas@epdcenter.es >
Co-authored-by: Yuyi Ao <yuyiao772@gmail.com >
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com >
Co-authored-by: mukesh-hai <mukesh@hippocraticai.com >
Co-authored-by: Pedram Razavi <pedram@sierra.ai >
2026-04-15 01:32:47 -07:00
wliao2
3abf858443
[Test] Refactor hard coded device string in test files under compile/quantization/models/model_executor folders ( #38901 )
...
Signed-off-by: Liao, Wei <wei.liao@intel.com >
2026-04-15 11:02:35 +08:00
f4b42df048
[Attention Backend] TurboQuant: 2-bit KV cache compression with 4x capacity ( #38479 )
...
Signed-off-by: vibhavagarwal5 <vibhavagarwal5@gmail.com >
Signed-off-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-04-14 19:57:13 -07:00
bnellnm
19ec9a0a62
[MoE Refactor] Refactor ZeroExpertFusedMoE into new framework ( #35549 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-04-14 16:11:20 -04:00
Jackmin801 and Robert Shaw
1a9353bb02
[MoE] Move GPT OSS Triton kernel experts into fused_moe/experts/ ( #39007 )
...
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Signed-off-by: Jackmin801 <ongjackm@gmail.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-04-14 19:27:39 +00:00
roikoren755
ecf5ff7ce3
[Mamba] Flashinfer selective_state_update ( #36162 )
...
Signed-off-by: Roi Koren <roik@nvidia.com >
2026-04-14 15:10:58 -04:00
zhanqiuhu
30679319e8
[CI][KVConnector][Metrics] Update multi KV connector edge case according to prefill stats changes ( #39808 )
...
Signed-off-by: Zhanqiu Hu <zhu@redhat.com >
2026-04-14 18:59:15 +00:00
danielafrimi and root
240f2636ca
[Kernel] Support TRTLLM GEN NVFP4 MoE for non-512-aligned hidden dims via weight padding ( #39510 )
...
Signed-off-by: root <root@lyris0017.lyris.clusters.nvidia.com >
Signed-off-by: Daniel Afrimi <dafrimi@nvidia.com >
Co-authored-by: root <root@lyris0017.lyris.clusters.nvidia.com >
2026-04-14 11:49:56 -07:00
omerpaz95 and Or Ozeri
be0c855ebd
[KV Offload] Unified memory layout for offloading workers ( #37206 )
...
Signed-off-by: omerpaz95 <omerpaz95@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-04-14 21:33:33 +03:00
Micah Williamson
4f4f8eaa78
[ROCm][CI] Fix condition for test_per_token_group_quant_fp8_packed ( #39730 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
2026-04-14 16:14:31 +00:00
bnellnm and Robert Shaw
e1e318af01
[MoE Refactor] Remove MoE DP chunking ( #39107 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-04-14 09:48:05 -04:00
6f786f2c50
[Bugfix][Model] Fix Devstral Small 2 HF format weight loading ( #39293 )
...
Signed-off-by: thomasmaindron <thomasmaindron@users.noreply.github.com >
Co-authored-by: thomasmaindron <thomasmaindron@users.noreply.github.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-14 10:11:18 +00:00
fxmarty-amd
4eee77b877
[fix][MOE] Fix MOE experts intermediate_size dimension not being narrowed before weight loading ( #39688 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-04-14 09:35:28 +00:00
Julien Debache
893b2affff
feat: add TxtSlicesDataset to allow sampling slices from txt file for benchmarking ( #30156 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
2026-04-14 09:20:03 +00:00
Shanshan Shen and Roger Wang
80118853f4
[MM][Perf][CG] Support ViT full CUDA graph for Qwen3-VL video inference ( #38061 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-04-14 16:49:32 +08:00
wang.yuqi and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
c0ecaed950
[Frontend] Offload blocking preprocessing & postprocessing ops to thread pool for pooling entrypoints. ( #39763 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-14 08:29:25 +00:00
lalit10 and Cyrus Leung
0008729abf
[Model] Use mm_features for Ernie-4.5 VL M-RoPE ( #39753 )
...
Signed-off-by: Lalit Laxminarayan Bangad <lalitbangad@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-04-14 01:11:52 -07:00
Mark McLoughlin and Or Ozeri
d3af8c1831
[Core][Metrics][BugFix] Replace num_cached_tokens/num_external_computed_tokens with PrefillStats ( #37460 )
...
Related to `Counters can only be incremented by non-negative amounts`
error with the `vllm:prompt_tokens_by_source_total` metric.
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: Or Ozeri <or@ozery.com >
2026-04-14 09:00:45 +01:00
noobHappylife
25b3242d8b
Fix Responses API streaming for multiple auto tool calls ( #39626 )
...
Signed-off-by: noobhappylife <aratar1991@hotmail.com >
2026-04-14 13:28:43 +08:00
김의진 and Chauncey
b075604da1
[Bugfix] Fix Gemma4 tool parser converting bare null to string "null" ( #39679 )
...
Signed-off-by: KimuGenie <baby11686@naver.com >
Signed-off-by: Chauncey <chaunceyjiang@gmail.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-04-14 04:44:46 +00:00
8213e8f880
Bug/test eagle dp v0 ( #38938 )
...
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-04-13 20:50:08 +00:00
Pedram Razavi
3693f922ff
[Bugfix][Pooling] Fix silent weight corruption with buffer-reusing iterators ( #39650 )
...
Signed-off-by: Pedram Razavi <pedram.razavi@gmail.com >
2026-04-13 19:37:27 +00:00
5c18b961d6
[Core][Metrics] expose waiting request breakdown via labeled metric (capacity/deferred) ( #38435 )
...
Signed-off-by: Mukesh Baphna <mukesh@hippocraticai.com >
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
Co-authored-by: Mark McLoughlin <markmc@redhat.com >
2026-04-13 15:30:55 -04:00
haosdent and DarkLight1337
995e9a209e
[Bugfix] Use is_integrated to detect UMA GPUs for memory reporting ( #35356 )
...
Signed-off-by: haosdent <haosdent@gmail.com >
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-04-13 11:07:40 -07:00
zhanqiuhu
10d9872d3a
[CI][Metrics] Fix local_cache_hit assertion after prompt tokens metrics updates ( #39709 )
...
Signed-off-by: ZhanqiuHu <zhu@redhat.com >
2026-04-13 15:16:56 +00:00
Tihomir Elek
8d825b87d6
[Bug] Fix TypeError when hf_config.architectures is None during model loading ( #38849 )
...
Signed-off-by: Tihomir Elek <tiho.elek@gmail.com >
2026-04-13 12:13:21 +01:00
Yufeng He and Chauncey
200a727e94
[Bugfix] Fix Responses API instructions leaking through previous_response_id ( #37727 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-04-13 08:46:33 +00:00
edbc1abd1c
feat: add max_tokens_per_doc in rerank request. ( #38827 )
...
Signed-off-by: Jesus Federico <jefp@amazon.com >
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-13 01:24:09 -07:00
Flora Feng
0e39202ca9
[Bugfix] Fix GLM tool parser streaming with MTP or stream interval ( #39253 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-04-13 05:10:30 +00:00
Jee Jee Li
715681c127
[LoRA] Support dual CUDA streams-Linear Layer ( #35721 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
2026-04-13 10:57:07 +08:00
Yongye Zhu
620e8924d9
[Bugfix] [Tests] Enforce out tensor device in kernel/moe/test_cutedsl_moe.py ( #39644 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
2026-04-12 17:08:08 -07:00
Animesh Jain
f00c5539d7
[compile] Bug fix for _decompose_size_nodes ( #38360 )
...
Signed-off-by: Animesh Jain <anijain@umich.edu >
2026-04-12 20:20:24 +00:00
Le Yang
21fab0a3db
fix(moe): fix RoutedExpertsCapturer assertion failure with DP>1 and MK path ( #37879 )
2026-04-12 10:28:17 -04:00
Nicolò Lucchesi
3244a2ebf2
[KVConnector][NIXL] Organize NIXL connector into its own directory ( #39354 )
...
The number of features supported by the connector has grown substantially
and the `nixl_connector.py` file has accumulated a lot of code. Creates a separate
directory and isolates connector/scheduler code in the hope of improving clarity
and maintainability.
Further refactor of components aimed at improving clarity and simplifying code
will follow soon.
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-12 13:10:50 +00:00
Mark McLoughlin
72ff142c37
[Core][Metrics] Remove vllm:prompt_tokens_recomputed metric ( #38709 )
...
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
2026-04-12 12:22:01 +03:00
Martin Hickey and Or Ozeri
cc07dad789
[HMA] [KVEvent] Enable GPU-side KV events for HMA ( #37688 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: Or Ozeri <or@ozery.com >
2026-04-12 10:01:02 +03:00
EdalatiAli
df1e30e74b
[Quant] add CompressedTensorsW8A8Mxfp8 for linear and MoE layers ( #38815 )
...
Signed-off-by: EdalatiAli <aliedalati@cohere.com >
2026-04-11 17:21:36 -06:00
Wei Zhao
59b2f7b640
[Perf] Fuse Zero Initializer for FP8 DeepGemm Block Quant Kernel ( #39547 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-04-11 07:16:51 -07:00
ShubyM
92feb9991d
[Gemma4][Bugfix]: Enable Gemma4ForCasualLM to load lora adapters correctly ( #38844 )
...
Signed-off-by: ShubyM <shubymishra20@gmail.com >
2026-04-11 09:06:49 +00:00
Yan Ma and Kunshang Ji
0f3ce4c74b
[XPU] Fix spec-decode UTs under tests/v1/spec_decode ( #38491 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-04-11 01:31:00 +00:00
Harry Mellor and Robert Shaw
fc645f1acc
Add structure to requirements/ directory ( #39024 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-04-10 13:46:41 -07:00
yzong-rh
e816a8811f
[Bugfix] Fix FlashInfer crash with kv_cache_dtype_skip_layers ( #39002 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
2026-04-10 18:50:47 +00:00
zhanqiuhu
e281cb721c
[CI] Add MultiConnector (Nixl+Offloading) e2e edge case tests ( #39343 )
...
Signed-off-by: ZhanqiuHu <zhu@redhat.com >
2026-04-10 17:35:03 +00:00
Jee Jee Li
ecd1ea1363
[Kernel] Porting the TRTLLM minimax_allreduce_rms kernels ( #37045 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
2026-04-11 00:20:20 +08:00
wang.yuqi
cb5f7501cb
[New Model]: jinaai/jina-reranker-v3 ( #38800 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-10 15:20:40 +00:00
Nicolò Lucchesi
c9dddc144b
[CI] Add Nixl+OffloadingConnector e2e integration tests ( #39200 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-10 21:40:40 +08:00
d468322dc1
[Kernel][Hardware][AMD] Add TritonW4A16LinearKernel for ROCm ( #37352 )
...
Signed-off-by: jatseng-ai <jatseng@amd.com >
Signed-off-by: jatseng-ai <janet.tseng@amd.com >
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Matthias Gehre <matthias.gehre@amd.com >
2026-04-10 10:25:27 +00:00
PatchyTIS and PatchouliTaisa
967146e7bd
[model] support FireRedLID ( #39290 )
...
Signed-off-by: PatchouliTaisa <patchychen@tencent.com >
Co-authored-by: PatchouliTaisa <patchychen@tencent.com >
2026-04-10 08:43:58 +00:00
8e8a3becd1
[ZenCPU] Make PT Backport Patch Accessible to vLLM ( #38205 )
...
Signed-off-by: Lalithnarayan C <Lalithnarayan.C@amd.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
2026-04-10 08:29:35 +00:00
Richard Zou
f44afef6d6
[compile] Allow strings in custom ops without regressing compilation times ( #38123 )
...
Signed-off-by: Richard Zou <zou3519@gmail.com >
2026-04-10 07:26:37 +00:00
