 sychen52
|
947138b6c2
|
Add nvfp4 kv cache support (#40177)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
|
2026-05-01 04:55:16 +00:00 |
|
|
|
0335316a9b
|
[BUG] Two phase pause to prevent deadlock (#39366)
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: Aaron Hao <ahao@anyscale.com>
Co-authored-by: Junjie Zhang <junj.jay.zhang@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-04-29 17:51:03 -04:00 |
|
|
Wei ZhaoandWei Zhao
|
8b49cf3a37
|
[Bugfix] Fix max_num_batched_token not captured in cuda graph (#40734)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Wei Zhao (Engrg-Hardware 1) <weizha@login-bia02.bia.clusters.nvidia.com>
|
2026-04-28 21:33:06 -07:00 |
|
|
Roy Wang
|
9e92de51c6
|
[Bugfix] Exclude numa_bind fields from ParallelConfig DP hash (#41098)
Signed-off-by: yasong <yasong.wang@inferact.ai>
|
2026-04-28 15:52:54 +08:00 |
|
|
|
7a1eb8ac2e
|
[Model] update for mimo v25 (#41029)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Copilot <copilot@github.com>
|
2026-04-27 21:52:54 -07:00 |
|
|
Matthew Bonanni
|
fd74c90d9c
|
[Attention][Spec Decode] Allow independent drafter attention backend selection (#39930)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-04-27 19:38:09 -07:00 |
|
|
|
c245d35ff4
|
[Model] Add MiMo-V2.5 support (#40967)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <zhujiangyun@inferact.ai>
Co-authored-by: yasong <yasong.wang@inferact.ai>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Copilot <copilot@github.com>
|
2026-04-27 13:26:51 +00:00 |
|
|
Zhanda ZhuandYubo Gao
|
5d5c776444
|
[Perf] FP8 FlashInfer Attn for ViT (#38065)
Signed-off-by: Zhanda Zhu <zhandazhu@gmail.com>
Co-authored-by: Yubo Gao <ybgao-nvidia@users.noreply.github.com>
|
2026-04-27 13:44:15 +08:00 |
|
|
Giancarlo Delfin
|
f5f9878514
|
[Model Runner V2] Fix rejection sampling acceptance rate gap vs MRV1 (#40651)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-04-26 19:12:08 -07:00 |
|
|
+7
|
4d51588e23
|
[Feat] DeepSeek V4 Rebased (#40860)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
|
2026-04-26 18:31:08 -07:00 |
|
|
|
32e45636e3
|
[torch.compile]: Disable Sequence Parallelism (SP) for piecewise compilation (#38373)
Signed-off-by: SouthWest7 <am1ao@qq.com>
Signed-off-by: Xinan Miao <1403572259@qq.com>
Co-authored-by: SouthWest7 <am1ao@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Wang Xingran <72983099+wangxingran222@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-04-26 17:44:42 +00:00 |
|
|
Lucas Kabela
|
ce6a199ecc
|
[BE][Bugfix] Respect TORCH_COMPILE_DISABLE env var at the vLLM config level for torch 2.12 (#40715)
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
|
2026-04-24 16:25:03 -07:00 |
|
|
Jinzhen Lin
|
9f771b3ab9
|
[Quantization] add humming quantization kernel (#34556)
|
2026-04-24 09:29:44 -04:00 |
|
|
Yuwen ZhouandKunshang Ji
|
4c34b2f6fc
|
[XPU] Enable torch.compile for XPU GDN attention (#39466)
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Yuwen Zhou <yuwen.zhou@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-04-24 16:26:16 +08:00 |
|
|
Benjamin Chislettandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
626daa2076
|
[Feat] Unified Synthetic Acceptance Rate for V1 and V2 (#40662)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-04-24 00:48:08 +00:00 |
|
|
stevenkuangandJee Jee Li
|
d0009ddb0b
|
[Model] Support Hy3 preview (#40681)
Signed-off-by: stevenkuang <stevenkuang@tencent.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-04-23 22:08:26 +08:00 |
|
|
Isotr0py
|
da1e7311ca
|
[Misc] use model arch converter for bidi models identification (#40701)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-04-23 13:42:52 +00:00 |
|
|
Wentao Yeandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
8f87eb4622
|
[Refactor] Clean up log once scope="local" (#40540)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-04-22 16:42:43 -04:00 |
|
|
fxmarty-amdandKyle Sayers
|
d622e27d2b
|
[NVFP4] NVFP4 MOE emulation fallback for H100/MI300/MI350, standardize TritonExperts usage for OCP MX emulation (#35737)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: Kyle Sayers <kylesayrs@gmail.com>
|
2026-04-22 08:58:54 -07:00 |
|
|
AllenDouandzixiao
|
9047288b68
|
support hotwords for FunASR model (#39674)
Signed-off-by: zixiao <shunli.dsl@alibaba-inc.com>
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
|
2026-04-22 02:25:06 -07:00 |
|
|
|
4eafc72928
|
[Audio] Bundle get_generation_prompt() params into SpeechToTextParams (#36268)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2026-04-22 12:24:18 +08:00 |
|
|
Matthew BonanniandTyler Michael Smith
|
96a85c5750
|
[Startup][UX] Enable CUDAGraph memory profiling by default (#38284)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-04-21 18:16:59 -04:00 |
|
|
Shanshan Shen
|
936e0b79aa
|
[MM][CG] Optimize default max_frames_per_batch auto-infer for ViT CUDA graph video inference (#40445)
Signed-off-by: shen-shanshan <467638484@qq.com>
|
2026-04-21 14:47:53 +00:00 |
|
|
Rita Brugarolas
|
21b086d0aa
|
[ROCm] Hotfix: guard MLA dual RMS norm fusion against older AITer versions (#40386)
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com>
|
2026-04-20 16:20:05 -05:00 |
|
|
Rita Brugarolas
|
fb5635d3f9
|
[ROCm] Add MLA dual RMS norm fusion (Q, KV) pass for DeepSeek/Kimi-K2 (#39242)
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com>
|
2026-04-20 14:56:27 +00:00 |
|
|
Vasiliy Kuznetsov
|
38fa87caca
|
mxfp8 online quant move to new frontend (#40152)
Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com>
|
2026-04-20 06:26:12 -07:00 |
|
|
Ilya Markov
|
50dd4cb427
|
[EPLB] Add nixl-based eplb communicator (#36276)
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
|
2026-04-20 10:24:23 +00:00 |
|
|
sychen52
|
6b2b7bd0eb
|
Add nvfp4 support to reshape_and_cache_flash (#37332)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
|
2026-04-17 07:28:00 -07:00 |
|
|
Li, Jiang
|
d02421a7db
|
[CPU] Refactor CPU affinity and memory management (#39781)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-04-17 21:01:08 +08:00 |
|
|
Asaf Gardin
|
ad2b1277f9
|
[Quantization] Consolidate experts_int8 with fp8 online quantization (#38463)
Signed-off-by: Josephasafg <ajgard7@gmail.com>
|
2026-04-16 13:12:20 -07:00 |
|
|
Collin McCarthy
|
27c0ca50a0
|
Update registry for Nemotron-v3 VL Nano/Super (#39747)
Signed-off-by: Collin McCarthy <cmccarthy@nvidia.com>
|
2026-04-15 16:09:11 -07:00 |
|
|
|
f4b42df048
|
[Attention Backend] TurboQuant: 2-bit KV cache compression with 4x capacity (#38479)
Signed-off-by: vibhavagarwal5 <vibhavagarwal5@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-04-14 19:57:13 -07:00 |
|
|
roikoren755
|
ecf5ff7ce3
|
[Mamba] Flashinfer selective_state_update (#36162)
Signed-off-by: Roi Koren <roik@nvidia.com>
|
2026-04-14 15:10:58 -04:00 |
|
|
Lucas Kabela
|
c08f3b2a62
|
Measure encoder compile time seperate from llm backbone (#39240)
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
|
2026-04-14 10:52:49 -04:00 |
|
|
bnellnmandRobert Shaw
|
e1e318af01
|
[MoE Refactor] Remove MoE DP chunking (#39107)
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-04-14 09:48:05 -04:00 |
|
|
Shanshan ShenandRoger Wang
|
80118853f4
|
[MM][Perf][CG] Support ViT full CUDA graph for Qwen3-VL video inference (#38061)
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-04-14 16:49:32 +08:00 |
|
|
Yongye Zhu
|
739e5945dc
|
[Quantization] [Refactor] Create special "GptOssMxfp4MoeMethod" (#39604)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-04-13 12:53:58 -04:00 |
|
|
Tihomir Elek
|
8d825b87d6
|
[Bug] Fix TypeError when hf_config.architectures is None during model loading (#38849)
Signed-off-by: Tihomir Elek <tiho.elek@gmail.com>
|
2026-04-13 12:13:21 +01:00 |
|
|
|
fa6ae31177
|
feat: rename logit_bias/logit_scale to logit_mean/logit_sigma for affine score calibration (#39530)
Signed-off-by: Jesus Federico <jefp@amazon.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-04-13 04:43:44 +00:00 |
|
|
Jee Jee Li
|
715681c127
|
[LoRA] Support dual CUDA streams-Linear Layer (#35721)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-04-13 10:57:07 +08:00 |
|
|
Nick Hill
|
ee3c0c83db
|
[Pooling] Disable async scheduling by default for pooling models (#39592)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-04-12 07:23:42 +00:00 |
|
|
Martin HickeyandOr Ozeri
|
cc07dad789
|
[HMA] [KVEvent] Enable GPU-side KV events for HMA (#37688)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Or Ozeri <or@ozery.com>
|
2026-04-12 10:01:02 +03:00 |
|
|
|
e7cfd7c5b9
|
Add Gemma4 Eagle3 support (#39450)
Signed-off-by: Rahul-Tuli <rtuli@redhat.com>
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
Co-authored-by: Rahul-Tuli <rtuli@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-04-10 12:35:35 -07:00 |
|
|
Jesus FedericoandClaude Opus 4.6
|
b87575d24b
|
feat: add logit_scale to PoolerConfig for affine score calibration (#39435)
Signed-off-by: Jesus Federico <jefp@amazon.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-04-10 17:21:14 +00:00 |
|
|
Jee Jee Li
|
ecd1ea1363
|
[Kernel] Porting the TRTLLM minimax_allreduce_rms kernels (#37045)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-04-11 00:20:20 +08:00 |
|
|
zhrrr
|
8f121f7879
|
[Model Runner V2] support auto resolve cudagraph mode/sizes based on attn backend (#32936)
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
|
2026-04-10 08:27:15 -07:00 |
|
|
Kyungmin Leeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
e7a1387e73
|
Add EXAONE-4.5 (#39388)
Signed-off-by: lkm2835 <lkm2835@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-04-09 20:53:26 -07:00 |
|
|
Chauncey
|
ecbfbb8d61
|
[Feature] Add auto-detection for reasoning_config when only reasoning_parser is set (#38214)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-04-10 01:36:26 +00:00 |
|
|
|
75e01a39a1
|
[Feature] NUMA binding support for GPU workers (#38635)
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: Jason Li <jasonlizhengjian@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-04-08 09:55:24 -07:00 |
|
|
Lucas Wilkinson
|
70406eb1dc
|
[Attention][V0 Deprecation] Deprecate accept output buffer (#39125)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-04-07 17:14:58 -04:00 |
|