Jee Jee Li
75aa189ba0
[LoRA] Cleanup VocabParallelEmbedding ( #53843 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-08-26 16:19:35 +08:00
Giancarlo Delfin
d154d90d6c
[Model Runner V2] batch-sharded sample ( #50465 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-08-24 15:49:57 -07:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 and Jee Jee Li
702e1d7186
[LoRA] feat: Support LoRA for DeepSeek V4 ( #53361 )
...
Signed-off-by: Hollow Man <hollowman@opensuse.org >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-08-24 03:57:57 +00:00
Shuai Shao and Jee Jee Li
a60c66e3dc
[Bugfix] Fix int32 index overflow in LoRA punica kernels at long context ( #53034 )
...
Signed-off-by: Shuai Shao <ss@rungalileo.io >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-08-21 10:37:06 +00:00
linitra24
e85d1b69cf
[Bugfix][LoRA] Use an explicit capability flag for tower connector LoRA ( #53092 )
...
Signed-off-by: linitra24 <Joy25810@foxmail.com >
2026-08-21 09:45:54 +00:00
Harry Mellor
1fe3a1571a
Reduce AutoWeightsLoader kwargs ( #53106 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-20 15:40:11 +00:00
linitra24 and Jee Jee Li
9b5f3454f2
[LoRA] Avoid false target matches for unsupported module types ( #52313 )
...
Signed-off-by: linitra24 <Joy25810@foxmail.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-08-19 16:48:11 +00:00
Eilam and Jee Jee Li
c2e7242ab7
[Bugfix][LoRA] Guard None group members in expand_packed_lora (partial LoRA on Qwen3.5/3.6 GatedDeltaNet) ( #47640 )
...
Signed-off-by: Eilam C <eilamc14@gmail.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-08-19 13:24:26 +00:00
Nick Hill and Claude Opus 4.8
615d4cfade
[Core] Check for GPU<->CPU syncs during CI ( #43107 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-14 17:42:05 -07:00
95c9144424
[LoRA][Gemma4] Support vision tower LoRA ( #42662 )
...
Signed-off-by: bk-201 <joy25810@foxmail.com >
Signed-off-by: linitra24 <Joy25810@foxmail.com >
Signed-off-by: linitra24 <joy25810@gmail.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: linitra24 <joy25810@gmail.com >
2026-08-13 22:07:43 +08:00
Wentao Ye
fd04edef3e
[Refactor] Delete dead code in models ( #51838 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-11 15:44:26 -04:00
Nick Hill
9b0afeb4f6
[Perf] Avoid some more unnecessary GPU<->CPU syncs ( #51458 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-08 17:13:26 -07:00
Harry Mellor
e7b8d59460
Fully generalise input embedding handling in Transformers modelling backend ( #51247 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-06 16:21:24 +00:00
b50fdebce0
[Bugfix] Fix level-2 sleep/wake/reload with enable_lora=True ( #39935 )
...
Signed-off-by: Silen Naihin <silen.naihin@gmail.com >
Co-authored-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-08-06 09:13:10 +08:00
bnellnm
373fe8b83e
[MoE Refactor] Remove MoE legacy code ( #51078 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-08-05 14:35:36 -07:00
bnellnm and Harry Mellor
6e311c6e20
[MoE Refactor] Rename FusedMoE to FusedMoEFactory ( #44941 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-31 11:28:18 +01:00
f3e9497e92
[Model] Add Inkling LoRA support [4/N] ( #48884 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-17 09:52:15 +08:00
ErenAta16
59b964f37d
fix(lora): validate LoRA rank is positive in PEFTHelper ( #48437 )
...
Signed-off-by: ErenAta16 <erena6466@gmail.com >
2026-07-16 05:22:52 +00:00
Karthik Kothuri and Claude
4f7fffb92f
[Core][LoRA] Support fp32 lm_head (head_dtype) on the LoRA path ( #48525 )
...
Signed-off-by: Karthik Kothuri <karthikkothuri2009@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-15 09:51:09 +08:00
107a03ba63
[Core] Support fp32 lm_head for generation models via head_dtype (RFC #48305 §3.6) ( #48390 )
...
Signed-off-by: Karthik Kothuri <karthikkothuri2009@gmail.com >
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-07-13 16:43:34 +08:00
Alejandro Paredes La Torre
9a48eef89a
[Bugfix][LoRA] Support ark_linear base layer in _get_lora_device ( #47690 )
...
Signed-off-by: AlejandroParedesLT <alejandroparedeslatorre@gmail.com >
2026-07-12 00:13:50 +00:00
Chaojun Zhang
b8c7c86533
[XPU][LoRA] Fix torch.compile DEVICE_LOST by avoiding view-mutation in LoRA shrink ( #47944 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
2026-07-09 06:08:30 +00:00
392d1b4d2e
[BugFix][LoRA] Refresh punica metadata when LoRA slots are reassigned under an unchanged mapping ( #47725 )
...
Signed-off-by: AmeenP <ameenp360@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-07 08:53:55 -07:00
Kunshang Ji and Harry Mellor
07f9baf756
Revert "[Platform] Replace torch.cuda.Event with torch.Event ( #47140 )" ( #47668 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-06 14:18:33 +01:00
Chaojun Zhang
13c49f9845
[xpu][lora]: Align LoRA implementation with Punica GPU: fix _apply_expand rank mismatch, add_inputs hardcode, and MoE EP ( #45368 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
2026-07-01 22:14:04 +08:00
Micah Williamson
b446792306
[ROCm][Bugfix] Fix Triton "out of resource: shared memory" Error In One-Shot LoRA MoE ( #47209 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
2026-06-30 23:24:36 -07:00
Kunshang Ji and Harry Mellor
e840f0d3f5
[Platform] Replace torch.cuda.Event with torch.Event ( #47140 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-30 08:39:59 -07:00
Harry Mellor
1ab9522935
Remove more unnecessary load_weights methods ( #47058 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-30 15:22:16 +01:00
Martin Hickey
07d33e575b
[MyPy] Fix mypy incompatible assignment errors in LRUCacheLoRAModelManager ( #44657 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-06-29 16:42:35 +01:00
Harry Mellor
5051698e41
Remove unnecessary load_weights methods ( #44589 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-29 01:52:23 -07:00
Lynn and Claude Opus 4.8
0bc479e6eb
[Perf][LoRA] Replace O(n) list.index() with a dict in convert_mapping ( #46542 )
...
Signed-off-by: Lynn <lynnhe02@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-06-24 21:41:46 +08:00
e51e700470
[LoRA] Gate all_gather on fully_sharded_loras inside _mcp_apply; rewrite regression test ( #45715 )
...
Signed-off-by: lcheng <lcheng321@gatech.edu >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-06-23 07:08:33 -07:00
Martin Hickey
ccd49f6821
[MyPy] Fix mypy for vllm/lora ( #41722 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-06-22 10:57:09 -04:00
Nick Hill
7df3d7dada
[Core] Ensure memory is pinned prior to async h2d copy ( #45424 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-20 20:02:24 -07:00
9ea3a4015b
[Bugfix] Fix corrupt outputs in MoE FP8 LoRA responses and MoE base model responses when LoRAs are loaded ( #42120 )
...
Signed-off-by: Nicholas Edelman <nedelman@nvidia.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-06-18 18:26:09 -07:00
Mohammad Miadh Angkad
823a0ab754
[Bugfix][MoE] Fix fused MoE expert mapping helper call sites ( #44897 )
...
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com >
2026-06-08 13:35:04 -07:00
bnellnm
dc68bd8c41
[MoE Refactor] FusedMoE/MoERunner inversion refactor ( #41184 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-06-08 10:42:58 -04:00
linitra24
271328e256
[LoRA] Fix dedup for post-replacement module aliases ( #44413 )
...
Signed-off-by: bk-201 <joy25810@foxmail.com >
2026-06-03 18:23:23 +00:00
Lucain and Claude Opus 4.6
11dfa3169d
Add vLLM library info to Hugging Face Hub requests ( #43857 )
...
Signed-off-by: Wauplin <lucainp@gmail.com >
Signed-off-by: Lucain Pouget <lucain@huggingface.co >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-05-29 14:04:58 +00:00
Chaojun Zhang
861b97765d
[XPU] Fix fused MoE LoRA kernel crash on XPU by using platform-agnos num_compute_units ( #43646 )
...
Signed-off-by: Chaojun,Zhang <chaojun.zhang@intel.com >
2026-05-26 03:40:32 -07:00
Jee Jee Li
ec5de7fa7d
[LoRA] Add one shot triton kernel For MoE LoRA ( #42290 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
2026-05-25 19:47:04 -07:00
Jee Jee Li
15f7cd33dc
[LoRA] Reduce memory of 2D weights when EP is set ( #42737 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-05-22 06:41:56 -07:00
Nick Hill
b82e908b4c
[Perf][4/n] Eliminate various GPU<->CPU syncs ( #42347 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-05-19 10:35:54 -04:00
7d5b033782
[LoRA] Support 2D and 3D MoE LoRA adapter at the same time ( #42242 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-05-18 15:22:26 +08:00
Jee Jee Li
32b7177909
[LoRA][Bugfix] Dedup LoRA wrapping for modules referenced from multiple attribute paths (MoE gate) ( #42757 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
2026-05-16 11:22:35 +08:00
yupeng and Jee Jee Li
31fa757cf9
[Misc] Make it simpler to replace out-of-tree layer classes with related LoRA layers. ( #42306 )
...
Signed-off-by: paulyu12 <507435917@qq.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-05-15 15:20:42 +08:00
4d591db470
[MoE Refactor] Introduce RoutedExperts alias for FusedMoE and don't store SharedExperts in MK ( #40735 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-05-12 13:37:44 -04:00
bnellnm
6427603ae8
[MoE Refactor] Move remaining experts classes to experts directory ( #42334 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-05-12 09:19:46 -04:00
Isotr0py
48698b1b9b
[Bugfix] Fuse Qwen3.5 in_qkvz_proj forwarding with LoRA enabled ( #37912 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-05-10 10:59:02 +00:00
Jee Jee Li
ecd0b60aad
[LoRA] Initial EP support for LoRA ( #40867 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-05-09 00:31:23 -07:00
Jee Jee Li