iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
16,919 Commits 3 Branches 0 Tags
1806d1adfc9b598bc6eb94de38a330aaad04c291
Commit Graph
73 Commits
Author SHA1 Message Date
Laith Sakka 6f20f81cbf Replace shape_invariants with simpler apprach in dynamic_arg_dims utilizing shape_id property. (#36194) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2026-04-29 18:32:15 +00:00
Lucas Kabela 0283f303d8 [BE] Fix compile time message to be consistent (use monitoring) (#40641) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-04-23 00:12:08 +00:00
Wentao Yeandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 8f87eb4622 [Refactor] Clean up log once scope="local" (#40540) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-22 16:42:43 -04:00
Alessandro Sangiorgi 2faad08362 [compile] Nest inductor cache under AOT compile dir (#39718) 
Signed-off-by: Alessandro Sangiorgi <asangior@redhat.com>
 2026-04-14 17:17:54 +00:00
Harry Mellor dfe5e31689 Don't compile vision encoder for Transformers backend (#30518) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-04-02 12:42:29 +00:00
Richard Zou 71a4a2fbd0 [BugFix] Fix order of compile logging (#38012) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-03-24 18:58:18 +00:00
Simon Mo 1fa1e53a73 Revert "[compile] Initialize passes at VllmBackend init" (#37733) 2026-03-20 21:35:49 -07:00
Angela Yi 12fd17eb51 [compile] Initialize passes at VllmBackend init (#35216) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2026-03-20 11:40:33 -07:00
Lucas Kabela 7769b58307 [torch.compile][BE][Multimodal] Remove requirement to set_model_tag to avoid cache conflict (#37345) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-03-19 17:26:12 +00:00
Richard Zou 09b6f99852 [compile] aot_compile should respect VLLM_DISABLE_COMPILE_CACHE (#36358) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-03-11 03:12:03 -07:00
Lucas Kabela 3fd03f1ec2 [BE] Rename should_torch_compile_mm_vit to should_torch_compile_mm_encoder (#36281) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-03-09 18:22:05 +00:00
Richard Zou 2dde535df1 [compile] Split compile/warmup monitoring (#36098) 2026-03-07 13:52:11 -08:00
Zhengxu Chen 2ed4722e26 [compile] Reduce log spam from compile. (#36044) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
 2026-03-05 00:48:36 +00:00
Richard Zou 5569f5218d [torch.compile] Stop lazily compiling (#35472) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-03-04 12:13:17 -08:00
Zhengxu Chen 5cc29cfb8b [compile] Improve error message during artifacts load failure. (#35115) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
 2026-02-24 22:01:09 -08:00
Zhengxu Chen c77f3e1207 [compile] Save aot compile artifacts atomically. (#35117) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
 2026-02-24 04:11:01 -08:00
Zhengxu Chen e4a5d8c653 [compile] Move torch_aot_compile directory under torch_compile_cache (#34831) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
 2026-02-20 08:46:45 -08:00
Andrey Talman f97ca67176 [Release 2.10] Update to Torch 2.10 - final release (#30525) 2026-02-08 13:51:09 -08:00
Angela Yi 4197168ea5 [ez] Remove checks for torch version <= 2.8 (#33209) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2026-01-28 16:03:56 -05:00
Lucas Kabela 15e302dfce [Misc][BE] Turn on strict type coverage for vllm/compilation (#31756) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-01-22 15:12:26 +00:00
dolpm 7c5dedc247 [AOT compilation] support torch.compile inductor artifacts in VllmCompiledFunction (#25205) 
Signed-off-by: dolpm <34420038+dolpm@users.noreply.github.com>
 2026-01-20 19:45:59 +00:00
Nicolò Lucchesi 74c583bc50 [Core] Whisper support torch.compile (#30385) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-01-19 10:02:31 +00:00
Angela Yi 7933638051 [misc] Remove is_torch_equal_or_newer(2.4) cases (#32296) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2026-01-13 23:22:07 -08:00
Laith Sakka 46eb30f519 make assume_32_bit_indexing configurable (#32044) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2026-01-10 23:15:46 -08:00
Lucas Kabela aaf4b70aae [Misc][BE] Type coverage for vllm/compilation [2/3] (#31744) 2026-01-09 18:30:38 -05:00
DevByteAIandLuka Govedič 1f214290d6 fix(compile): apply partition wrapper when loading AOT cached functions (#31536) 
Signed-off-by: Devbyteai <abud6673@gmail.com>
Signed-off-by: DevByteAI <161969603+devbyteai@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-01-08 17:27:26 +08:00
Nick Hill bd877162eb [BugFix] Support online dense model DP without overhead (#30739) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: njhill <nickhill123@gmail.com>
 2026-01-02 23:36:38 +08:00
Zhengxu Chen 177c391db2 [compile] Disable aot when eager backend is used. (#30810) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
 2025-12-17 01:55:56 -08:00
Laith Sakka f569c654e1 enable unbacked with aot_compile (#30462) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2025-12-14 08:14:06 +00:00
Laith Sakka 763963aa73 set assume_32bit_indexing and pass unbacked hints (#30459) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2025-12-13 15:36:53 +00:00
Laith Sakka 87aee9ed2b Add evaluate_guards option to DynamicShapesConfig (#27432) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2025-12-08 10:46:15 -05:00
Angela Yi e7296b08da [bugfix] Pass globals to aot_compiled function (#29428) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2025-12-05 16:54:26 +00:00
Laith Sakka 1f0d184590 [aot_compile]change VLLM backend to read fake args from example_value (#29104) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2025-12-04 17:33:45 -05:00
Laith Sakka 7a228b5305 Add option to use unbacked, and backed size obl dynamic shapes for more sounds compilation. (#26199) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2025-11-24 10:12:41 -05:00
Didier Durand 2bb4435cb7 [Doc]: fix typos in various files (#28567) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-11-15 19:27:50 +00:00
Laith Sakka 2e0ad629b0 Avoid bytecode hook and simplify TorchCompileWrapperWithCustomDipatch (#25110) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2025-11-14 14:11:10 -08:00
Lucas Kabela 94666612a9 [Misc][qwen2_5_vl][torch.compile] Enable supports_torch_compile on generic nn.Module and demonstrate speedup on Qwen Vision model (#23207) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Signed-off-by: Lucas Kabela <lucasakabela@gmail.com>
 2025-10-28 22:36:43 +00:00
Zhengxu ChenandCyrus Leung e3d8186666 [compile] Add fallback path to AOT compile when serialization fails. (#27350) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-10-28 12:54:26 -04:00
Zhengxu ChenandCyrus Leung a00d6254e9 [compile] Disable dynamo guards check for AOT compilation. (#27288) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-10-28 12:58:12 +00:00
Isotr0py 6ac5e06f7c [Chore] Clean up pytorch helper functions in vllm.utils (#26908) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: isotr0py <2037008807@qq.com>
 2025-10-18 09:48:22 -07:00
Harry Mellor 6c9fdbf725 [Docs] Replace rst style double-backtick with md single-backtick (#27091) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-17 02:47:34 -07:00
Cyrus Leung 4d4d6bad19 [Chore] Separate out vllm.utils.importlib (#27022) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-17 00:48:59 +00:00
Morrison TurnanskyandLuka Govedič 96b9aa5aa0 [Frontend][torch.compile] CompilationConfig Overhaul (#20283): name change compilation level to compilation mode, deprecation compilation level (#26355) 
Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-10-15 02:51:16 +00:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Zhengxu Chen eef921f45e AOT Compilation for torch.compile (Bundled) (#24274) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
 2025-10-10 19:02:11 -04:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
fhl2000andLuka Govedič f075693da7 [V1] address post issues related to #20059 (part 1) (#23046) 
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-26 15:58:19 -04:00
Kyle Sayers de94289a98 [Core] Support weight_loader_v2 for UnquantizedLinearMethod (#23036) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-09-23 18:30:26 -06:00
Boyuan FengandLuka Govedič 8945b001db [torch.compile] CUDAGraph Inductor partition integration (#24281) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Boyuan Feng <fby.1994@gmail.com>
Signed-off-by: boyuanfeng <boyuan@meta.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-20 01:02:15 +00:00
Yong Hoon Shin dfd2382039 [torch.compile] Support conditional torch.compile per module (#22269) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-08-20 16:52:59 +00:00