iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,204 Commits 3 Branches 0 Tags
709eadbb0bff8fa00a22a3ade30327fecd61be4b
Commit Graph
279 Commits
Author SHA1 Message Date
Zhengxu Chen 5cc29cfb8b [compile] Improve error message during artifacts load failure. (#35115) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
 2026-02-24 22:01:09 -08:00
Rohan Potdar f38f8c9742 [ROCm]: Enable customop and rope+kvcache fusion for AITER RoPE (#35180) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-02-25 04:36:40 +00:00
Zhengxu Chen c77f3e1207 [compile] Save aot compile artifacts atomically. (#35117) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
 2026-02-24 04:11:01 -08:00
Rohan PotdarcharlifuLuka GovedičDouglas Lehr
 		2ff4e51152 [ROCm] AITER fused RoPE+KVCache (#33443) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: charlifu <charlifu@amd.com>
Signed-off-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com>
Co-authored-by: charlifu <charlifu@amd.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-02-23 19:06:00 -08:00
Zhengxu ChenandCyrus Leung f863994084 [compile] Fix torch.compile time discrepancy in logging. (#34912) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-20 08:47:14 -08:00
Zhengxu Chen e4a5d8c653 [compile] Move torch_aot_compile directory under torch_compile_cache (#34831) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
 2026-02-20 08:46:45 -08:00
Manrique Vargas ad5aa6bd9f fix(docs): fix typos in comments and docstrings (#34836) 
Signed-off-by: machov <mv1742@nyu.edu>
 2026-02-18 23:17:41 -08:00
ElizaWszolaandLuka Govedič a88b3be7c4 [Bugfix] Fix quant RMS norm fusion for quantization with TMA-aligned scales (#33255) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-02-17 23:35:04 -08:00
Richard Zou e30cedd44b [torch.compile] Stop doing unnecessary FakeTensorProp in PiecewiseCompileInterpreter (#34093) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-02-10 19:15:40 -08:00
Richard Zou 341eed3d30 [torch.compile] Disable recursive pre_grad_passes (#34092) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-02-10 18:02:31 -05:00
Charlie Fu bb9f97308d [torch.compile][Fusion] Fix attention fusion pass removing kv_udpate op. (#33945) 
Signed-off-by: charlifu <charlifu@amd.com>
 2026-02-09 16:15:43 -05:00
Mohammad Miadh Angkad d4f123cc48 [Kernel] FlashInfer: switch allreduce fusion to unified API (#33985) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-02-09 15:43:24 +00:00
Andrey Talman f97ca67176 [Release 2.10] Update to Torch 2.10 - final release (#30525) 2026-02-08 13:51:09 -08:00
Andreas Karatzas ab10d79855 [ROCm][Bugfix] fix act_quant_fusion module import error (#34069) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-07 19:21:12 -08:00
Richard Zou 81fe69cae5 [torch.compile] Stop compiling identical artifacts (#34003) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-02-07 05:24:48 -08:00
IkennaandLuka Govedič 906077181b [Bugfix] Fix QK Norm+RoPE fusion pattern matching on B200+FP8 (#33967) 
Signed-off-by: Ikenna <ikennachifo@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-02-07 02:27:33 +00:00
Wentao Ye 67a746e87f [Log] Optimize duplicate startup log (#33944) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-02-06 17:49:56 +00:00
Luka Govedič ac32e66cf9 [torch.compile] Reorganize vllm/compilation and tests/compile (0/N for vLLM IR) (#33731) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: ProExpertProg <luka.govedic@gmail.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-02-06 04:19:49 -08:00
emricksini-h 325ab6b0a8 [Feature] OTEL tracing during loading (#31162) 2026-02-05 16:59:28 -08:00
Richard Zou 9f14c9224d Revert "[torch.compile] Significantly speed up cold start times" (#33820) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-02-04 21:59:59 +00:00
Zhengxu Chen bcd2f74c0d [compile] Clean up AOT compile bypass on evaluate_guards. (#33578) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
 2026-02-04 02:12:53 -08:00
Wentao Ye 5e1e0a0fbd [Refactor] Remove unused dead code (#33718) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-02-03 21:25:11 -08:00
Richard Zou b1bb18de8d [torch.compile] Significantly speed up cold start times (#33641) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-02-03 09:12:11 -08:00
Richard Zou fd9c83d0e0 [torch.compile] Document the workaround to standalone_compile failing (#33571) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-02-03 07:16:55 +00:00
Luka GovedičandRichard Zou 15f40b20aa [fix][torch.compile] Fix cold-start compilation time increase by adding kv cache update to splitting ops (#33441) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Richard Zou <zou3519@gmail.com>
 2026-01-31 06:48:34 -08:00
Angela Yi 608b556507 [ez] Add structured torch.compile logs (#33213) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2026-01-31 21:00:54 +08:00
Angela Yi 5a66c9cc76 [ez] Delete torch25_custom_graph_pass (#33287) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2026-01-29 16:47:05 +00:00
Angela Yi 07ea184f00 [ez] Delete more torch version checks <= 2.8 (#33288) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2026-01-29 05:28:46 +00:00
Angela Yi 4197168ea5 [ez] Remove checks for torch version <= 2.8 (#33209) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2026-01-28 16:03:56 -05:00
Rohan Potdar 59bcc5b6f2 Use aiter triton fused_add_rmsnorm_pad for gpt-oss (#30976) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-01-28 20:47:47 +00:00
Harry Mellor f1acbd68c5 [CI] Enable mypy import following for vllm/compilation (#33199) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-01-28 08:59:54 +00:00
Harry Mellor 2eb673a088 Add flake8-implicit-str-concat rules to Ruff (#33191) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-01-28 04:56:10 +00:00
Matthew Bonanni a608b4c6c2 [5/N][Attention] Finish eliminating vllm/attention folder (#32064) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-27 10:02:51 -05:00
Roberto L. Castro fcb9df99bd [Perf][Kernel] Optimize FP4 quantization kernels (SM100F) (#32520) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
 2026-01-24 18:45:27 -07:00
Luka GovedičandMichael Goin 5e4e0e51f4 [torch.compile] Compile CustomOp.forward_native for SiluAndMul and QuantFP8 to avoid raw torch ops inside opaque custom ops (#32806) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-01-22 19:52:26 -08:00
Lucas Kabela 15e302dfce [Misc][BE] Turn on strict type coverage for vllm/compilation (#31756) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-01-22 15:12:26 +00:00
Robert Shaw 42135d6898 [MoE Refactor] Oracle Select FP8+NVFP4 Kernels In Priority (#32414) 2026-01-21 08:22:33 -05:00
Lucas WilkinsonandMatthew Bonanni 2261340806 [Misc] Remove pad_for_cudagraphs from config (#30143) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-20 15:05:48 -05:00
dolpm 7c5dedc247 [AOT compilation] support torch.compile inductor artifacts in VllmCompiledFunction (#25205) 
Signed-off-by: dolpm <34420038+dolpm@users.noreply.github.com>
 2026-01-20 19:45:59 +00:00
Nicolò Lucchesi 74c583bc50 [Core] Whisper support torch.compile (#30385) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-01-19 10:02:31 +00:00
Richard Zou bd292be0c0 [BugFix] Python file source reading can fail on UnicodeDecodeError (#32416) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-01-15 20:01:41 +00:00
Angela Yi 7933638051 [misc] Remove is_torch_equal_or_newer(2.4) cases (#32296) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2026-01-13 23:22:07 -08:00
cjackal 15b33ff064 [Misc] improve warning/assert messages (#32226) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-01-13 03:11:23 +00:00
Lucas Kabela ad8818bb5e [Misc][BE] Type coverage for vllm/compilation [3/3] (#31748) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-01-12 19:24:38 +00:00
Laith Sakka 46eb30f519 make assume_32_bit_indexing configurable (#32044) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2026-01-10 23:15:46 -08:00
Kevin McKayClaude Opus 4.5Cyrus Leung
 		4dc0d606b7 [Bugfix] Narrow broad exceptions in compilation backends (#31616) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-01-09 21:39:22 -05:00
Lucas Kabela aaf4b70aae [Misc][BE] Type coverage for vllm/compilation [2/3] (#31744) 2026-01-09 18:30:38 -05:00
DevByteAIandLuka Govedič 1f214290d6 fix(compile): apply partition wrapper when loading AOT cached functions (#31536) 
Signed-off-by: Devbyteai <abud6673@gmail.com>
Signed-off-by: DevByteAI <161969603+devbyteai@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-01-08 17:27:26 +08:00
Lucas Kabela 873480d133 [Misc][BE] Type coverage for vllm/compilation [1/3] (#31554) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-01-06 20:37:51 -05:00
Nick Hill bd877162eb [BugFix] Support online dense model DP without overhead (#30739) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: njhill <nickhill123@gmail.com>
 2026-01-02 23:36:38 +08:00