iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
50 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Luka Govedič 02e8f26cea [torch.compile] Turn on silu+fp4 quant fusion by default for O1+ (#34718) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
 2026-02-18 03:29:15 +00:00
Luka Govedičandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 23d825aba1 [torch.compile] Disable ar-rms fusion for ds3-fp4 & DP, fix CI test (#34392) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-15 06:33:57 -08:00
Luka Govedič addac0e653 [torch.compile] Enable AR+rms fusion by default available for -O2 (#34299) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
 2026-02-11 00:30:00 -08:00
Luka Govedič 781ddf7868 [CI][torch.compile] Fix incorrect filtering for E2E fusion tests on B200 (#34031) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
 2026-02-09 10:05:14 -05:00
Luka Govedič ac32e66cf9 [torch.compile] Reorganize vllm/compilation and tests/compile (0/N for vLLM IR) (#33731) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: ProExpertProg <luka.govedic@gmail.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-02-06 04:19:49 -08:00
Luka Govedič e3bf79ffa0 Revert "[Attention][FA3] Update FA3 to include new swizzle optimization" (#33841) 2026-02-04 19:54:27 -08:00
Luka Govedič 4d9513537d [CI][torch.compile] Reduce e2e fusion test time (#33293) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: ProExpertProg <luka.govedic@gmail.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-02-04 19:09:03 -05:00
Luka Govedič 672023877b Change defaults for vllm bench startup (#33489) 2026-01-31 23:46:01 -08:00
Luka GovedičandRichard Zou 15f40b20aa [fix][torch.compile] Fix cold-start compilation time increase by adding kv cache update to splitting ops (#33441) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Richard Zou <zou3519@gmail.com>
 2026-01-31 06:48:34 -08:00
Luka Govedič bbbd696af9 [torch.compile][CI] Add back attn fusion on hopper/ada (#32940) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
 2026-01-23 16:49:20 +00:00
Luka GovedičandMichael Goin 5e4e0e51f4 [torch.compile] Compile CustomOp.forward_native for SiluAndMul and QuantFP8 to avoid raw torch ops inside opaque custom ops (#32806) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-01-22 19:52:26 -08:00
Luka Govedič bd7157a071 [torch.compile] Enable attention and allreduce fusion without custom ops enabled (#24604) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-10-17 08:10:23 -06:00
Luka Govedič 2dcd12d357 [torch.compile] Fix tests for torch==2.9 inductor partition (#26116) 
Signed-off-by: ProExpertProg <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
 2025-10-14 19:55:02 -04:00
Luka Govedič d588cd2406 [Bugfix] fix custom op test (#25429) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
 2025-09-23 00:07:43 +00:00
Luka Govedič d5e0fca264 [torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) (#24542) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-22 12:30:05 -07:00
Luka Govedič 4f35be10a9 [BugFix] Fix topk_softmax assert (#19764) 
Signed-off-by: Luka Govedic <lgovedic@redhat.com>
 2025-08-27 09:47:28 -07:00
Luka Govedič 4e8614e88b Move checklist in PR template (#22852) 
Signed-off-by: Luka Govedic <lgovedic@redhat.com>
 2025-08-13 21:38:35 +00:00
Luka Govedič 762be26a8e [Bugfix] Upgrade depyf to 0.19 and streamline custom pass logging (#20777) 
Signed-off-by: Luka Govedic <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
 2025-07-11 00:15:22 -07:00
Luka Govedičandmgoin 31d5c1797f [Perf][fp8] Use CustomOp abstraction for fp8 quant for better perf (#19830) 
Signed-off-by: Luka Govedic <lgovedic@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-11 04:56:28 +00:00
Luka Govedič 6d42ce8315 [CLI] Improve CLI arg parsing for -O/--compilation-config (#20156) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-07-01 01:03:13 +00:00
Luka Govedič aafabaa0d5 [Fix][torch.compile] Enable custom ops by default when Inductor off (#20102) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-06-27 09:00:42 -06:00
Luka Govedič 3597b06a4f [CUDA] Enable full cudagraph for FlashMLA (#18581) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-06-13 18:12:26 +00:00
Luka GovedičandSage Moore f98548b9da [torch.compile][ROCm] Fuse quantization onto attention using a torch.compile pass (#16756) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
 2025-06-12 08:31:04 -07:00
Luka Govedič 2d8476e465 [BugFix][V1] Fix memory profiling bug (#18974) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-06-07 10:34:51 -07:00
Luka Govedič c55d804672 [BugFix] Pydantic part 2 (#18911) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-05-31 03:39:28 -07:00
Luka Govedič f274581f44 [BugFix] Update pydantic to fix error on python 3.10 (#18852) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-05-29 03:05:46 -07:00
Luka Govedič a3896c7f02 [Build] Fixes for CMake install (#18570) 2025-05-27 20:49:24 -04:00
Luka Govedič 83f74c698f [Fix][ROCm] Enforce eager for all encoder-decoder models on ROCm (#18154) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
 2025-05-14 22:04:43 -07:00
Luka Govedič 176a95c670 [Fix] Support CUDAGraph capture for encoder-decoder on ROCm (#18104) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
 2025-05-13 19:31:42 -07:00
Luka Govedič aaec845f8e [ROCm] [Attention] Cleanup ROCm output passing (#16431) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
 2025-04-18 05:46:45 +00:00
Luka Govedič 9cdde47289 [BugFix] Fix fusion test and add them to CI (#16287) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-04-08 23:46:45 -07:00
Luka Govedič 04437e313d [Bugfix] [torch.compile] Add Dynamo metrics context during compilation (#15639) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-03-28 14:01:09 -06:00
Luka Govedič f622dbcf39 [Fix] [torch.compile] Improve UUID system for custom passes (#15249) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-03-24 01:54:07 +00:00
Luka Govedič e1744502c2 [FP8] Refactor apply_fp8_linear and apply_fp8_linear_generic into an object (#14390) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-03-07 05:20:16 +00:00
Luka Govedič bd56c983d6 [torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename RedundantReshapesPass to NoopEliminationPass (#10902) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-02-28 16:20:11 -07:00
Luka GovedičandVarun Sundar Rabindranath 30870b4f66 [torch.compile] Dynamic fp8 + rms_norm fusion (#10906) 
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2024-12-13 03:19:23 +00:00
Luka Govedič 8b0fe06c89 [torch.compile] Inductor code caching fix (#10273) 
Signed-off-by: luka <luka@neuralmagic.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
 2024-11-20 21:44:57 -08:00
Luka GovedičandDaniël de Kok bf2ddc6610 [bugfix] Fix static asymmetric quantization case (#10334) 
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
 2024-11-15 09:35:11 +08:00
Luka Govedičandyoukaichao 4f93dfe952 [torch.compile] Fuse RMSNorm with quant (#9138) 
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: youkaichao <youkaichao@126.com>
 2024-11-08 21:20:08 +00:00
Luka Govedič 87713c6053 [CI/Build] Ignore .gitignored files for shellcheck (#10162) 
Signed-off-by: luka <luka@neuralmagic.com>
 2024-11-08 19:53:36 +00:00
Luka Govedič 51c24c9736 [Build] Fix FetchContent multiple build issue (#9596) 
Signed-off-by: luka <luka@neuralmagic.com>
 2024-10-23 12:43:07 +08:00
Luka Govedič 0f41fbe5a3 [torch.compile] Fine-grained CustomOp enabling mechanism (#9300) 2024-10-17 18:36:37 +00:00
Luka Govedič 172d1cd276 [Kernel] AQ AZP 4/4: Integrate asymmetric quantization to linear method (#7271) 2024-09-27 14:25:10 -04:00
Luka GovedičandYuan 57a0702e63 [Bugfix] Fix CPU CMake build (#8723) 
Co-authored-by: Yuan <yuan.zhou@intel.com>
 2024-09-22 20:40:46 -07:00
Luka Govedič 71c60491f2 [Kernel] Build flash-attn from source (#8245) 2024-09-20 23:27:10 -07:00
Luka Govedič 5d73ae49d6 [Kernel] AQ AZP 3/4: Asymmetric quantization kernels (#7270) 2024-09-16 11:52:40 -07:00
Luka GovedičandMichael Goin 7937009a7e [Kernel] Replaced blockReduce[...] functions with cub::BlockReduce (#7233) 
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2024-08-21 20:18:00 -04:00
Luka Govedič 5fb4a3f678 [Bugfix][Kernel] Increased atol to fix failing tests (#7305) 2024-08-08 12:16:13 -04:00
Luka GovedičandTyler Michael Smith 8d59dbb000 [Kernel] Add per-tensor and per-token AZP epilogues (#5941) 
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-08-06 18:17:08 +00:00
Luka GovedičChih-Chieh-YangLucas Wilkinson
 		5bfd1bbc98 [Kernel] Adding bias epilogue support for cutlass_scaled_mm (#5560) 
Co-authored-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2024-06-26 15:16:00 +00:00