iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,341 Commits 3 Branches 0 Tags
patches
Commit Graph
445 Commits
Author SHA1 Message Date
afriedrimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Claude Opus 5.5
 		cc2e4fecf2 [ROCm] [fusion] support ROCm aiter fusion of per tensor quant + rmsnorm (#52831) 
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 15:19:07 -05:00
Jack HuandDouglas Lehr ef38e8d006 [Model][Hardware][AMD][Kernel] Part 2/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_ATTN (#42755) 
Signed-off-by: Jack Hu <Jack.Hu@amd.com>
Signed-off-by: Jack Hu <jingxujaster@hotmail.com>
Signed-off-by: Jack Hu <jack.hu@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-10-15 17:10:14 -07:00
TaimysTaimysHarry Mellor
 		c068db0e1d [Bugfix][Compile] Replace Transformers AOT caching with graph caching (#58918) 
Signed-off-by: fangyao fan <2281693949@qq.com>
Signed-off-by: Taimys <ffy2281693949@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Taimys <ffy2281693949@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-10-13 19:17:29 +00:00
Mikko Tukiainen abad32a718 [ROCm][Perf] Fix Gemma AR+RMS prefill fallback (#58547) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Signed-off-by: Mikko Tukiainen <mikko.tukiainen@amd.com>
 2026-10-12 19:53:08 +00:00
stefankoncarevic 8c175f403a [CI][ROCm] Compare greedy output only in the AITER attn+quant test (#61185) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-10-12 08:20:35 -05:00
Nick Hill c059bee812 [MRV2] Switch CI tests still running on MRV1 unnecessarily (#61014) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-11 09:57:07 -07:00
stefankoncarevic b74233860d [CI][ROCm] Wait for VRAM to settle before building LLM in test_full_graph (#60872) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-10-09 16:48:43 +00:00
Woosuk Kwon 194bd72be7 [Compile] Remove torch.compile-based sequence parallelism and AsyncTP (#60517) 2026-10-08 08:28:23 -07:00
Cheese CakeOpenAI CodexClaude Opus 5.5
 		b9767299db [Bugfix] Respect Inductor deterministic mode in combo-kernel defaults (#59074) 
Signed-off-by: cheesecake <farzanaman99@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 20:08:11 +01:00
eky-amdandClaude Fable 5 29f955cce0 [ROCm] Fuse MLA dual RMSNorm + FP8 group quant for DeepSeek-R1 (#54857) 
Signed-off-by: Ethan Ky <ethan.ky@amd.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-10-06 11:26:38 -05:00
Divakar Verma 2a537887de [CI] Use vllm_runner in fusions_e2e conftest for reliable GPU cleanup (#59525) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-10-02 11:21:21 -05:00
Stefano CastagnettaandClaude Opus 5 f0b4b36890 [Bugfix][Quantization] Detect NVFP4 in ModelOpt mixed-precision checkpoints (#56050) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-10-02 12:30:22 +00:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
Jakub ZakrzewskiNicolò Lucchesimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		effdea9a53 [MM][Mistral] Add compile support for Pixtral vision encoders (#57168) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-30 20:27:12 +08:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 834ad456d3 [Refactor] Remove dead tests code (#58916) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-30 04:39:03 +00:00
Martin HickeyHarry MellorClaude Opus 5.5
 		5dd693122c [MyPy][2/N] Fix mypy errors in small tests/ dirs (#51043) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 21:27:45 +01:00
Jung JiyuandMisha Goin cb4f016c5d [Bugfix] Fix standalone torch.compile cache loading after relocation (#52142) 
Signed-off-by: jungjiyu <libraryofjiyu@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-29 12:53:20 -04:00
Andreas Karatzas 6ebb5bd64f [ROCm][CI] Expand MI355 mirrors and route MIG-sized jobs to DPX (#59137) 
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
 2026-09-29 03:40:07 -05:00
Andreas KaratzasandOpenAI Codex 78fc1e6274 [ROCm][CI] Add MI355 TP2 AR-RMS and B200 fusion mirrors (#58284) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-29 07:10:26 +00:00
Cheese CakeOpenAI CodexNick HillWentao Ye
 		4e59d53ab5 [MRV2] Support stock torch.compile mode (#59079) 
Signed-off-by: cheesecake <farzanaman99@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-28 20:17:48 -07:00
Laith Sakka b39f760cc7 [torch.compile] Canonicalize functionalized split slices for fusion pa… (#57299) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2026-09-28 20:21:59 -04:00
Thang NguyenandClaude Opus 5.5 66d2d4b2e7 [CI] Split Dynamic Shapes out of (H200 MIG 35GB) PyTorch Compilation + (MI300) mirror (#58451) 
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-28 00:32:11 -07:00
Andreas Karatzas 0239bd2549 [CI] Stabilize batch submission in full CUDA graph tests (#58810) 2026-09-26 11:35:56 -04:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		31cc226401 [ROCm][CI] Run the MLA attention+quant fusion test on ROCm (#58717) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 14:45:32 -05:00
fxmarty-amdmgoinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		90a9515006 [Cleanup] Remove online quantization support in fp8.py in favor of online shorthands (#53585) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 12:48:14 -07:00
Nick HillandKimi 074d57fce1 [Bugfix] Keep JIT warmup under enforce-eager when fault tolerance is on (#58593) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi <noreply@moonshot.cn>
 2026-09-24 12:07:32 -07:00
Andreas KaratzasandOpenAI Codex 23110a0c9e [Bugfix] Capture prefill kernels for mixed FULL graphs (#58275) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-24 02:05:07 -07:00
stefankoncarevicAndreas KaratzasAndreas KaratzasCodex
 		e9f167129c [CI][ROCM] Add the Fusion E2E TP2 Quick group on MI355, and the AITER MLA fix it needs (#58369) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-23 21:01:58 -05:00
vorapolsiloaiSantosh HiremathDouglas Lehr
 		b715699df4 [ROCm][Perf] Extend QK-norm/RoPE/KV-cache fusion to MRoPE (#50212) 
Signed-off-by: Vorapol Assavasangthong <Vorapol.Assavasangthong@amd.com>
Co-authored-by: Santosh Hiremath <Santosh.Hiremath@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-09-23 11:11:57 -05:00
Andreas KaratzasandOpenAI Codex 26879f3260 [ROCm][Compile] Fuse AITER static FP8 attention output (#58099) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-23 03:16:52 -05:00
Andreas Karatzas 0eb42dbbfc [ROCm][Compile] Support BF16 AsyncTP fusion (#58098) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-22 20:36:38 -05:00
Misha Goin 01e15bc94c [Bugfix] Backport Inductor custom-op pattern matching fix (#58189) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-22 14:26:49 -07:00
Mikko TukiainenandCursor 68363d64ec [ROCm][Perf] Avoid extra reshape kernel in Qwen GDN output norm (#47842) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-22 14:38:16 +08:00
Andreas KaratzasandOpenAI Codex 8902dbb177 [CI][ROCm] Mirror remaining portable suites without duplicate coverage (#57877) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-21 04:16:26 -05:00
Sheral KumarAndreas KaratzasCodexAndreas Karatzas
 		01f1f58f10 [ROCm][CI] Query HIP device memory for test GPU teardown waits. (#57450) 
Signed-off-by: Sheral Kumar <shekumar@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-20 15:20:39 -05:00
Andy LoandOpenAI Codex 09c379ab61 [Bugfix] Max-load throughput cliff when max_num_seqs is not a multiple of 8 (#57355) 
Signed-off-by: Andy Lo <andy@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-17 18:09:18 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Samuel NordmannmgoinClaude Opus 4.8ElizaWszolaOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		03dc26e639 [Perf][Kernel][Quantization] Fuse ReLU2 with static FP8 activation quantization (#53793) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: snordmann <snordmann@nvidia.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-14 10:26:15 -07:00
JooHo LeeAndreas KaratzasCyrus Leung
 		78e84261ab [Performance] Add Triton kernel for Gemma3n sparse GELU (#48498) 
Signed-off-by: BWAAEEEK <jooho414@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-14 07:42:15 +00:00
Qidong SuandOpenAI Codex 9e137d8d23 [Kernel] Enable optimized FlashInfer add-RMSNorm NVFP4 fusion (#51925) 
Signed-off-by: Qidong Su <qidongs@nvidia.com>
Signed-off-by: Qidong Su <soodoshll@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-08 17:20:32 -07:00
Roderick Wuandmgoin 2c9d68f80b [Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering (#54809) 
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com>
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-08 12:45:12 -07:00
sychen52andCyrus Leung 1b2c591cd0 [Bugfix] speedup nvfp4 kv for FMHA (#55031) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-08 17:25:42 +00:00
Jiangyun ZhukhluujiahaoKimi Code
 		8ebc5b0a18 [Bugfix] Fix NVFP4 fused SiLU+mul scale allocation and global scale direction (#55643) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: jiahao <jxia77@terpmail.umd.edu>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-08 08:26:07 -07:00
Misha GoinClaude Opus 4.8ElizaWszola
 		b7624069ff [Fusion] Manual ActivationQuantFusionPass initial application (#51415) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
 2026-09-03 13:37:31 -04:00
maithilijoshi20 9debcd5990 [Test][Qwen3-VL] Cover compiled DeepStack input contract (#53529) 
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com>
 2026-08-31 10:46:50 +00:00
Kevin H. Luu b2dc864bb6 [Bugfix][Spec Decode] Keep default CUDA graph sizes memory-safe (#54418) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-08-30 18:02:31 +00:00
Rahul ChalamalaJanelle CaiDevin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>Devin
 		d3d79ffc1e [Bugfix][Spec Decode] Capture the widest uniform decode batch by default (#50488) 
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Rahul Chalamala <rahul@modal.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin <devin@cognition.ai>
 2026-08-28 22:38:13 -07:00
Khushali Desai 23ab0cfdbc speculative decoding under tensor parallelism (TP>1) , workspace creation select max hidden dim of target and draft model (#52193) 
Signed-off-by: khushali9 <khushali.desai9@gmail.com>
 2026-08-24 21:33:36 +00:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00