iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
35 Commits
Author SHA1 Message Date
Kunshang Ji 8ad54a991b [Platform] Add current_platform.num_compute_units interface (#35042) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-02-24 22:22:49 -08:00
Andrey Talman f97ca67176 [Release 2.10] Update to Torch 2.10 - final release (#30525) 2026-02-08 13:51:09 -08:00
Frank Wang 45f8fd6f97 [Feature] Enable TRITON_ATTN for Batch Invariance (#33688) 
Signed-off-by: frankwang28 <frank.wbb@hotmail.com>
 2026-02-04 13:27:34 +08:00
Frank WangandWentao Ye 8f5d51203b Disable Cascade Attention for Batch Invariance (#32561) 
Signed-off-by: frankwang28 <frank.wbb@hotmail.com>
Signed-off-by: Frank Wang <41319051+frankwang28@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-01-30 10:00:46 -05:00
Matthew Bonanni 2612ba9285 [1/N][Attention] Restructure attention: move files (#31916) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-09 13:10:24 -08:00
Wentao YeandCyrus Leung 6628758233 [Bug] Fix batch invariant in torch 2.10 (#30907) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-12-18 07:27:51 -08:00
Matthew Bonanniandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 60dbf7d8f1 Update batch invariant to use attention config (#30704) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-15 15:24:16 -05:00
Roberto L. Castroandyoukaichao 4fa7ce46f3 [Feature] Add SM103 (Blackwell Ultra) Support to vLLM (#30484) 
Signed-off-by: LopezCastroRoberto <robertol.c510@gmail.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-12-12 19:34:23 -08:00
quanliuandWentao Ye 5dcd593baf [Feature] Batch-Invariant Support for FA2 and LoRA (#30018) 
Signed-off-by: quanliu <18646313696@163.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-12-09 10:01:38 -05:00
Wentao Ye d9417096d1 [Feature] Batch invariant: Enable TRITON_MLA without prefix-caching (#29125) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-08 19:31:57 -05:00
Wentao Yeandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 0b0aa874e8 [Perf] Optimize batch invariant BMM, 18.1% Throughput improvement, 10.7% TTFT improvement (#29345) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-11-26 09:38:52 -07:00
Wentao Ye 699bca76c0 [UX] Raise error for attn backend of batch invariant (#29348) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-11-24 17:49:01 -07:00
Bram WastiandWentao Ye 5f7209a793 [tiny] Remove unsupported TRITON_MLA backend from batch invariance (#28832) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
Signed-off-by: Bram Wasti <bwasti@fb.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-11-22 21:00:50 +08:00
Wentao Ye e1eefa4c40 [Bug] Fix torch warning of tf32 usage (#29112) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-11-21 01:54:59 +00:00
Wentao Ye 2c52c7fd9a [Bug] Fix torch dynamo warning Dynamo detected a call to a functools.lru_cache (#29038) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-11-20 16:52:23 +08:00
Wentao Ye 1607e664f0 [Bug] Fix Batch Invariant MLA test (#28967) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-11-19 21:18:32 +00:00
Wentao Ye 3ddcf46011 [Refactor] Remove Unused Func in Batch Invariant (#28881) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-11-17 20:29:29 -08:00
Lukas Geiger ac0bb2c307 [Core] Cache vllm_is_batch_invariant (#28304) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-11-12 05:03:01 +00:00
Paul Zhang faedbb4d4f [Feature] Extend batch invariant torch.compile to B200 (#27856) 
Signed-off-by: PaulZhang12 <paulzhan@fb.com>
 2025-11-05 10:04:49 -08:00
Wentao Ye 7e4be74104 [Bug] Batch invariant: Fix flash attn MLA RuntimeError: scheduler_metadata must have shape (metadata_size) (#27884) 2025-11-04 14:05:55 +08:00
Paul ZhangandWentao Ye e7acb20076 [Feature] Batch invariant torch.compile (#27660) 
Signed-off-by: PaulZhang12 <paulzhan@fb.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-10-30 13:11:29 -07:00
Bram Wastiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> ded8ada86a Add more dims for batch invariant shims (#27489) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
Signed-off-by: Bram Wasti <bwasti@fb.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-30 05:28:45 +00:00
Wentao Ye 6afc28a9ba [Test] Batch Invariant: Unit test using parameterized backend (#27478) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-28 13:51:35 -07:00
Wentao Ye 86ed77022d [Feature] Batch Invariant for R1 TP 8 on Blackwell (#27229) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-21 10:25:55 -07:00
Jianyu Huang 221bf72577 output type conversion fix (#27159) 2025-10-19 08:10:07 +00:00
Bram Wasti b2f78cbad4 [small][batch invariance] Rename the env and internal flags to simplify usage (#26855) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
 2025-10-16 21:40:25 +00:00
Bram WastiandWentao Ye 7d8975de84 Deepseek-v3 Batch Invariant on 8xH100 (#26609) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-10-15 22:06:02 -07:00
Bram Wasti 3263799056 [unrevert] Add batch invariant kernel override for FlashInfer backend [2/n] (#26373) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
Signed-off-by: Bram Wasti <bwasti@fb.com>
 2025-10-13 10:24:53 -04:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Harry Mellor b893d661b1 Fix per file ruff ignores related to simplification (#26259) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 20:31:53 +00:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Cyrus Leung 1838cd4860 Revert "Add batch invariant kernel override for FlashInfer backend [2/n]" (#26220) 2025-10-04 02:45:08 -07:00
Bram WastiandWentao Ye 2f7dbc9b42 Add batch invariant kernel override for FlashInfer backend [2/n] (#25769) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
Signed-off-by: Bram Wasti <bwasti@fb.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-10-03 19:49:30 -07:00
Tyler Michael Smith f9df8b4ad7 [Bugfix] Fix triton import precommit failure (#25803) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-09-27 07:13:11 -07:00
Bram Wasti dc48ba0c75 Kernel-override Determinism [1/n] (#25603) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
 2025-09-26 16:59:09 -07:00