iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,895 Commits 3 Branches 0 Tags
e0317b28ce3ba4c9bd2ff69d885e3f34688a6d71
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Matthew BonanniandClaude Opus 5.5 d5f356dae0 [HiSparse] Own GPU residency instead of editing core block-table rows (#60647) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 21:25:44 -07:00
Matthew BonanniandClaude Opus 5.5 4c43254d52 [HiSparse] Report host-pool block residency as HiSparse metrics (#59485) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 16:45:34 -06:00
Matthew BonanniandClaude Opus 5.5 1afb029839 [Bugfix][HiSparse] Route by swap-row capacity, not the index-group workspace (#60682) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 19:45:44 +00:00
Matthew BonanniandClaude Opus 5.5 4ed4c4433b [Bugfix][HiSparse] Drop the unaccounted, write-only prefill mirror staging buffer (#60429) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 14:00:49 +00:00
Matthew BonanniandClaude Opus 5.5 e82b80099a [Perf][HiSparse] Cache per-request residency instead of rescanning every page each step (#60083) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 13:04:55 -04:00
Matthew BonanniandClaude Opus 5.5 8542c3eff5 [CI] Fix test_mixed_warmup_gate after #57053 (#60116) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 21:17:00 +00:00
Matthew BonanniandClaude Opus 5.5 54d93af9fb [Bugfix][HiSparse] Reject cudagraph_mode=FULL at startup (#59688) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 12:42:27 -04:00
Matthew BonanniandClaude Opus 5.5 9e931a0c6c [CI] Add CODEOWNERS for HiSparse (#60060) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 15:18:29 +00:00
Matthew BonanniClaude Opus 5.5Lucas Wilkinson
 		f7999d2e44 [Bugfix][HiSparse] Size the KV cache from the groups HiSparse allocates (#59450) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-02 04:11:55 +00:00
Matthew BonanniandClaude Opus 5.5 3a69636645 [Bugfix][HiSparse] Fix a chunked-prefill preemption livelock (#59494) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 13:02:31 -04:00
Matthew BonanniClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		3eb6cec22a [Bugfix][HiSparse] Adopt GPU prefix copies after the hit's allocation (#59282) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-30 15:43:09 -04:00
Matthew BonanniandClaude Opus 5.5 a21a3f2ec7 [CI] Add supports_multimodal_inputs to test_executor_replace's mock config (#59476) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 19:04:47 +00:00
Matthew BonanniandClaude Opus 5.5 2798f66860 [Bugfix][HiSparse] Resolve MTP verification rows with a union residency kernel (#59235) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 03:01:45 +00:00
Matthew BonanniCodexLucas WilkinsonLucas WilkinsonOpenAI Codex
 		82a85dc1d2 [Attention] Add Triton/FlashInfer composite for multimodal prefix attention (#56305) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-13 14:29:41 +00:00
Matthew BonanniOpenAI CodexLucas WilkinsonLucas WilkinsonNicolò LucchesiOpenAI CodexClaude Opus 5Lucas Wilkinson
 		d43bb2f37f [3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-09-12 00:36:47 -04:00
Matthew BonanniOpenAI CodexOpenAI Codex
 		e0aaef85f3 [2/N][KV Connector][NIXL] Support per-region transfer geometry (#53780) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-08 16:01:12 -04:00
Matthew BonanniandOpenAI Codex de9250ac9e [Attention] Enable masked MHA for GLM-5 head dimensions (#53785) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-27 21:17:12 +00:00
Matthew Bonanni d85708f7a4 [1/N][KV Connector] Identify externally transferable KV cache groups (#53779) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-08-27 11:21:24 -04:00
Matthew BonanniandOpenAI Codex 75dea9b4ae [Bugfix] Update FlashMLA for sparse decode workspace fix (#53755) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-27 13:37:29 +00:00
Matthew Bonanniandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 8e46accab2 [Attention] Vectorize sparse MLA mask loads (#52217) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-18 20:32:41 -07:00
Matthew BonanniClaude Opus 5OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b38e111d3e [Attention][MLA] Per-request scheduling for MLA chunked context (#50613) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 16:45:44 +00:00
Matthew Bonanni a0cd2b69b3 [Bugfix] Universally align block table width to 128 tokens (#50302) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-31 08:52:17 -07:00
Matthew BonanniandOpenAI Codex 82ae4164ee [2/N][Attention] Enable masked MHA for sparse MLA prefills (#48770) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-31 06:01:24 -07:00
Matthew Bonanniandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> bf2b45b5d6 [Attention] Integrate FlashAttention 4 SM100 headdim 256 support (#42669) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-07-27 18:25:50 +00:00
Matthew BonannihaosdentClaude Opus 4.7Justin Wood
 		58b2012aa2 [copy of #45208] CuMem slept-L1 fragmentation accounting (#49208) 
Signed-off-by: haosdent <haosdent@gmail.com>
Signed-off-by: Justin Wood <justin.m.wood@me.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: haosdent <haosdent@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Justin Wood <jwood@me.com>
 2026-07-20 23:08:11 +00:00
Matthew Bonanniandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> bd091079cb [Attention] FlashAttention 4 SM100 FP8 kv cache support (#42569) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-20 10:53:27 -07:00
Matthew BonanniOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		67f9046e4a [Bugfix] Sparse MLA: enable fp8_ds_mla dense prefill (#48642) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-16 22:44:04 +00:00
Matthew Bonanni 313d01f507 [CI][Bugfix] Fix FlashAttention reported MLA dimension support (#48631) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-14 21:33:02 +00:00
Matthew BonanniandClaude Opus 4.8 c4f5cd60da [1/N] Add dense MHA path for sparse MLA short sequences (#47327) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-14 00:29:56 +00:00
Matthew BonanniandClaude 735def4fcf [Bugfix] Fix FlashMLA dense fp8 metadata crash (num_sm_parts clamp) (#48045) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-10 12:24:52 -07:00
Matthew BonanniandClaude Opus 4.8 3dd910da42 [Bugfix] Allow non-contiguous query in FlashInfer FP8 query quantization (#47908) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-07 20:11:34 +00:00
Matthew Bonanni 8fc1b2d046 Fix FA4 dynamic_causal for full attention layers (#46659) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-06-29 14:23:34 -07:00
Matthew BonanniandClaude 040df8f2ea [CI] Fix attention benchmark smoke test (#45728) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-16 13:43:37 +00:00
Matthew Bonanni cf567cbc71 [Attention] Improve attention benchmarks: configs and profiling (#39336) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-06-12 16:24:25 -04:00
Matthew BonanniandClaude 557781131a [Misc] Remove stray empty file (#44350) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-02 12:53:03 -07:00
Matthew Bonanni ea0d045a05 [FlashAttention] Sync FA with upstream (#44065) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-06-02 07:15:37 -07:00
Matthew Bonanni aa6138169f [MLA][Attention] Add OOT MLA prefill backend registration mechanism (#43325) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-05-26 19:56:09 -07:00
Matthew Bonanni 9898f94abe [Attention] Remove deprecated MLA prefill arguments (#42555) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-05-14 10:34:06 -07:00
Matthew Bonanni dcacdf9a88 [Attention] Sync FA with upstream (#41052) 2026-05-12 23:34:18 -04:00
Matthew Bonanni 01b9b5af67 [Attention] Minor refactor: layer takes ownership of the MLA prefill backend (#41744) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-05-05 23:22:41 +00:00
Matthew Bonanni be5983b874 [Docs] Add non-causal support to attention backend docs (#41643) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-05-04 20:35:15 +00:00
Matthew BonanniMichael GoinLucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f3fef12350 [Attention] Abstract the MLA prefill backends and eliminate cuDNN (#32623) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-01 13:36:20 -04:00
Matthew Bonanni f03d82efdd [UX][Bugfix] Fix OOM by setting PyTorch max_split_size_mb during model loading (#41268) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-30 07:46:54 -07:00
Matthew Bonanni fd74c90d9c [Attention][Spec Decode] Allow independent drafter attention backend selection (#39930) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-27 19:38:09 -07:00
Matthew Bonanni cde8d24710 [Spec Decode] Move SpecDecodeBaseProposer out of eagle.py (#40732) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-23 22:28:27 +00:00
Matthew BonanniandTyler Michael Smith 96a85c5750 [Startup][UX] Enable CUDAGraph memory profiling by default (#38284) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-04-21 18:16:59 -04:00
Matthew Bonanni ab5666eb7c [UX] Bump version in CG memory profiling log message (#40465) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-21 15:26:06 +00:00
Matthew Bonanniandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> c075702eae [Misc][UX] Suppress confusing num_gpu_blocks log lines (#40402) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-20 22:32:46 +00:00
Matthew Bonanni 219bb5b8c0 [Misc] Update committers.md (#40058) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-16 13:48:41 -07:00
Matthew Bonanni c77e596e2e [FlashAttention] Don't overwrite flash_attn_interface.py when installing precompiled (#39932) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-15 16:43:15 -04:00
Matthew Bonanni 308cec5864 [FlashAttention] Symlink FA4 instead of copying when using VLLM_FLASH_ATTN_SRC_DIR (#38814) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-08 12:04:34 +00:00
Matthew Bonanni 9c81f35b1a [Attention][MLA] Re-enable FA4 as default MLA prefill backend (#38819) 2026-04-06 17:51:46 -04:00
Matthew Bonanni 05e68e1f81 [CI] Fix test_nixl_connector (#38838) 2026-04-02 17:52:13 -07:00
Matthew Bonanni 116f4be405 [1/N][Cleanup] Standardize on use of is_quantized_kv_cache (#38659) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-01 04:08:01 +00:00
Matthew BonanniandSandishKumarHN 757068dc65 [Bugfix][Async] Fix async spec decoding with hybrid models (#38556) 
Signed-off-by: SandishKumarHN <sandishkumarhn@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: SandishKumarHN <sandishkumarhn@gmail.com>
 2026-03-31 11:08:54 -04:00
Matthew Bonanni 7d65463528 [WIP][CI][Bugfix] Fix test_run_eagle_dp (#38584) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-31 12:30:25 +02:00
Matthew Bonanni 2c734ed0e0 [Bugfix][MLA] Change default SM100 MLA prefill backend back to TRT-LLM (#38562) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-30 09:51:24 -07:00
Matthew BonannizhuhaoranzhrrrLucas WilkinsonBenjamin Chislett
 		fafe76b4af [Async][Spec Decoding] Zero-bubble async scheduling + spec decoding (#32951) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: zhrrr <43847754+izhuhaoran@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2026-03-23 15:37:22 -04:00
Matthew Bonanni 93f3c8e531 [Misc] Add float16 to CacheDType (#37199) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-16 13:24:48 -07:00
Matthew Bonanni c88ea8338b [MTP][Sparse MLA] Take advantage of native MTP support in indexer when possible (#36982) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-16 13:51:21 -04:00
Matthew Bonanni 9efc4db965 [Bugfix] Fix DeepSeek-V3.2 tokenizer stripping spaces (#37004) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-13 22:55:36 +00:00
Matthew Bonanni f444c05c32 [Attention] Use FA4 for MLA prefill (#34732) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-12 12:10:17 -04:00
Matthew Bonanni 5f77ef15ae [Misc][Attention] Clean up unused method in CPU_ATTN (#36673) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-10 21:27:22 -07:00
Matthew Bonanni 8ab3d7427c [Bugfix] Fix DeepSeek V3.2 OOM during CG memory profiling (#36691) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-11 03:01:07 +00:00
Matthew Bonanni 9095cbbfb6 [Bugfix][Sparse MLA] report indexer CG support properly (#36519) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-10 09:14:31 -07:00
Matthew Bonanni 4e571ce643 [MTP][Misc] Clean up dead code (#36507) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-09 14:43:06 -04:00
Matthew Bonanni 00c4cb5606 [Bugfix] Clear stale CG keys after memory profiling (#36416) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-09 11:56:00 -04:00
Matthew Bonanni 77a73458e3 Reapply [Attention] Refactor check_and_update_config (#35122) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-09 07:17:14 -07:00
Matthew Bonanni ebb9cc5f2b [UX][Startup] Account for CUDA graphs during memory profiling (#30515) 2026-03-07 13:49:23 -08:00
Matthew Bonanni 8e1fd5baf0 [CI] Bump num_speculative_tokens to 3 in nightly DeepSeek tests (#35882) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-03 09:26:44 -08:00
Matthew Bonanni 2562e0271e [MTP] Validate that MTP weights are actually loaded (#35548) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-02-28 12:27:40 +08:00
Matthew Bonanni a9e15e040d Add @MatthewBonanni to CODEOWNERS (#35207) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-02-24 10:45:10 -07:00
Matthew BonanniandRobert Shaw 662205d34e [Bugfix] Fix Basic Models Test (#34818) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-19 14:49:07 -08:00
Matthew Bonanni 7743152957 [Attention] Refactor check_and_update_config (#33600) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-02-17 17:06:54 -08:00
Matthew BonanniandJee Jee Li dc5fa77a4e [Bugfix][MTP][Sparse MLA] Allow sparse MLA with MTP to run with FULL cudagraphs (#34457) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-02-17 14:01:27 -05:00
Matthew BonanniLucas WilkinsonLucas Wilkinson
 		f2c47886fd [Attention] Add FlashInfer Sparse MLA backend (#33451) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-02-12 17:21:54 +00:00
Matthew Bonanni 4293c00b84 [Benchmarks] Fix attention benchmark smoke test (#34269) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-02-10 16:04:07 -05:00
Matthew Bonanni 4145e50d85 [Bugfix] Fix DSV3.2 NVFP4 (#33932) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-02-05 19:22:19 +00:00
Matthew Bonanni bd8da29a66 [Bugfix] Fix sparse MLA metadata building (#33579) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-02-03 15:29:48 -08:00
Matthew Bonanni 5d1aef3004 [UX] Format attention backend log line (#33570) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-02-02 18:57:12 +00:00
Matthew Bonanni 9f8cb81b44 [CI] Add DeepSeek V3.2 nightly eval (#33566) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-02-02 16:10:02 +00:00
Matthew Bonanni aaa901ad55 [Attention] Move MLA forward from backend to layer (#33284) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-30 19:30:00 -08:00
Matthew Bonanni 77c4f45c6c [7/N][Attention][Docs] Add documentation for attention backends (#32477) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-28 17:20:22 -05:00
Matthew BonanniandClaude e82fa448c4 Add attention benchmarking tools (#26835) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-01-28 00:09:20 +00:00
Matthew Bonanni 1cbccb6dba [Attention] Use has_flashinfer helper (#33177) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-27 18:33:17 +00:00
Matthew Bonanni a608b4c6c2 [5/N][Attention] Finish eliminating vllm/attention folder (#32064) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-27 10:02:51 -05:00
Matthew Bonanni 2d6b537157 [Bugfix][CI] Fix pre-commit (#32956) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-23 10:26:56 -08:00
Matthew Bonanni 955b43a5a5 [Bugfix][Attention] Explicitly report support for kv_cache_dtype bfloat16 (#32795) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-22 19:05:18 +00:00
Matthew Bonanni 300622e609 [CI][Attention] Add more CI dependencies for attention tests (#32487) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-22 18:44:56 +00:00
Matthew BonanniandWentao Ye 1a1fc3bbc0 [Attention][MLA] Make FLASHINFER_MLA the default MLA backend on Blackwell, and TRTLLM the default prefill (#32615) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-01-19 18:41:34 -05:00
Matthew Bonanni 2e7c89e708 Revert "[Attention][MLA] Make FLASHINFER_MLA the default MLA backen… (#32484) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-17 04:42:39 +00:00
Matthew Bonanni bcf2333cd6 [CI] Fix LM Eval Large Models (H100) (#32423) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-16 00:52:49 +00:00
Matthew BonanniandWentao Ye 8ebfacaa75 [Attention][MLA] Make FLASHINFER_MLA the default MLA backend on Blackwell, and TRTLLM the default prefill (#32339) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-01-15 09:49:57 -05:00
Matthew BonanniandLuka Govedič 2263d44b68 [4/N][Attention] Move MLA common to model_executor (#32060) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-01-13 09:08:45 -08:00
Matthew Bonanni 98f60e5acb [6/N][Attention] Move utils to more appropriate locations (#32215) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-13 05:38:52 -08:00
Matthew Bonanni 20228cb851 [3/N][Attention] Move AttentionMetadata-related code from utils.py to backend.py (#32054) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-12 09:13:56 -08:00
Matthew Bonanni 0308901975 [2/N][Attention] Fix pre-commit errors (#32052) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-10 00:27:15 +00:00
Matthew Bonanni 2612ba9285 [1/N][Attention] Restructure attention: move files (#31916) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-09 13:10:24 -08:00
Matthew Bonanni 0d7667419f [0/N][Attention] Fix miscellaneous pre-commit issues (#31924) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-08 01:15:17 +00:00
Matthew BonanniBenjamin ChislettBenjamin Chislett
 		d49899732e [Spec Decode][UX] Add acceptance stats to vllm bench serve report (#31739) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-01-06 21:21:42 +00:00