 Mark McLoughlin
|
2942970d44
|
[Metrics] Hide deprecated metrics with gpu_ prefix (#24245)
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
|
2025-09-15 20:15:57 -06:00 |
|
|
Wentao Ye
|
b42566f440
|
[Bug] Fix is_flashmla_supported Check Error (#24774)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2025-09-15 20:10:55 -06:00 |
|
|
Alexander Matveev
|
aae725af7c
|
[Performance] Remove redundant clone() calls in cutlass_mla (#24891)
|
2025-09-15 20:21:53 +00:00 |
|
|
Andrew Xia
|
25aba2b6a3
|
[gpt-oss] Add IncompleteDetails to ResponsesRepsonse (#24561)
Signed-off-by: Andrew Xia <axia@meta.com>
|
2025-09-15 13:07:55 -07:00 |
|
|
Nicolò Lucchesi
|
2e41f5abca
|
[XPU] Set consistent default KV cache layout (#24745)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2025-09-15 18:09:34 +08:00 |
|
|
Chao Leiandleichao.lc
|
8de261b04a
|
[P/D]kv_output_aggregator support P TP > D TP (#23917)
Signed-off-by: LCAIZJ <leichao139636@163.com>
Co-authored-by: leichao.lc <leichao.lc@antgroup.com>
|
2025-09-15 11:36:06 +02:00 |
|
|
Ning Xie
|
59e17dd4a0
|
[Misc] rename interval to max_recent_requests (#24229)
Signed-off-by: Andy Xie <andy.xning@gmail.com>
|
2025-09-15 09:18:42 +00:00 |
|
|
Ning Xie
|
3f3313981c
|
[kv cache] update num_free_blocks in the end (#24228)
Signed-off-by: Andy Xie <andy.xning@gmail.com>
|
2025-09-15 05:15:12 +00:00 |
|
|
Chen Zhang
|
8e5cdcda4e
|
[Hybrid Allocator] Support Pipeline Parallel (#23974)
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
|
2025-09-14 15:55:17 -07:00 |
|
|
wuhangandRobert Shaw
|
90f3f7d73e
|
[Spec Decoding]Support Spec Decoding Metrics in DP Mode (#24049)
Signed-off-by: wuhang <wuhang6@huawei.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2025-09-14 21:11:09 +00:00 |
|
|
Woosuk Kwon
|
3e903b6cb4
|
[Chore] Minor simplification for non-PP path (#24810)
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
|
2025-09-13 17:41:36 -07:00 |
|
|
Russell Bryant
|
4dad72f0d9
|
[Misc] Correct an outdated comment. (#24765)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
|
2025-09-13 00:34:53 -07:00 |
|
|
Jee Jee Li
|
99bfef841f
|
[Bugfix] Fix GPUModelRunner has no attribute lora_manager (#24762)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2025-09-12 23:55:14 -07:00 |
|
|
|
4fdd6f5cbf
|
[Core] Support async scheduling with uniproc executor (#24219)
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2025-09-12 16:34:28 -07:00 |
|
|
Tao He
|
8226dd56bf
|
[Qwen3Next] Fixes the cuda graph capture conditions under large batch sizes (#24660) (#24667)
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
|
2025-09-12 22:31:32 +00:00 |
|
|
Matthew Bonanni
|
7ba32aa60b
|
[Attention][FlashInfer] Enable FP8 FlashInfer (TRTLLM) MLA decode (#24705)
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
|
2025-09-12 15:45:53 -06:00 |
|
|
Cyrus Leung
|
010acc6e1e
|
[Bugfix] Fix incompatibility between #20452 and #24548 (#24754)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-09-12 11:17:29 -07:00 |
|
|
Michael Goin
|
9d2a44606d
|
[UX] Remove AsyncLLM torch profiler disabled log (#24609)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2025-09-12 10:08:44 -07:00 |
|
|
dongluw
|
a5b84f1cbf
|
[Core] Shared memory based object store for Multimodal data caching and IPC (#20452)
Signed-off-by: donglu <donglu@cohere.com>
|
2025-09-12 07:54:17 -07:00 |
|
|
Didier Durand
|
bcb06d7baf
|
[Doc]: fix typos in various files (#24726)
Signed-off-by: Didier Durand <durand.didier@gmail.com>
|
2025-09-12 06:43:12 -07:00 |
|
|
Flora Feng
|
0377802c20
|
[Multimodal] Remove legacy multimodal fields in favor of MultiModalFeatureSpec (#24548)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2025-09-12 21:42:23 +08:00 |
|
|
Ignacio Sica
|
7a1c4025f1
|
[Kernel] [CPU] refactor cpu_attn.py:_run_sdpa_forward for better memory access (#24701)
Signed-off-by: ignaciosica <mignacio.sica@gmail.com>
|
2025-09-12 19:23:07 +08:00 |
|
|
|
40b6c9122b
|
[V1] feat:add engine v1 tracing (#20372)
Signed-off-by: Mu Huai <tianbowen.tbw@antgroup.com>
Signed-off-by: Ye Zhang <zhysishu@gmail.com>
Signed-off-by: RichardoMu <44485717+RichardoMrMu@users.noreply.github.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Mu Huai <tianbowen.tbw@antgroup.com>
Co-authored-by: Ye Zhang <zhysishu@gmail.com>
Co-authored-by: Benjamin Bartels <benjamin@bartels.dev>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: 瑜琮 <ly186375@antfin.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
|
2025-09-11 17:10:39 -07:00 |
|
|
Zhewen Li
|
7d4651997a
|
[CI/Build] Add bc-linter to vLLM CI (#21234)
Signed-off-by: zhewenli <zhewenli@meta.com>
|
2025-09-11 15:34:36 -07:00 |
|
|
Xiaozhu Meng
|
e42af78b18
|
[flashinfer] [kernel] support for fp8 kv cache for trtllm prefill attention (#24197)
Signed-off-by: Xiaozhu <mxz297@gmail.com>
|
2025-09-11 14:20:09 -07:00 |
|
|
Andrew Xia
|
79ac59f32e
|
Update Spec Decode metrics to include drafted and accepted token throughput (#24127)
Signed-off-by: Andrew Xia <axia@meta.com>
|
2025-09-11 19:58:43 +00:00 |
|
|
co63oc
|
e26fef8397
|
fix some typos (#24616)
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
|
2025-09-11 10:48:46 -07:00 |
|
|
Boyuan FengandHarry Mellor
|
94e6b2d55f
|
Allow users to specify kv cache memory size (#21489)
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-09-11 13:41:07 +00:00 |
|
|
Harry Mellor
|
5f5271f1ee
|
Move LoRAConfig from config/__init__.py to config/lora.py (#24644)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-09-11 11:01:38 +00:00 |
|
|
Tao HeandJee Jee Li
|
e93f4cc9e3
|
Add the support for the qwen3 next model (a hybrid attention model). (#24526)
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2025-09-11 15:32:09 +08:00 |
|
|
Didier Durand
|
e2b1f863aa
|
[Doc]: fixing doc typos (#24635)
Signed-off-by: Didier Durand <durand.didier@gmail.com>
|
2025-09-10 23:19:28 -07:00 |
|
|
Nick Hill
|
e2d8c27f68
|
[BugFix] Fix pipeline parallel (#24621)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-09-10 23:05:30 -07:00 |
|
|
Hanjie QiuandMichael Goin
|
dcb28a332b
|
[Kernel] Flashinfer MLA (trtllm-gen) decode kernel integration (#21078)
Signed-off-by: hjjq <hanjieq@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2025-09-10 15:31:10 -07:00 |
|
|
|
fba7856581
|
[Perf] Warmup FlashInfer attention during startup (#23439)
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>
|
2025-09-10 15:03:17 -07:00 |
|
|
Chen Zhang
|
b5e383cd8b
|
[gpt-oss] raise error for flashinfer backend without trtllm (#24482)
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
|
2025-09-10 14:33:13 -07:00 |
|
|
|
9a161307f5
|
[torch.compile][ROCm][V1] Enable attention output FP8 fusion for V1 attention backends (#19767)
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
|
2025-09-10 13:59:55 -07:00 |
|
|
Russell BryantandNickLucche
|
37e8182bfe
|
[v1] Add Whisper model support (encoder-decoder) (#21088)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
|
2025-09-10 13:53:35 -07:00 |
|
|
Thien Tran
|
a0933c3bd6
|
[Bugfix] Enable FP8 KV cache for FlashInfer and Triton backend on non-sm100 GPUs (#24577)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2025-09-10 12:33:41 -07:00 |
|
|
co63ocandRussell Bryant
|
3144d90217
|
fix some typos (#24167)
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
|
2025-09-10 06:21:23 -07:00 |
|
|
Nick Hillandguoze.lin
|
f4f1a8df22
|
[BugFix] Ensure integrity of reused CPU tensors during async scheduling (#24527)
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: guoze.lin <guozelin@tencent.com>
|
2025-09-10 21:15:14 +08:00 |
|
|
Lucas WilkinsonandMatthew Bonanni
|
0ae43dbf8c
|
[Attention] add DCP support for FLASH_ATTN_MLA backend (#24453)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2025-09-10 17:19:26 +08:00 |
|
|
RemyandLi, Jiang <jiang1.li@intel.com>
|
feaf202e93
|
[Bugfix] Guard _may_reorder_batch for encoder-only models on CPU (#24319) (#24348)
Signed-off-by: Remy <eunhwan.shin@dtonic.io>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2025-09-10 14:24:42 +08:00 |
|
|
Chenheli Hua
|
009d689b0c
|
[Core] Simplify and unify mm uuid handling & auto-generated mm hash overrides processing. (#24271)
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
|
2025-09-09 21:36:09 -07:00 |
|
|
Yihua Cheng
|
b4a01aaf95
|
[KV Connector] More async support for get_num_new_matched_tokens (#23620)
Signed-off-by: ApostaC <yihua98@uchicago.edu>
|
2025-09-09 21:23:37 -07:00 |
|
|
Nick Hill
|
f88e84016f
|
[BugFix] Fix async core engine client finalizer (#24540)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-09-09 21:07:13 -07:00 |
|
|
Yong Hoon Shin
|
dc625ea6b8
|
[Perf] Convert np array to torch tensor to index into block table for attn chunking (#24474)
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
|
2025-09-09 20:01:06 -07:00 |
|
|
Wentao Yeandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
15de5ff9ea
|
[Feature] Disallow FlashMLA on Blackwell (#24521)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-09-09 14:59:34 -04:00 |
|
|
Chenyaaang
|
c3f9773b2c
|
[TPU] Fix tpu structured decoding in mixed batches (#24458)
Signed-off-by: Chenyaaang <chenyangli@google.com>
|
2025-09-09 11:04:25 -07:00 |
|
|
Micah WilliamsonandGregory Shtrasberg
|
1c63a16b65
|
[Core] Run garbage collector after CUDA graph capture to fix throughput regression (#24128)
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
|
2025-09-09 10:38:10 -04:00 |
|
|
d.transposed
|
922d3b401b
|
[Bugfix] Handle the edge case in detokenizer where processed tokens contain both stop str and eos token (#23938)
Signed-off-by: dtransposed <damian.bogunowicz@gmail.com>
|
2025-09-09 07:30:24 -07:00 |
|