Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

36735fd772 [BugFix] Fix multiple/duplicate stdout prefixes (#36822) Nick Hill 2026-03-11 21:23:21 -07:00
6ecabe4936 [CI Failure] Fix Language Models Test (Extended Pooling) daily CI Failure (#36761) wang.yuqi 2026-03-12 12:22:05 +08:00
2f8b4ce0c0 [Model Runner V2] Do not initialize sampler for non-last PP ranks (#36824) Woosuk Kwon 2026-03-11 20:55:28 -07:00
2ef69456f5 [LMCache] Fault Tolerance Mechanism (#36586) Yuwei An 2026-03-11 20:54:39 -07:00
17852aa503 more models for vLLM Benchmark Suite (#35086) Louie Tsai 2026-03-11 20:36:51 -07:00
8647c6cf51 [Bugfix] Fix minimax_m2 tool parser when stream interval > 1 (#35895) Flora Feng 2026-03-11 22:25:14 -04:00
513949f95f [XPU][Doc] Remove manual OneAPI install step, now handled by torch-xpu (#36831) Kunshang Jiandcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> 2026-03-12 09:46:02 +08:00
262b76a09f [Frontend] Exclude anthropic billing header to avoid prefix cache miss (#36829) Nick Hillandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-03-11 18:20:34 -07:00
c34ba6b961 [Perf] Optimize compute maxsim using batched version, 3.2% E2E throughput improvement (#36710) Wentao Ye 2026-03-11 20:37:01 -04:00
24062b704f [ROCm][CI/Build] Add gfx1152/gfx1153 (Krackan) to HIP supported architectures (#36499) Matthias Gehre 2026-03-12 00:14:40 +01:00
d6b61e5166 [BUG] Fix async rlhf tests (#35811) Aaron Hao 2026-03-11 15:06:10 -07:00
cf632499ee [Kernel] [Helion] [15/N] Split config files into per-platform files (#36698) Yanan CaoandClaude Opus 4.6 2026-03-11 14:25:29 -07:00
a3774a8198 [Kernel] [Helion] [12/N] Use FakeTensorMode to avoid GPU allocation during config key computation (#36563) Yanan CaoandClaude Opus 4.6 2026-03-11 14:25:16 -07:00
0ce21c46a0 [Kernel] [Helion] [14/N] Set autotune_ignore_errors=True during autotuning (#36683) Yanan CaoandClaude Opus 4.6 2026-03-11 14:25:04 -07:00
55eed6b7a5 [Model Runner V2] Add WhisperModelState [6/N] (#35790) Woosuk Kwon 2026-03-11 14:20:38 -07:00
c77181e534 [Model Runner V2] Add probabilistic rejection sampling for spec decoding (#35461) Giancarlo Delfin 2026-03-11 14:04:32 -07:00
12001f2ebc [LMCache] Pass TP size in lookup for MLA multi-reader locking (#36129) maobaolongandYihua Cheng 2026-03-12 04:45:20 +08:00
7ee5d5093b [BugFix][kv_offload] Fix offloading decodes with async scheduling (#33881) Or OzeriandNicolò Lucchesi 2026-03-11 22:43:40 +02:00
428bc718bd [Bugfix][ROCm] Strip block_size before attention backend validation (#36274) jennyyyyzhenandLu Fang 2026-03-11 13:37:31 -07:00
ff1e3d9c63 [BugFix]: add bagel to MM_PREFIX_LM_MODELS (#36316) 汪志鹏 2026-03-12 03:55:59 +08:00
35bdca5431 [Refactor] Remove dead code in KV connector (#36424) Wentao Ye 2026-03-11 15:40:17 -04:00
8a24842765 [ROCm] add tuned moe_wna16_triton kernel configs for CDNA4 (#35093) Amanzhol Salykov 2026-03-11 20:00:08 +01:00
65986db6ba Make Gemma and Gemma 2 accept inputs_embeds like Gemma 3 (#36787) Harry Mellor 2026-03-11 18:12:43 +00:00
9556af87d5 [torch.compile] Add support for non-contiguous fused RMSNorm + group quant (#36551)
Luka Govedič Copilot ProExpertProg
2026-03-11 13:56:55 -04:00
a1a3523a56 [KVConnector] Support worker -> scheduler metadata (#31964) Or OzeriandNicolò Lucchesi 2026-03-11 19:36:37 +02:00
741f4e046b fix: align lfm2 thumbnail token counting with HF (#36707) tianshu-Michael-yu 2026-03-11 10:28:38 -07:00
a5d06dc557 Add 320 dimension size support to MLA (#36161) Julien Denize 2026-03-11 18:21:22 +01:00
5efa206a8c Fix ExaoneMoeMTP test that never ran in Transformers v4 (#36792) Harry Mellor 2026-03-11 17:10:23 +00:00
196802dfa6 [Misc] Clean up renderers (#36770) Cyrus Leung 2026-03-12 00:39:29 +08:00
c84b519cf3 [Bugfix] Fix negative max_tokens when input prompt is too long (#36789) Isotr0py 2026-03-12 00:30:51 +08:00
741ecf0630 [CI] Add bfcl tool call correctness eval (#36560) Flora FengandRobert Shaw 2026-03-11 12:27:36 -04:00
b7e5a588d8 [Bugfix] Fix DP/EP Shared Expert With Monolithic Kernels (#36061) Robert ShawandRobert Shaw 2026-03-11 12:07:14 -04:00
822e250ab7 [torch.compile] Use FakeTensors instead of real GPU tensors for single-size compilation (#36093) Richard Zou 2026-03-11 12:07:09 -04:00
bea02cdf93 Fix routed experts capture for hybrid models (Mamba + Attention) (#35744) Hongxin Xuandarlenxu 2026-03-11 23:53:10 +08:00
a3ea760ea5 Add 'none' reasoning effort to ChatCompletionRequest (#36238) Julien Denize 2026-03-11 16:45:34 +01:00
35db669f1d Correct link to supported hardware on vllm.ai (#36798) Harry Mellor 2026-03-11 15:43:28 +00:00
afebeffbfb Add support to Mistral large 3 eagle with dense layers (#36163) Julien Denizeandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-03-11 16:42:56 +01:00
5573894737 Kimi k2.5 MLA based eagle3 (#36361) Jhao-Ting ChenandIzzy Putterman 2026-03-11 08:36:11 -07:00
d5816c8c2f Fix tied weights in weight mapping test for Transformers v5 (#36788) Harry Mellor 2026-03-11 15:10:26 +00:00
8ccbcda5c0 [Model Runner V2] Remove unused warmup_for_prefill method (#36762) Woosuk Kwon 2026-03-11 08:02:44 -07:00
a9e532afe2 [ROCm][Perf] Allow MTP lens > 1 in Sparse MLA (#36681) tvirolai-amd 2026-03-11 16:43:03 +02:00
f3163bba67 Disable docs build skipping until a better solution is found (#36790) Harry Mellor 2026-03-11 13:53:23 +00:00
700a1ddc65 [Misc] Use envs module to get VLLM_DISABLED_KERNELS (#35776) Martin Hickey 2026-03-11 13:37:46 +00:00
f33251ffc8 [Bugfix] Fix Mistral-small --format (#36782) Silvia Colabrese 2026-03-11 12:47:52 +01:00
e584dce52b Add XPU MLA Sparse backend for DeepSeek v3.2 (#33230) Wuxun Zhang 2026-03-11 19:19:15 +08:00
40c0461f24 [openapi] refactor render related openapi [3/N] (#36749) Ning Xie 2026-03-11 18:14:34 +08:00
724759684c [Bugfix] Fix Qwen3-VL timestamp mismatch when using num_frames without fps (#36136) Weiguang LiandClaude Opus 4.6 2026-03-11 18:13:06 +08:00
9c34e9d24f Disable cascade attention by default (#36318) Michael Goin 2026-03-11 11:12:23 +01:00
09b6f99852 [compile] aot_compile should respect VLLM_DISABLE_COMPILE_CACHE (#36358) Richard Zou 2026-03-11 06:12:03 -04:00
c87fb515ed fix(lora): use replaced_module_name in pooling model name check (#36402)
Ethan T.Jee Jee Li Claude Opus 4.6
2026-03-11 18:11:27 +08:00
5353c9b016 platforms: Fix Ray DP startup crash (#36665) Itay Alroy 2026-03-11 12:08:55 +02:00
13e79fc811 [ci] Update rtol for test_classification (#36556) Angela YiandRichard Zou 2026-03-11 03:08:16 -07:00
9d07a3d6e4 Add: Eagle3 support for Qwen3.5 (#36658) Rahul Tuli 2026-03-11 15:37:42 +05:30
646b85544b [Refactor] Remove Molmo2 processor wrapper (#36667) Cyrus Leung 2026-03-11 18:07:20 +08:00
4286cc5ec2 fix(minicpmv): fix audio inference by handling meta device in init_re… (#36751) tc-mb 2026-03-11 18:06:28 +08:00
545d18d81b [Bugfix] Support other quantization methods in glm41v (#36321)
LoganJane g00887675/loganJane Isotr0py
2026-03-11 17:48:05 +08:00
e661b9ee83 [NemotronH] Small fix reasoning parser (#36635) roikoren755 2026-03-11 11:44:41 +02:00
c910eeb125 [XPU]Bug fix for some unexpected error when use AgRs backend on XPU device. (#36593) YiSheng5 2026-03-11 17:17:46 +08:00
f4ae58b38b Remove unused config field from Gemma2 (#36672) Harry Mellor 2026-03-11 08:51:19 +00:00
e568cf88bc [UX] Infer dtype for local checkpoint (#36218) Isotr0py 2026-03-11 16:50:04 +08:00
098d844731 [NIXL][1/N] Refactor kernel_block_size detection (#35752) Nicolò Lucchesi 2026-03-11 09:11:23 +01:00
a40ee486f2 [Bugfix] Add Multiple of 16 block_size to triton fallback on rocm Attention to support qwen3_5 (#35923)
JartX akaratza TJian
2026-03-11 08:45:57 +01:00
eac2dc2b41 AITER MLA backend: Avoid CPU sync in _build_decode (#35765) pschlan-amd 2026-03-11 08:25:00 +01:00
d5080aeaa4 [Refactor] Remove deadcode in Responses API serving (#36726) Flora FengandSigned-off-by: yewentao256 <zhyanwentao@126.com> 2026-03-11 03:11:41 -04:00
f22d6e0267 [Hardware][NIXL] set default kv buffer type for different platform (#36438) liuzhenweiandKunshang Ji 2026-03-11 13:19:28 +08:00
76c6e6da08 [XPU] Support block fp8 moe by fallback to TritonExpert on XPU (#36458) Kunshang Ji 2026-03-11 12:54:09 +08:00
4184653775 feat: add RISC-V support for CPU backend (v2) (#36578) typer-JandLi, Jiang <jiang1.li@intel.com> 2026-03-11 12:51:39 +08:00
4aaaf8c8ce feat(spec_decode): fuse EAGLE step slot mapping and metadata updates (#33503) Sladyn 2026-03-10 21:35:33 -07:00
4bf533623b [Doc] Fix duplicate words in comments (#36713) Hongbin Guo 2026-03-11 12:28:31 +08:00
5f77ef15ae [Misc][Attention] Clean up unused method in CPU_ATTN (#36673) Matthew Bonanni 2026-03-11 00:27:22 -04:00
7d6abdd022 [Fix] Use torch.empty for output in attention+quant fusion (#31785) elvischenv 2026-03-11 12:26:14 +08:00
a8ff2cca92 [Perf] Optimize scheduler overhead for PD disaggregation, around 5% E2E perf improvement (#35781) Wentao YeandOr Ozeri 2026-03-11 00:25:30 -04:00
42fadebecb [Model] Add support for moonshotai/Kimi-Audio-7B-Instruct (#36127) tunglinwood 2026-03-11 12:24:48 +08:00
a197eda9c3 Add tuned H100 MoE configs for LFM2 8B and 24B (#36699) tianshu-Michael-yu 2026-03-10 21:22:02 -07:00
82b110d50e [ci] Bound nvidia-cudnn-frontend version (#36719) Kevin H. Luu 2026-03-10 21:17:35 -07:00
9040cd40af [DSV3.2][MTP] Optimize Indexer MTP handling (#36723) Benjamin Chislett 2026-03-11 00:16:56 -04:00
fa0d353acf [Bugfix] Surface exceptions from non-blocking execute_model in UniProcExecutor to avoid DP deadlocks (#35194) fangyuchu 2026-03-11 11:22:21 +08:00
b386bb3d7c fix bugs when token_classify & classify run concurrently (#36614) Augusto Yao 2026-03-11 11:16:34 +08:00
fe714dd507 [openapi server] log exception in exception handler(2/N) (#36201) Ning Xie 2026-03-11 11:16:30 +08:00
8ab3d7427c [Bugfix] Fix DeepSeek V3.2 OOM during CG memory profiling (#36691) Matthew Bonanni 2026-03-10 23:01:07 -04:00
84e436ed1c [Bug] Fix TRTLLM Block FP8 MoE Monolithic (#36296) Wei ZhaoandRobert Shaw 2026-03-10 22:04:47 -04:00
81939e7733 [ROCm][CI] Making some tests optional to reduce workload (#36090) Andreas Karatzas 2026-03-10 18:45:27 -05:00
195d1ca3e8 [Minor] Enhance error message for TRTLLM decode uniformity check (#36609) Woosuk Kwon 2026-03-10 15:38:45 -07:00
8d983d7cd6 [Model Runner V2] Add initial CI tests (#36041) Nick Hill 2026-03-10 14:55:21 -07:00
65b2f405dc [Core] Simplify core kv-cache blocks initialization logic (#36521) Nick Hill 2026-03-10 13:20:02 -07:00
2a68464c5b [Test] test_async_scheduling.py improvements (#36340) Nick Hill 2026-03-10 11:17:26 -07:00
bdd8981dab [compile] Apply stored functorch config while finalizing loaded artifacts. (#36582) Zhengxu Chen 2026-03-10 12:34:35 -04:00
f088a831dd [Model Runner V2] Use unpadded num_tokens for PW CUDA graph attn metadata (#36626) Woosuk Kwon 2026-03-10 09:30:56 -07:00
f83b933b84 [CI] Bump mypy version to 1.19.1 (#36104) Harry Mellor 2026-03-10 16:18:28 +00:00
82f3f30e26 [ROCm][Perf] Enable sparse_mla's cudagraph on ROCm platform (#35719) Pleaplusone 2026-03-11 00:14:35 +08:00
9095cbbfb6 [Bugfix][Sparse MLA] report indexer CG support properly (#36519) Matthew Bonanni 2026-03-10 12:14:31 -04:00
721ae79f50 Improvements to wvSplitKrc skinny GEMM solution (#34304) Hashem Hashemi 2026-03-10 09:14:27 -07:00
aefc59f088 FunASR model bugfix (#36633) AllenDouandzixiao 2026-03-10 23:14:21 +08:00
d88f28da05 Fix hf_override_fn when it modifies model_type (#35200) Harry Mellor 2026-03-10 15:03:18 +00:00
106ff69c4e feat(kv-offload): Strategy A — StoreReusedOffloadingManager gates CPU stores on reuse frequency (#35342)
Srinivasoo7 srinivas_oo7 Srinivasoo7 Or Ozeri
2026-03-10 09:43:40 -05:00
ca5fb4bbd8 [Bugfix] Avoid merging empty-only partitions into splitting-op subgraphs (#36595) Jiangyun Zhu 2026-03-10 22:39:01 +08:00
cf88b23749 fix: check HTTP status in batch read_file to prevent silent failures (#36397)
Alvin Tang gambletan Claude Opus 4.6
2026-03-10 22:22:40 +08:00
a3189a08b0 [Model] Consolidate score logic by introduce score_type (#36479) wang.yuqi 2026-03-10 21:32:25 +08:00
409c4e632d [Misc] fix typo: homogenous-> homogeneous (2 lines change) (#36508) SoluMilken 2026-03-10 21:25:37 +08:00
8850738b70 [Bugfix] Fix processor signature (#36630) Raushan TurganbayandHarry Mellor 2026-03-10 14:20:47 +01:00

... 85 86 87 88 89 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86