Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

4a4fdabe28 [Misc] Aligning tokwise pooler heads for consistency (#43041) Taneem Ibrahim 2026-05-19 01:16:42 -05:00
f1e3f0e6d6 [XPU] Use custom op collective behavior (#41354)
Chaojun Zhangmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Kunshang Ji
2026-05-19 14:14:59 +08:00
9fd8487d2f [Docs] Add SVG images for pooling models. (#42626)
Gracie Guo (UX)Gracie Guo wang.yuqi
2026-05-19 13:50:38 +08:00
27f4ba9481 fix: use keyword arguments for shard_id and expert_id in weight_loade… (#42671) Junyan Xuandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-18 22:29:04 -07:00
6e889b582b [ci] Route 28 gpu_1_queue tests to h200_35gb queue (#43030) Kevin H. LuuandClaude Opus 4.6 2026-05-18 21:58:36 -07:00
fab07e4d0f [Bugfix][KV Connector] Fix SimpleCPUOffloadScheduler TOCTOU between Phase A and Phase B (#42289)
Qiuyang Yue Claude Opus 4.7 gemini-code-assist
2026-05-18 21:22:33 -07:00
3ca8db2ef8 add cutedsl dsv4 indexer fp8 kernel (#42899) gnovackandgeorge 2026-05-18 21:17:56 -07:00
87b08c5f64 [Model Refactoring] Move DeepSeek V4 layers to models/deepseek_v4/ [2/N] (#43039) Woosuk Kwon 2026-05-18 21:00:58 -07:00
fba010dd74 [Bugfix][MRV2] Fix KVCache tensor explicit kernel_block_size dim (#42766) Nicolò LucchesiandNick Hill 2026-05-19 05:25:41 +02:00
da03e549b3 [UX] Add a persistent cache for FlashInfer autotuning (#42537) Mohammad Miadh Angkad 2026-05-19 11:25:37 +08:00
36dcaf25d8 [XPU] add gptq(int4) support (#37844) Kunshang Ji 2026-05-19 03:17:09 +00:00
8f16c4a5c0 [BugFix][CPU][Spec Decode] Fix Eagle implementation on CPU backend (#42468) Ofir Zafrir 2026-05-19 06:16:07 +03:00
afd7b1dce9 [Bugfix] Use platform-agnostic device in example_connector load (#42926) Revital Surandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-19 06:12:04 +03:00
287471b994 [Model Refactoring] Migrate DeepSeek V4 to vllm/models/ [1/N] (#43004) Woosuk Kwon 2026-05-18 19:50:02 -07:00
239b5ff30c [Frontend] Add --spec-method/--spec-model/--spec-tokens CLI aliases (#42476) Michael GoinandClaude 2026-05-18 20:22:27 -04:00
f85c76d701 [CI/Build] Bump nvidia-cutlass-dsl to 4.5.1 (#42991) Artem Perevedentsev 2026-05-19 02:58:15 +03:00
a171e6b52d Add parallel drafting to v2 model runner unsupported features (#43010) shanjiaz 2026-05-18 19:39:09 -04:00
37ece593c1 [Perf] Padded nvfp4 quant kernel to remove additional copy, 2.4%~5.7% e2e performance improvement (#42774) Wentao Ye 2026-05-18 19:38:12 -04:00
57fef4e0bf [Refactor] Extract shared coerce_to_schema_type utility from Minimax M2 tool parser (#43006) Flora Feng 2026-05-18 17:55:39 -04:00
0191354827 [Perf][MLA] Enable FULL cudagraph capture for TRITON_MLA decode (#42885) haosdent 2026-05-19 05:29:10 +08:00
cd49a05d5a [Refactor] Remove dead code (#42889) Wentao Ye 2026-05-18 16:41:22 -04:00
84747489de Tier offload followup (#42529) Ronen Schaffer 2026-05-18 22:41:58 +03:00
8fc1c284b9 [ROCm] Guard AITER GDN decode fast path by layout (#42880) Tuukka Sarvi 2026-05-18 21:56:22 +03:00
ce88f01c9a [Docs] update attribution to reflect EDEN foundation (#41666) Amit Portnoy 2026-05-18 21:22:56 +03:00
00e20e76f7 [Refactor] Remove dead cuda kernels (#42767) Wentao Ye 2026-05-18 14:14:21 -04:00
9758a6e5c5 [BugFix] support PP for Cohere vision model (#42819) czhu-cohere 2026-05-18 11:12:06 -07:00
a2c8fc6657 [ROCm][Quantization][3/N] Refactor quark_moe w4a4 w/ oracle (#41436) Bowen Bao 2026-05-18 10:46:13 -07:00
6859ca7615 [Bugfix] fix swiglu limit issue for humming backend + deepseek v4 (#42541) Jinzhen LinandMichael Goin 2026-05-19 01:32:26 +08:00
67f58ce23f [Bugfix] Fix DSV4 MTP after ROCm mHC integration (#42930) Mohammad Miadh Angkad 2026-05-19 01:02:01 +08:00
8c296de63b [Perf] Re-enable flashinfer autotune by default and cleanup (#42857) Wei Zhao 2026-05-18 12:12:27 -04:00
b12745e4f3 Fix --convert passed without --runner on causal models (#42935) Harry Mellor 2026-05-19 00:56:09 +09:00
e26736973a [Model Runner V2] Fix prompt logprobs calculation Sizes of tensors must match error (#42778) Wentao Ye 2026-05-18 11:27:21 -04:00
47829b1159 [Bugfix] mamba: run single-token extends as decodes (#42430) Netanel Haber 2026-05-18 18:26:00 +03:00
4a39b4f553 [Model] Add Apertus Tool Parser (#41154) Blanc Swan 2026-05-18 17:20:04 +02:00
78e7a7b9b0 Refactor AWQ Marlin MoE onto modular WNA16 oracle (#42483)
Siddharth Bedekar Robert Shaw OpenAI Codexgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-18 08:02:43 -07:00
f5d3dc7115 [Model Runner v2] Support update_config (#42783) Michael GoinandClaude Opus 4.7 2026-05-18 10:26:07 -04:00
1ac10f159a Revert "[torch.compile] Add patch for fullgraph compilation" (#42686) (#42913)
vllm-agent Luka Govedič Zhewen Li
2026-05-18 06:02:51 -07:00
e5417657e5 [KV Connector][Offloading] Flush all pending jobs on last step (#42611)
liranschour Or Ozerimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-18 15:59:42 +03:00
2e40faf08b [XPU][CI] Temporarily skip test_moe_lora_align_block_size_mixed_base_and_lora[1] in Intel GPU CI (#42954) xiangdong 2026-05-18 20:34:48 +08:00
69c91d010a [MRv2] Default to MRv1 when a connector is present (#42955) Nicolò Lucchesi 2026-05-18 14:34:16 +02:00
737bfa3a43 [Bugfix][Hybrid][NemotronH] Fix mamba_cache_mode=all + speculative decoding crash (#41233) roikoren755 2026-05-18 14:54:00 +03:00
e414e1f1c0 [Bugfix][KV Offload] count appended GPU blocks in store group_sizes (#42945) Kfir Toledo 2026-05-18 14:36:02 +03:00
df852ed503 fix: remove unused norm for dpskv4 (#41710) inisisandYongye Zhu 2026-05-18 18:33:29 +08:00
88a860d754 [CPU] Add MXFP4 W4A16 MoE support (#41922) Yuwen Zhou 2026-05-18 18:04:45 +08:00
cac81b6eda [CPU Backend] Improve cpu thread utilization (#42666) Tianmu Liandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-18 03:04:41 -07:00
b4601ad43f [CPU] Add fused GDN support for AMX CPU platform (#42707) Li, Jiang 2026-05-18 18:04:36 +08:00
2267f70070 [Kernel] Pack topk id/weights triton kernel (#42527) Jee Jee Li 2026-05-18 18:04:31 +08:00
965d076148 [CPU] Specify required KV cache layout for CPU attention backend (#42740) Tony LinandLi, Jiang <jiang1.li@intel.com> 2026-05-18 17:38:54 +08:00
c38bed4248 delete xpu ci (#42582)
wenjun liu Kunshang Jimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-18 16:36:45 +08:00
998714b21b [Perf] Add do_not_specialize in fused FP8 RoPE kernel (#42849) Xin Yang 2026-05-18 01:32:46 -07:00
9537542537 Revert checkpoint specific workaround in Transformers modelling backend (#42923) Harry Mellor 2026-05-18 17:31:06 +09:00
5ab6d1b3fd [Model] [Perf] Use flatten for Qwen3.5's GDN output projection (#42311) Rishapveer Singh 2026-05-18 10:14:36 +02:00
7d5b033782 [LoRA] Support 2D and 3D MoE LoRA adapter at the same time (#42242)
Jee Jee Ligemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Roger Wang
2026-05-18 15:22:26 +08:00
e3aeee5ff8 [Bugfix] moe lora align kernel grid (#40131)
Nguyễn Thế Duy Jee Jee Li Jee Jee Li
2026-05-18 14:17:53 +07:00
c1f7854342 Improve logging when docs build is skipped (#42929) Harry Mellor 2026-05-18 15:33:32 +09:00
23c15acd77 [BugFix] Kimi-K2.5: skip vision tower dtype conversion when using quantization (#42869) gaozihao-shy 2026-05-18 13:07:16 +08:00
b50646e5ef [ROCm][CI] Stabilize ROCm pooling and multimodal CI (#42909) Andreas Karatzas 2026-05-17 22:57:59 -05:00
990f49bdcb [MM][CG] Enable encoder Cudagraph for Step3VL (#42224) Soyaazz 2026-05-18 11:19:13 +08:00
107210442d [CI] Add NIXL EP import canary (#42567) AlecandOpenAI Codex 2026-05-17 19:11:46 -07:00
03ddc1c9bc [Perf] Wire silu_and_mul_per_block_quant into TritonFP8MoE (MiniMax-M2) (#42497) Yiliu Dongandqianlihuang 2026-05-18 09:57:04 +08:00
966903eb93 [torch.compile] Add patch for fullgraph compilation (#42686) Luka Govedič 2026-05-17 15:49:16 -04:00
599e75f432 [ROCm] [Bugfix] Fix DeepSeek V4 Functionality and Accuracy (#42810) TJian 2026-05-18 00:18:50 +08:00
1c8e9c0399 Refactor: Pass num_labels explicitly to PoolerClassify instead of reading from global config (#42851) Taneem Ibrahim 2026-05-17 09:40:21 -05:00
0fa888465e [XPU] fix weight scale shape (#42725) zofiaandKunshang Ji 2026-05-17 16:55:10 +08:00
ff712f6447 [MRV2][XPU] add Model Runner V2 log (#42710) liuzhenwei 2026-05-17 12:15:50 +08:00
504a26ce2b Support bf16 for mamba ssm cache (#41680) Qi Zhou 2026-05-16 17:54:58 -07:00
a94189295b Fix Weight loading for Qwen3.5-MTP and Qwen3-VL using runai_streamer (#42716) weizhoublue 2026-05-17 08:54:27 +08:00
0867497368 [CI/Build] Bump flashinfer to v0.6.11.post2 (#41711) Artem PerevedentsevandVadim Gimpelson 2026-05-17 00:55:12 +03:00
36e74c9ea4 [KV Connector] Support disk offloading in MooncakeStoreConnector (#42689)
Zhewen Li Zhewen Li Claude Opus 4.7
2026-05-16 13:34:15 -07:00
787bc0d031 Add unit tests for pooler activation functions (#42824) Taneem Ibrahim 2026-05-16 14:58:16 -04:00
d1586e1a12 Fix: Propagate pinned model revisions into Ultravox secondary weight loading (#42830) weizhoublue 2026-05-17 01:02:54 +08:00
8a56da3845 [Experimental] Breakable CUDA graph (#42304) Jiangyun Zhu 2026-05-16 22:04:12 +08:00
4db300e95f [ROCm][CI] Removed problematic command override mechanism (#42807) Andreas Karatzas 2026-05-16 04:35:05 -05:00
657b42b592 [Docker][KVConnector] Build mooncake-transfer-engine from source (#42114)
Zhewen Li Zhewen Li khluu
2026-05-16 00:26:25 -07:00
32b7177909 [LoRA][Bugfix] Dedup LoRA wrapping for modules referenced from multiple attribute paths (MoE gate) (#42757) Jee Jee Li 2026-05-16 11:22:35 +08:00
39c67d714e fix: add API key authorization to /v2 endpoints (#42594)
DustHunter Qwen-Coder wang.yuqi
2026-05-16 09:29:27 +08:00
87a2adcb43 [Misc] Add common random prefix option to structured-output serving benchmark (#41632) Viktor Pusandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-16 02:44:48 +02:00
852f567444 [Bugfix] Respect explicit --kv-cache-dtype over checkpoint kv_cache_scheme (#42782) Michael Goin 2026-05-15 20:15:52 -04:00
b2a27b82d9 [Kernel][UX] Add --linear-backend arg for linear kernel selection (#39538) Michael Goin 2026-05-15 20:07:39 -04:00
d0921bafef [Bugfix] Unwrap VLM wrappers for EPLB on Model Runner V2 (#42706) Keyi Li 2026-05-15 16:20:33 -07:00
1ccdf87507 [Bugfix] Fix layerwise reload alias-buffer corruption (#42481)
rasdani OpenAI Codex Roger Wang
2026-05-15 15:20:53 -07:00
bd9dbe6060 [ROCm][Bugfix] Fix fused_mla_dual_rms_norm for AITER API rename _fused_qk_rmsnorm (#42606) Rita Brugarolas 2026-05-15 13:50:03 -07:00
de2d76f352 [Build] Switch CUDA 12.9 wheel builds to PyTorch manylinux_2_28 base (#41668) Michael GoinandClaude 2026-05-15 16:46:16 -04:00
9a7a273dfe Add HumanEval and GSM8K benchmarks to datasets (#42648) Sergei SkvortsovandMichael Goin 2026-05-15 21:01:21 +01:00
b2c58ee942 [FlashAttn] Fix supports_kv_cache_dtype() accepting unhandled fp8 kv-cache dtype variants (#42685) Lanze LiuandMatthew Bonanni 2026-05-15 12:34:59 -07:00
4d67d3bde2 [ROCm] Restore fast top_k_per_row kernels for sparse MLA when topk_tokens=2048 (#42072) frida-andersson 2026-05-15 21:02:57 +02:00
06d020bb6e [Bugfix] Fix SM121 (DGX Spark) exclusion from Marlin/CUTLASS FP8 paths (#35568)
Blake Ledden Claude Opus 4.6 Pavani Majety
2026-05-15 13:59:00 -04:00
f45c210885 [LMCacheMPConnector] Prioritize importing the lmcache_mp_connector from lmcache (#42596) chunxiaozheng 2026-05-16 01:46:31 +08:00
be7a03ea65 [ROCm] Widen AITER fused AR RMSNorm 1-stage gate (#42409) akii96 2026-05-15 20:44:38 +03:00
6147c70224 [Model Runner v2] Support reload weights (sleep mode) (#42673) Wentao Yeandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-15 12:41:23 -04:00
0162596603 [Model Runner V2] FP32 gumbel sampling. (#41775) PatchyTISandPatchouliTaisa 2026-05-16 00:20:08 +08:00
46a95815d3 [ROCm][MLA] FP8 ASM prefill for AITER dense MLA backend on gfx950 (#42509)
Markus Hartikainen clintg6 frida-andersson Cursor
2026-05-15 18:56:58 +03:00
fb5bd03f51 [Perf] Set IR Op Priority Once at Worker Init (#42631) BadrBasowid 2026-05-15 23:56:13 +08:00
ee58665aac [Bugfix] Fix DeepGEMM context lens contiguity in MLA indexer (#42135) Mohammad Miadh Angkad 2026-05-15 23:29:58 +08:00
491e8d8539 [Perf] Optimize MLA attention _v_up_proj bmm by removing additional copy (#42561) Wentao Ye 2026-05-15 11:14:26 -04:00
af9616d845 [Model Runner V2] Fix kv_connector pre_forward order (#42676) Wentao Ye 2026-05-15 11:13:59 -04:00
d792d993c1 [ROCm] Widen OAI Triton MoE capability range to include gfx12 (RDNA4) (#37826) L.B.R.andL.B.R. 2026-05-15 15:59:57 +01:00
e0a45f1455 [Feat][RL] IPC weight sync optimizations: multigpu support and chunked packed tensors (#37476) Aaron Hao 2026-05-15 07:53:06 -07:00
0fe7550254 [Bugfix] DFlash FP8 KV-Cache (#42692) Benjamin Chislett 2026-05-15 10:29:45 -04:00
95cfe102a5 [Bugfix] Ensure embeding model compilation on CPU (#42709) Li, Jiang 2026-05-15 18:58:19 +08:00

... 65 66 67 68 69 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86