Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

58fcaa0baa [Feat][Core] Add disk offloading support to SimpleCPUOffloadConnector (#49644) Guanyi ChenandClaude 2026-08-07 15:37:47 +08:00
21ea5b4fa1 [rl] Stateful Trainer Send: NCCL + Sparse NCCL [3/N] (#50902) Aaron HaoandClaude Opus 5 2026-08-07 00:35:59 -07:00
43d691ec6b [ROCm][Perf] Kimi-K3 Shard Latent MoE up-projection for ROCm path (#51253) kliuae 2026-08-07 14:11:18 +08:00
b706fd1628 [CI][XPU] Work around intermittent segfault in Intel XPU CI with VLLM_DISABLE_COMPILE_CACHE=1 (#51337) Chaojun Zhang 2026-08-07 13:18:25 +08:00
c810e5ee99 [Bugfix] Fix Mamba all-mode CPU offload boundary alignment (#51100) Qianxu Wangandqianxu 2026-08-06 21:37:48 -07:00
dd856e48bb [Bugfix][EPD][Model Runner V2] Skip gather mm embeddings for encoder only instance (#51222) Tianyu Guo 2026-08-07 11:40:47 +08:00
72c0d67657 [Feat] Support thinking_token_budget in Model Runner V2 (#46727) ChaunceyandNick Hill 2026-08-07 11:36:18 +08:00
5ac2684976 [CI] Re-enable FI autotune in GSM8K config for Qwen3.5-35B-A3B (#51293) Artem Perevedentsev 2026-08-07 05:42:34 +03:00
0406ba22c4 fix pre-commit broken (#51341) Kunshang Ji 2026-08-07 09:17:39 +08:00
e08111211b [Kernel] Support Nvfp4 Cutedsl Moe Swiglu-oai and Relu2(non-gated) Activation (#47106)
Bi Tiekai Claude Opus 5 Yongye Zhu
2026-08-07 08:42:45 +08:00
b1e12d142d [V1] Copy NaN-in-logits counts to host asynchronously (#51304) Nick Hill 2026-08-06 17:05:03 -07:00
d35eb6c440 [CI] Exclude KV-connector subtree from broad source dependencies (#51046) Nicolò Lucchesi 2026-08-07 01:51:10 +02:00
8170c23c4f [Quantization] Share online weight scales across TP (#49764)
Matej Sirovatka OpenAI Codex aoshen02 Claude Opus 5 Roger Wang
2026-08-06 16:34:33 -07:00
27930df9c2 [Model Runner V2] Fix -1 placeholder draft token ids in rejection sam… (#50939) Giancarlo DelfinandNick Hill 2026-08-06 16:32:00 -07:00
9bca7d840d docs(governance): refresh committers list, add TSC note, update project leads (#51300) Simon Mo 2026-08-06 16:28:17 -07:00
9464529612 [ModelRunner v2] Enable decoder token-wise pooling (#50931) Taneem Ibrahim 2026-08-07 00:21:52 +02:00
4d341ca829 fix: resolve silent request skipping in PRIORITY scheduling (#49206) Tejas 2026-08-07 03:51:21 +05:30
a07086e403 [Misc] Upgrade fastsafetensors version, fix metadata is null (#50827)
rongfu.leng Nick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-07 05:48:56 +08:00
c5d470ac4c [ModelRunner V2] Minor indexing optimizations (#51210) Nick Hill 2026-08-06 14:44:13 -07:00
d6af803f43 [Bugfix] Fix packed KV block zeroing stride (#50276) wangxian001andNick Hill 2026-08-07 04:49:06 +08:00
adc3e03517 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[aA][bB]" (#48977) Wentao Ye 2026-08-06 15:30:15 -04:00
d8eabdbfbe [ROCm][MLA] Use asm decode for non-divisor small head counts (#50578)
vanshbhatia-amd vanshbhatia-amd Cursor Markus Hartikainen Liuyinfeng01
2026-08-06 14:56:53 -04:00
7b9f2dad89 [Frontend] Watch frontend processes during engine startup (#43417)
Bugen Zhao OpenAI Codex Nick Hill
2026-08-07 02:17:24 +08:00
4f851bef6c [ROCm][CI] Update AITER AR+RMS e2e fusion counts for final-norm coverage (#51273) Divakar Verma 2026-08-06 14:05:16 -04:00
c56f169d9a [Bugfix] Keep mamba align prefill chunks block-aligned past last_cache_position (#51113)
Yifan Qiao Hernan yanghui1-arch Claude Opus 5
2026-08-06 10:21:00 -07:00
81be2e09ae [Weight processing] Copy over new_data attributes in replace_parameter (#49601)
fxmarty-amd Claudemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 19:03:37 +02:00
b38e111d3e [Attention][MLA] Per-request scheduling for MLA chunked context (#50613)
Matthew Bonanni Claude Opus 5 OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 12:45:44 -04:00
e7b8d59460 Fully generalise input embedding handling in Transformers modelling backend (#51247) Harry Mellor 2026-08-06 17:21:24 +01:00
566c80edf9 [CI] Run basic fullgraph correctness on one GPU (#51271) Michael GoinandOpenAI Codex 2026-08-06 12:09:30 -04:00
7b4ed49628 attn_res kernel latency improvements (#50185) gnovack 2026-08-06 08:53:20 -07:00
41e7746b82 Update vllm to point to flash-attention commit that builds FA3 with torch stable API. (Retry) (#49599) Chris LeonardandShengqi Chen 2026-08-06 11:37:46 -04:00
62a86318de [VocabParallelEmbedding] fix extra_repr fields concat (#51224) Ning Xie 2026-08-06 20:22:18 +08:00
1e05b21d61 Remove the XPU branch of topk_softplus_sqrt (#51242) Liangqiusong 2026-08-06 20:18:45 +08:00
46e6a83ce1 [Bugfix][KV Offload] Clean up resources after initialization failure (#51227) AlexHuangandOpenAI Codex 2026-08-06 19:42:13 +08:00
5fba75aefe [Bugfix][Model] Add missing fused_qkv_a_proj to Kimi-Linear packed_modules_mapping (#51249) yjzandJianDan0212 2026-08-06 19:39:17 +08:00
865781e627 [Rust Frontend] Add standalone Rust renderer (#50289) Sage 2026-08-06 14:23:26 +03:00
ef2615c2e0 [Bugfix][KV Offload] Fall back when MADV_POPULATE_WRITE is unsupported (#51116)
AlexHuang Guy Girmonsky Claude Opus 5 OpenAI Codex
2026-08-06 18:48:01 +08:00
22013f74ff Interns2mobius support (#51149) Lyu Han 2026-08-06 18:39:35 +08:00
e07532b035 [MISC][Bench] refactor throughput and reuse serve's get samples (#50981) Jared Wen 2026-08-06 18:31:23 +08:00
872fd5973e [Bugfix][LoRA] Guard TrtLlm BF16 MoE LoRA gate on activation type (#51002) anhtra3889andKhang Ngo 2026-08-06 17:12:54 +07:00
2fa490470d [Model] Fused mm preprocess normalisation on the Device (#50411) wang.yuqiandCyrus Leung 2026-08-06 18:11:14 +08:00
9c22668436 [Quantization] Preserve precision in online NVFP4 expert packing (#50029) Matej Sirovatka 2026-08-06 02:23:54 -07:00
276f0bb5c1 [XPU] Register fake meta kernel for fp4_gemm (#50946) Chaojun ZhangandKunshang Ji 2026-08-06 17:18:42 +08:00
13726c80fe [CPU] Add MLA backend so DeepSeek-V2/V3 can run on CPU (#49453) maobaolongandLi, Jiang <jiang1.li@intel.com> 2026-08-06 16:51:41 +08:00
ad5280255b [GLM Perf] DSv32/glm use skip topk for MTP case, 2.0x kernel performance improvement (#50904) Wentao Ye 2026-08-06 04:48:36 -04:00
2dfb8ba590 [Perf][CUDA] Programmatic dependent launch for the DSA decode kernels (#50230)
xiaozhoupy Claudemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 15:53:17 +08:00
febea17f6a [Model] Fix weight prefix mapping for native Qwen3.5 text-only checkp… (#50355) zofiaandopencode 2026-08-06 15:40:44 +08:00
470297c143 [HPC Attention Backend] hpc attention backend support bf16 kv cache with fp8 weight (#50980)
Cheng Jiang chengvjiang VAthree
2026-08-06 15:40:14 +08:00
7e724dca89 [ROCm] Fix AITER all-reduce fusion coverage (#50802) Andreas Karatzas 2026-08-06 01:52:23 -05:00
5370461649 [BugFix][KV Cache] Fix hybrid prefix caching with hidden-state extraction (#51108) Canlin Guo 2026-08-06 14:42:37 +08:00
8060260e1d [Linear] [Kernel] add block-wise scaled_mm (#49932) zofiaandKunshang Ji 2026-08-06 14:40:26 +08:00
8c0f029ac7 [XPU] update warning of XPU Graph (#50236) liuzhenweiandKunshang Ji 2026-08-06 14:33:33 +08:00
2e35c529b8 [Bugfix][Quantization] Fix MXFP4 conversion for FlashInfer CUTLASS (#51038) Gabriel WuandYongye Zhu 2026-08-06 14:19:22 +08:00
c0202c5603 [Bugfix][Spec Decode] Auto-enable async scheduling for draft models (#48341) JooHo Lee 2026-08-06 14:51:40 +09:00
b976e980ef [XPU] Route AWQ linear through choose_mp_linear_kernel (#50840)
zofia opencode Kunshang Jimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 13:50:11 +08:00
2a0323b4e0 [XPU][Test] Support MultiConnector accuracy testing on XPU (#51160) liuzhenwei 2026-08-06 13:49:48 +08:00
d779835a11 [XPU] Support MXFP8 linear weights for INC DeepSeek V4 model (#48476)
Xiaochang Wu Copilot Kunshang Ji
2026-08-06 13:17:48 +08:00
777b01d1a8 [MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference (#45254) Qiuyang Yue 2026-08-05 22:04:19 -07:00
821717118f [BugFix][Pooling] Skip weight-prefix probe when model has WeightsMapper (#50890) Li JuqiandCyrus Leung 2026-08-06 12:12:33 +08:00
f84df12c6a [Bugfix][MM] Fix MiniCPM-V placeholder replacement and image processor loading on Transformers v5 (#48413) Yunzhu LuandCyrus Leung 2026-08-06 12:12:00 +08:00
22170354d8 [Feature] Parse request priority from HTTP header (#51089) Chauncey 2026-08-06 12:10:29 +08:00
2e09247c2d [Docs] List Intel XPU attention backends (#51215) baodi 2026-08-06 10:58:28 +08:00
d54b58cc48 [Hardware] Use torch.accelerator.empty_host_cache() for host cache cl… (#51107) liuzhenwei 2026-08-06 10:27:57 +08:00
81bc196913 [Model Runner V2] Cache draft logits in model's LM head dtype (#50910) Giancarlo Delfin 2026-08-05 19:12:46 -07:00
f85c1d2f84 K3: remove the add operation for megamoe path (#51146) Jee Jee Li 2026-08-06 09:48:25 +08:00
71f975af43 [ROCm][CI] Add MLA decode accuracy and determinism tests (#50480) Aarushi Jain 2026-08-05 20:35:56 -05:00
9f3169960a [MoE] Align TRTLLM MXFP4 autotune buckets (#50942) Thien TranandOpenAI Codex 2026-08-06 09:17:53 +08:00
b50fdebce0 [Bugfix] Fix level-2 sleep/wake/reload with enable_lora=True (#39935)
Silen Naihin Dakai An Jiangyun Zhu
2026-08-05 18:13:10 -07:00
8cfa01cdd2 [BugFix] Dense multinode DP rescope with regression test (#49212) aoshen02andAndy Lo 2026-08-06 09:12:03 +08:00
47a4e410ba [Bugfix][Spec Decode] Fix NaN handling in rejection sampler tl.argmax (#50183) gabriel-peracioandKimi K3 2026-08-05 22:01:09 -03:00
d36f24b66a [KV Connector][Mooncake] Add tenant ID support to MooncakeStoreConnector (#48069) LZWandRoger Wang 2026-08-06 08:31:18 +08:00
7c77868cdf [Bugfix] Size and iterate w13 by shard count for non-gated MoE (#51125)
aoshen02 Matej Sirovatka Rishi Puri Hsiao-Yuan Chen Claude Opus 5
2026-08-06 08:24:47 +08:00
811622c410 [Refactor][PCP] Make PCPManager construction extensible (#50066) QiuandLucas Wilkinson 2026-08-06 07:05:14 +08:00
b30291cf0c [EPD] Remove duplicate image preprocessing in EPD and enable preprocess on GPU (#50390) Tianyu Guo 2026-08-06 06:59:54 +08:00
38ebd97bca [ROCm] Relax MLA rope+cache test tolerances for bf16 (#51083) Rohan PotdarandClaude Opus 4.8 2026-08-05 17:50:52 -05:00
877975897d [K3 Perf] Combine multiple all gather together for SP, 1.5~3x kernel level performance improvement (#51070) Wentao Ye 2026-08-05 17:42:41 -04:00
373fe8b83e [MoE Refactor] Remove MoE legacy code (#51078) bnellnm 2026-08-05 17:35:36 -04:00
4282fe52d5 [ROCm][CI] Add More AITER quantization/MoE kernel tests (#49375) Micah WilliamsonandAndreas Karatzas 2026-08-05 16:27:33 -05:00
482c613423 [Bugfix][Humming] Preserve ModelOpt FP8 weight dimensions (#51093) Netanel Haber 2026-08-06 00:08:55 +03:00
e76b71f659 [CI Bug] Fix pydantic_core._pydantic_core.ValidationError: Input should be a valid integer (#51179) Wentao Ye 2026-08-05 16:57:47 -04:00
65addac701 [ROCm][CI] Keep rocprofiler-sdk out of DeepEP HT MoE test workers (#51173) stefankoncarevicandAndreas Karatzas 2026-08-05 22:28:41 +02:00
c2d8009044 [ROCm] Work around DeepEP teardown SIGSEGV in MoE test harness (#51174) Rohan PotdarandClaude Opus 4.8 2026-08-05 14:44:42 -05:00
e6d67fddb4 [CI] Prune PyTorch Fullgraph Test (#51074) Michael Goin 2026-08-05 15:27:27 -04:00
8f158d0ee2 [MoE] Share apply_moe_activation support metadata (#44359) Michael GoinandOpenAI Codex 2026-08-05 15:05:47 -04:00
14e57ad47d [Docker][KVConnector] Install mooncake from official wheels instead of a custom build (#51067)
Zhewen Li Zhewen Li Claude Opus 5
2026-08-05 11:58:44 -07:00
613411a90c [CI Bug] Fix Chunked prefill is required for mamba cache mode 'align'. (#51177) Wentao Ye 2026-08-05 14:33:45 -04:00
bc37fc970e Revert [Misc] Avoid importing nixl_ep on every vllm serve config (#50879) (#51176) fxmarty-amd 2026-08-05 20:04:40 +02:00
23c0f337b7 [CI bug] Fix Each KV cache group's real block_size must be divisible by has h_block_size (#51180) Wentao Ye 2026-08-05 14:03:17 -04:00
9833aa53d5 [Bugfix] Fail fast with a clear error when CPU offload region exceeds available space (#50358)
AlexHuang OpenAI Codex Or Ozeri
2026-08-06 01:59:55 +08:00
e8586b2878 [Build] Remove Ubuntu build-stage option from the CUDA dockerfile (#51060) Michael GoinandShengqi Chen 2026-08-05 13:50:58 -04:00
8543522ca7 [KV Offload] Support out-of-tree secondary tier managers via module_path (#51007) Ronen SchafferandOr Ozeri 2026-08-05 20:30:53 +03:00
d4da0c55af [Model][Frontend] Add Ling 3.0 Flash BF16, MTP, and parser support (#51045) zexplorerhj 2026-08-06 00:38:28 +08:00
66b3c0e61f [Kernel][Model] Optimize FA4 mm_prefix range lookup (#50294)
Li Juqi Matthew Bonanni Claude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 22:33:49 +08:00
62c5e21621 [KV Offloading] Support partial-tail prefix reuse with fine-grained prefix matching (#50507) ChaunceyandOr Ozeri 2026-08-05 22:31:50 +08:00
08b8613b7b [Bugfix][Model] Fix MiniMax-M3 NVFP4 inference correctness (#48929) Gabriel WuandPavani Majety 2026-08-05 22:25:46 +08:00
cd930c8e78 [Bugfix] Fix MLA kv_b_proj activation dtype with Marlin FP8 (#38771)
Jacob Zhang Matthew Bonannimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 07:25:41 -07:00
397094da17 Resolve revision to commit_hash once per model load, via huggingface_hub's resolve_revision (#49990)
Lucain OpenAI Codex Claude Opus 5 Harry Mellor
2026-08-05 15:49:56 +02:00
a9b39d6dcb [Bugfix] Enable chunked prefill for qwen3.5-0.8B ppl test (#51153) music-dino 2026-08-05 15:21:43 +02:00
beca88e59e [BugFix][K3] Skip moe_intermediate padding when EP is enabled (#51131) Ziming Huang 2026-08-05 17:22:55 +08:00
a3b86752fb [CI] Fix CI authorization notification fallback (#51095) Kevin H. LuuandOpenAI Codex 2026-08-05 01:34:48 -07:00

... 36 37 38 39 40 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86