Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

20727e2841 [Hardware][AMD] Enable fused bf16→fp32 router GEMM on ROCm (#50268) Matvei PashkovskiiandClaude 2026-08-12 05:46:13 +03:00
f97e502969 [Perf] Avoid more GPU<->CPU syncs on the model execution path (#51738) Nick Hill 2026-08-11 19:30:34 -07:00
23f25aac14 [CI] Stabilize tensor IPC multiprocessing tests (#50804) Andreas Karatzas 2026-08-11 21:16:43 -05:00
f067737b2a [Bugfix][TieredOffloading] : Return HIT_PENDING when KV promotion is triggered (#51840) Varun Sundar RabindranathandVarun Sundar Rabindranath 2026-08-11 22:07:16 -04:00
793ca6998a [Docs][RL] Rewrite weight-transfer docs; standardize examples (#51729) Aaron HaoandClaude Opus 5 2026-08-11 18:53:15 -07:00
9b3382685e [Tools] vLLM Recipes conversion : support different data types variants and strategies (#51878) Louie Tsai 2026-08-11 18:42:08 -07:00
045a986f3b [ROCm][CI] Speed Up ROCm Skinny GEMM Tests (reduced parameterizations, (#51877) Micah WilliamsonandAndreas Karatzas 2026-08-11 20:40:13 -05:00
47ececb58e [Perf][ROCm] Dual-stream decode with hipgraphs (#48223) Simon Danielsson 2026-08-12 03:32:24 +02:00
466855a2bf [ROCm] Enable DeepSeek-V4 on gfx11 (#47017) PikaPikachu 2026-08-12 08:55:10 +08:00
0a94d85a66 [Bugfix][MRV2] Require all requests to be decoding for uniform-decode dispatch (#51865)
Nick Hill Rahul Chalamala Janelle Cai
2026-08-11 17:15:06 -07:00
1d2d83a07f [Attention] Add FlashInfer XQA decode support on SM12x (#49718)
Andrii Skliar Andrii Skliar OpenAI Codex
2026-08-12 01:32:49 +02:00
a367cdbca0 [Profiler] Add minimal Triton Proton profiling backend (#48789) 鐘天楽 2026-08-11 14:51:57 -07:00
61874f9842 [4/N][KV-Cache Layout Refactor] Promote local KV cache specs via a class-changing replace helper (#51612) Lucas WilkinsonandClaude 2026-08-11 17:29:29 -04:00
cb30f6f8dd [Testing] Fix test_sharded_state_loader (#51736) Richard Zou 2026-08-11 17:16:09 -04:00
3e372c5ff2 [Bugfix][ROCm] Give KV-first attention blocks their own page in hybrid models (#51837) stefankoncarevic 2026-08-11 22:52:30 +02:00
0f0cb918b7 [ROCm][MoE] Fix expert_map vs AITER expert_mask for non-AITER experts under EP (#49758) Rohan PotdarandClaude Opus 4.8 2026-08-11 15:33:02 -05:00
36f4630d89 [Bugfix][MoE] Fix fused block-scale orientation (#50727) Andreas Karatzasandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-11 15:05:38 -05:00
fd04edef3e [Refactor] Delete dead code in models (#51838) Wentao Ye 2026-08-11 15:44:26 -04:00
ded6c452c3 [Bugfix] Support HF-config compat for Inkling (#51850) Michael Goin 2026-08-11 15:32:28 -04:00
b2ab096017 [Docs] Fix broken autorefs cross-reference in TurboQuant v2 docstring (#51857) Harry MellorandAndreas Karatzas 2026-08-11 20:22:56 +01:00
cc668c5b72 [CI][Bugfix][V1] Remove stale FlashAttention metadata arguments (#51854) Andreas Karatzas 2026-08-11 13:54:56 -05:00
1ab2801dde [ROCm] Remove stale SDPA and skinny GEMM workarounds (#50907) Andreas KaratzasandOpenAI Codex 2026-08-11 13:19:16 -05:00
4f2f31b82e [Bugfix][Attention] Forward per-head FP8 descales through FA4 (#51363)
Yi Liu OpenAI Codex Wentao Ye
2026-08-12 02:05:20 +08:00
52be12cfac feat: allow shared expert overlapping for FlashInfer one-sided all-to-all (#50569) Julien Debache 2026-08-11 19:06:22 +02:00
9cc347ae42 Enable MiniCPMV for vLLM in CI (#45042) Harry Mellor 2026-08-11 17:30:39 +01:00
ca9c8cbd1b [CI] Support partial torch requirement contexts (#51832) Taneem IbrahimandWentao Ye 2026-08-11 18:25:14 +02:00
e3fe212eaf [Bugfix] Generalize KV block zeroing to AttentionSpec (#51749) Michael Goinandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-11 12:13:09 -04:00
513f83e7ec [Bugfix] Take the sliding window from the layer, not the KV cache group (#51756) Nick Hill 2026-08-11 09:10:46 -07:00
c65aa2ee6c Bump Transformers version to 5.15.0 (#51668) Harry Mellor 2026-08-11 17:00:51 +01:00
457a5f34db [Bugfix][MRV2] Support encoder timing stats in model runner V2 (#50020) Guan-Ming Chiuandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-12 00:00:49 +08:00
5426311d91 [Kernel][ROCm][Perf] FlyDSL decode-attention kernel for 4-bit TurboQuant KV cache (#47896)
aditi-amd root Andreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 08:45:39 -07:00
d648236199 [XPU][CI] Fix ExampleConnector KV cache device selection (#51806) liuzhenweiandKunshang Ji 2026-08-11 23:40:29 +08:00
5af7c8dad7 [Bugfix] Align Qwen GDN gates with speculative tokens (#51812) Jiangyun ZhuandOpenAI Codex 2026-08-11 23:35:30 +08:00
5b5eae2394 [Misc] Enable test_silu_mul_fp8_quant_deep_gemm on XPU (#49444) pmanczakandKunshang Ji 2026-08-11 17:19:14 +02:00
4988df2eb8 [Config] Update default _max_num_batched_tokens from 8192 to 16384 (#51726) Wentao Ye 2026-08-11 11:09:34 -04:00
0fb9897df1 [Bugfix][MoE] Support GELU tanh in FlashInfer B12x MoE (#51819)
Andrii Skliar Andrii Skliar OpenAI Codex
2026-08-11 17:07:58 +02:00
87668ab69b [Bugfix] Guard DeepSeek V4 MRV1 piecewise CUDA graphs (#51768) Woosuk KwonandOpenAI Codex 2026-08-11 07:34:44 -07:00
dd7cc85f17 Add MoE output contract for MoE tail fusion (#51407) Jee Jee Li 2026-08-11 22:23:59 +08:00
12bea3eedc [Bugfix][ROCm] Fix DeepSeek V4 DSpark probabilistic startup (#51145) Tuukka Sarvi 2026-08-11 17:15:31 +03:00
b64a2708b0 Bound generation inputs before expensive work (#51447)
Clinton Thomas Lucas Bourtoulemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 10:03:50 -04:00
5b184f775a connects vLLM Recipes with vLLM's native config-based deployment and benchmark (#51308) Louie Tsai 2026-08-11 06:53:51 -07:00
f863387250 [XPU] Fix UVA weight offloading (non-pinned-tensor views and static Triton launcher) (#51770) Chaojun Zhang 2026-08-11 21:48:44 +08:00
78e7fdd1ab [Bugfix][CPU] Make the Apple Silicon BF16 probe fall back instead of raising (#51627) Kush Zingade 2026-08-11 21:35:07 +08:00
6c95a641e9 [2/N][Feat][Perf] Add new warmup infrastructure for JITs. Add predicate filtering for JIT warmup, and migrate Inkling FA4 (#49315)
Roberto L. Castro Lucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 13:46:07 +02:00
d8f840071e [Model] Enable tower and connector LoRA for Keye (#51780) liushujia122 2026-08-11 18:35:59 +08:00
0fec3d652b [Bugfix][KV Offload] Centralize shared mmap cleanup in CPU worker (#51622) AlexHuang 2026-08-11 18:22:15 +08:00
ce07118669 [Bugfix][Core] Preserve Mamba running CoW after external hits (#51766) Dao007foreverandOpenAI Codex 2026-08-11 03:15:24 -07:00
490259c1f6 profiler: add PrivateUse1 activity support for custom backends (#50977) David Holtz 2026-08-11 11:38:58 +02:00
3fb7bb46d6 [Perf] Avoid repeated multimodal prompt update scans (#51774) Tianyu GuoandOpenAI Codex 2026-08-11 17:23:24 +08:00
52c70b210c [XPU] [Linear] enable torch linear backend for blockwise gemm on xpu (#50826) zofiaandKunshang Ji 2026-08-11 16:43:33 +08:00
529d010351 [Doc] Fix typos in speculative decoding docs (#51500) Kyungmin LeeandHarry Mellor 2026-08-11 17:17:05 +09:00
a311916a29 [MRV2][Spec] Fuse AR speculator multi-step decodes back into one CUDA graph (#46849) Yizhou 2026-08-11 15:06:22 +08:00
1044d19998 [ROCm][DSV4] Preserve native MXFP4 TP8 shard allocation (#51473) Fangzhou Ai 2026-08-11 02:57:27 -04:00
be2e274ef5 Fix docs on main (#51773) Harry Mellor 2026-08-11 07:52:04 +01:00
2acb055ed7 [CPU][Zen] Route BF16 MoE inference through zentorch on AMD (#44201) Priyansh JainandAndreas Karatzas 2026-08-11 11:50:05 +05:30
b76cf758db [XPU] install xpu-manager for device monitor (#50831) Yan MaandKunshang Ji 2026-08-11 14:18:41 +08:00
75903e5067 [CI][AMD] Persist the openai-harmony tiktoken vocab cache across jobs (#51666) stefankoncarevicandAndreas Karatzas 2026-08-11 08:11:06 +02:00
608c12473f [Attention] Fix MLA prefill workspace allocation size (#51733) Wei Zhao 2026-08-11 01:34:30 -04:00
c76a425278 [Kernel] Optimize long-context MLA cache gathers (#51739)
Yongye Zhu OpenAI Codex Claude Opus 5
2026-08-10 22:23:43 -07:00
65b7662d3f [Bugfix][Frontend] Report Cohere stop sequences correctly (#51556) cherry77-cloudandOpenAI Codex 2026-08-11 12:49:49 +08:00
dc5101fb1b replace batch_norm to numerically identical without cudnn (#51734) Khushali Desaiandwang.yuqi 2026-08-10 21:43:20 -07:00
90fd4a333f Preserve revision pins in secondary artifact loaders (#51446) Clinton ThomasandLucas Bourtoule 2026-08-11 00:33:10 -04:00
adc1200597 [Misc] Use VLLMValidationError in scoring input validation (#51753) Frank 2026-08-10 21:12:22 -07:00
07443bea29 [BugFix][Core] free_blocks: restore prepend (LIFO) reuse order when prefix caching is off (#51482) Ming Huang 2026-08-10 20:56:25 -07:00
0914ed2e81 [Perf] Adaptive budget for spec scheduled input tokens, ~60% better Kimi K3 DSpark TTFT (#51725) Wentao Ye 2026-08-10 23:52:21 -04:00
0f2ea973be [KV Connector][Offloading] Keep per-layer KV registration when canonical_layout is requested (#51688) Itay EtelisandItay Etelis 2026-08-11 06:03:59 +03:00
419b51b385 [Bugfix][ROCm][CI] Stabilize build context and source caches (#51721) Andreas Karatzas 2026-08-10 21:51:39 -05:00
99e62b802c [Bugfix][Model Loader] Defer post-load attention weight processing (#49519) aoshen02andRoger Wang 2026-08-11 10:16:57 +08:00
f37dff22fc [Bugfix] Import each packed IPC export once on the consumer side (#51259) acmore 2026-08-11 10:05:27 +08:00
33584901ba [Frontend] Add content_parts to /inference/v1/generate for raw multim… (#51478)
aoshen02 Claude Opus 4.6mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 10:00:12 +08:00
1a1727330a [CI] Solidify speculative decoding E2E coverage (#50713) Andreas KaratzasandOpenAI Codex 2026-08-10 20:56:39 -05:00
48bada6ea4 Fix chat completion 500 on non-object JSON bodies (#51654) Tarun Kumar 2026-08-11 07:01:19 +05:30
f1e921b6d6 [Rust Frontend] Support dynamic tools from developer messages (#51144) Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-11 11:30:38 +10:00
b2506d62ae [MM][CG][BugFix] Fix Ernie-4.5-VL encoder CG postprocess for multi-path outputs (#51461) Qiuyang Yue 2026-08-10 17:39:04 -07:00
d8c70f2243 [Rust Frontend] Upgrade MiniJinja to 2.22 & remove method lookup workaround (#51235) Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-11 10:23:24 +10:00
8a9f9f762a [CI] Parallelize release image publishing (#51735) Kevin H. LuuandOpenAI Codex 2026-08-10 16:38:22 -07:00
8977ea8895 [Perf] Skip detokenization in offline beam search (#50333) Myeongwon Kim 2026-08-11 01:17:38 +02:00
8bcc916a98 [Bugfix] Fix DeepSeek V4/3.2 tokenizer vocab size overcount crashing guided decoding (#51727) Flora FengandWentao Ye 2026-08-10 19:00:08 -04:00
3e174bb73c [Model Runner V2][MTP] Share topk index buffer between draft steps (#47352) Giancarlo DelfinandWoosuk Kwon 2026-08-10 15:46:13 -07:00
c3cac8c63d [Bugfix][MiMo] Apply vision attention sinks in the window attention path (#49815) Almog Tavor 2026-08-11 01:36:42 +03:00
98a4144a41 [Bugfix] Preserve non-logitproc entry points in tests (#51097) Tristan Rice 2026-08-10 15:14:53 -07:00
fa722b9f01 [Rust Frontend][gRPC] Add explicit data-parallel rank routing (#51178) Connor Carpenterandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-10 15:05:24 -07:00
3c6b2e9c08 [CI] Add /ci cancel command (#51732) Kevin H. LuuandOpenAI Codex 2026-08-10 14:55:26 -07:00
635dd6aae6 [Build][gRPC] Publish protobuf schemas to Buf (#51276) Connor Carpenterandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-10 13:34:39 -07:00
63ac04a61e [Kimi-K3] DCP support (#50484)
Summer Yang Claude Fable 5 云挚 foraxe Woosuk Kwon
2026-08-10 12:54:26 -07:00
3f142bd85e [Bugfix] Align deepseek v4 parser thinking default with tokenizer (#51296) Flora Feng 2026-08-10 15:30:30 -04:00
d40c3e3c00 Fix DSpark warmup without sparse index buffer (#50693)
xijiaat Wentao Ye OpenAI Codex
2026-08-11 03:12:19 +08:00
1482d2e015 [ROCm][DistInf] Enable vLLM DI CI with buildkite/slurm (#47030)
Chaitanya Sri Krishna Lolla Sheral Kumar avininjamay8 tej Cursor
2026-08-11 00:09:26 +05:30
05f0a80016 [Bugfix][Structured Output] Mask request stop tokens in xgrammar until grammar terminates (#49227) Flora Feng 2026-08-10 14:29:20 -04:00
405bc86768 [Perf] Launch the top-k/top-p Triton sampler kernel with 8 warps (#51507)
kiroxu Copilot Nick Hill Claude Opus 5
2026-08-11 01:55:35 +08:00
79c865b838 [Perf] Narrow DeepSeek V4 eager CUDA graph region (#51430) Woosuk KwonandOpenAI Codex 2026-08-10 10:52:10 -07:00
21c667aa64 [Docker] Cache test dependencies before vLLM install (#51184)
Michael Goin OpenAI Codex OpenAI Codex
2026-08-10 13:34:59 -04:00
355a338b8f [BugFix][SpecDecode] Fix dspark parallel_drafting_token_id init bug (#51602) wangxiyuanandBenjamin Chislett 2026-08-11 01:33:10 +08:00
3a749ce816 [Build] Skip precompiled wheel fetch during metadata hooks (#51424) Michael Goin 2026-08-10 12:38:53 -04:00
0e2d78028c [Bugfix][Kimi-K3] Give the AMD packed KDA decode kernel the state-index stride (#51682) Lyu, Xudong 2026-08-11 00:05:20 +08:00
640a09086d Fix DoS via sample-rate forgery bypassing audio decode duration guard (#49948) Juan Pérez de AlgabaandCursor 2026-08-10 17:49:17 +02:00
37fbf52084 [ROCm][MLA] [K3] Fix fp8 KV cache decode on the AITER MLA backend (#51011) fanxingran 2026-08-10 23:42:10 +08:00
8fea1d306f Fix uniform_random routing simulation to sample without replacement (#43680)
Elvir Crnčević Claude Opus 4.6gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Tyler Michael Smith
2026-08-10 17:28:57 +02:00
fac808b36f [Perf][Hybrid] 3D-grid tiling of the state-copy Triton kernels (#49436) Francesco FuscoandClaude 2026-08-10 17:02:49 +02:00
bd6536071c [XPU][Test] Pin block size in test_multi_connector (#51213) liuzhenwei 2026-08-10 21:57:45 +08:00

... 34 35 36 37 38 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86