Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

160f7f0840 [ROCm][CI] Extended Fused MoE and FP8 MoE test support (#41100) Andreas Karatzas 2026-08-19 10:19:42 -05:00
be06873198 [Bugfix] compressed-tensors: restore int8 grouped WNA16 MoE support (#52002) Yohann PrigentandMisha Goin 2026-08-19 16:55:59 +02:00
2d7f42b4f3 [Build] Add InstantTensor to CUDA dependencies (#52801) Misha GoinandOpenAI Codex 2026-08-19 10:55:09 -04:00
db92053e97 [Core] Skip broadcasting mm tensor data to workers for prefix-cache-covered items (#52041) Xiaoan (Sean) LiuandNick Hill 2026-08-19 07:16:39 -07:00
2b7fcbf527 [Kernel] SM120: stop routing misaligned-M blockwise FP8 GEMMs to the small-M swapAB config (#52775) Gabriel Wu 2026-08-19 21:25:29 +08:00
c2e7242ab7 [Bugfix][LoRA] Guard None group members in expand_packed_lora (partial LoRA on Qwen3.5/3.6 GatedDeltaNet) (#47640) EilamandJee Jee Li 2026-08-19 16:24:26 +03:00
92bdee05cb [Bugfix][Frontend] Return all choices from /inference/v1/generate when n > 1 (#52399) Quentin Gallouédec 2026-08-19 08:48:45 -04:00
58de6cbdc2 Add NemotronH_Omni_Reasoning_V3 as a supported Nemotron architecture (#52929) Nave Assaf 2026-08-19 15:34:50 +03:00
2f54100a59 [CI] Fix docs build (#52937) Harry Mellor 2026-08-19 13:25:46 +01:00
58302b4591 [Doc] Fix group numbering in Case 3 of hybrid_kv_cache_manager.md (#52160) QWERQWERQWE86 2026-08-19 19:38:34 +08:00
b160cab156 [Bugfix] Restore model info caching for package backends (#52690) hy123andOpenAI Codex 2026-08-19 19:18:10 +08:00
cba06764d7 [Platform] Fill in the missing backend parameter for torch.compile (#51781) wangxiyuan 2026-08-19 18:31:19 +08:00
eac636a7fa [Frontend] Move api_server.py out openai folder (#52131) wang.yuqi 2026-08-19 17:43:46 +08:00
340b7e4909 fix: reject string schemas that mix pattern/format with length bounds (#49996) Yufeng He 2026-08-19 17:29:32 +08:00
93eea4f665 [XPU][CI] downgrade sentencepiece (#52904) Qiming Zhang 2026-08-19 02:01:12 -07:00
9a9aa2b017 [Bugfix] Redact api_key in non-default args log (#52523) LuckyStep 2026-08-19 15:52:53 +08:00
842dd8fd96 [Pooling] Use semantic task validation errors (#52867) Taneem Ibrahim 2026-08-19 02:21:08 -05:00
b09bd69b5b [Model][NVIDIA] Route DSA models to the CUDA non-compiled path (#52861) Woosuk Kwon 2026-08-19 00:13:52 -07:00
ee11730751 [BugFix] Revert incorrect MM keep_on_cpu=True changes (#52881) Nick Hill 2026-08-19 00:07:40 -07:00
5a4c8d9924 [Bugfix][LoRA] Add embedding_modules for Qwen3.5 CausalLM (#48850)
Agoni-02 m0_68169237 Cursor Composer Jee Jee Li
2026-08-19 12:55:20 +08:00
86a89a9348 [Bugfix][Frontend] Run the serve arg checks for vllm launch too (#52825) Vineeth Sai Varikuntla 2026-08-18 21:48:50 -07:00
08afae2786 [ModelRunner v2] Enable MRV2 for pooling models by default (#48290) Taneem Ibrahim 2026-08-18 23:36:05 -05:00
63ff748f65 [Attention][MLA] FlashMLA sparse: DCP on the fp8_ds_mla mixed-batch path + MTP (#46514)
drakosha Claudemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-19 07:02:09 +03:00
3130573630 [Model] Add GraniteSWA and GraniteMoeSWA via existing Granite (#52706)
Davis Wertheimer Claude Harry Mellor
2026-08-18 23:32:46 -04:00
8e46accab2 [Attention] Vectorize sparse MLA mask loads (#52217) Matthew Bonanniandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-18 23:32:41 -04:00
e575b5f1a9 [XPU][CI] fix hf runner (#52730) Qiming Zhang 2026-08-18 20:30:47 -07:00
deeeae75d0 [MM] Keep more metadata tensors on CPU (#52827) Nick Hill 2026-08-18 19:56:33 -07:00
aabc1a0a0b [Bugfix][Benchmark] Check readiness before tokenizer init in rust vllm-bench (#51863) Tyler Michael SmithandOpenAI Codex 2026-08-18 22:52:25 -04:00
03b87dc42f [XPU] upgrade requirements/test/xpu.txt (#52672) Qiming Zhang 2026-08-18 19:47:37 -07:00
f936a267f9 [Bugfix][XPU] Fix Mamba state pointer overflow (#48109) Willow Lopez 2026-08-19 10:45:41 +08:00
e4d61d0d22 [CPU] Add AMX-only high-performance MLA backend for DeepSeek V2/V3/R1 (#52616) Li, Jiang 2026-08-19 10:28:52 +08:00
f485081e8b [XPU] Support EC connector KV Offloading on XPU (#49532) Chaojun Zhang 2026-08-19 10:25:23 +08:00
b1d9337e92 [EPD] Allow KV consumers to omit MM embeddings (#52697) liuzhenwei 2026-08-19 10:16:08 +08:00
a2257f95b7 [Elastic EP] Reduce eager-mode reconfiguration downtime (#51885) Itay Alroy 2026-08-19 05:13:06 +03:00
d36cc4254e [Bugfix][Elastic EP] Reject scale below the minimum data parallel size (#52702) Itay EtelisandItay Etelis 2026-08-19 04:37:29 +03:00
b05ae5dc00 [CI][Bugfix] Complete DeepSeek-V4 FSE test fixture contract (#52842)
Kevin H. Luu Thomas Parnell Claude
2026-08-18 18:04:10 -07:00
f1178f3a06 Revert DSv4 eager workspace reuse (#52836) Woosuk KwonandOpenAI Codex 2026-08-18 17:02:59 -07:00
a9f4afb66f [Bugfix] Fix DeepSeek V4 mHC broadcast buffer for dummy load (#51368) ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 2026-08-19 03:02:18 +03:00
ef47a897e2 [Core] Make prefix-cache NONE_HASH deterministic by default (#51875) Russell BryantandClaude Opus 4.8 2026-08-18 19:14:39 -04:00
8f4a7f45c5 [ROCm][CI] Gating more ROCm tests (#44969) Andreas KaratzasandOpenAI Codex 2026-08-18 17:45:47 -05:00
aa6abec49a [CI][ROCm] Prevent Git maintenance races during shallow fetches (#52810) Andreas KaratzasandOpenAI Codex 2026-08-18 17:34:50 -05:00
203926c477 [ROCm][CI] Add AMD CI Pull-Request Commands (#52822) Andreas Karatzas 2026-08-18 17:16:25 -05:00
5f7a20b316 [nv] add pcp support in dsv3.2 (#52046) Summer Yang 2026-08-18 14:59:08 -07:00
d3fafe0c27 [Quantization] Remove the dead ocp_mx_scheme branch from moe_kernel_quantize_input (#52603) xuebwang-amdandClaude Opus 5 2026-08-19 05:42:45 +08:00
6066bb3d50 [ROCM][CI] Attention test speedup (#52763) stefankoncarevic 2026-08-18 23:40:30 +02:00
5d8a4cf976 [ROCm] Pad non-aligned AITER MLA heads (#51647) yinfengLiuandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-19 05:37:10 +08:00
12f64b39d2 [Bugfix][Structured Output] Stop XGrammar token batches at termination (#52805) Flora Feng 2026-08-18 17:21:43 -04:00
7ddb50788d [Bugfix][DP] Don't assume the engines started when forwarding a wake (#51481) aoshen02andClaude Opus 5 2026-08-19 04:49:35 +08:00
6a391a931a [Rust Frontend][RL] add routed expert prompt offset (#52703) Biswa Pandaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-18 13:04:28 -07:00
8d6b18329a [CI] Standardize test job labels by device (#52659) Kevin H. LuuandOpenAI Codex 2026-08-18 12:50:32 -07:00
9842d70145 [DBO][CI] Increase the coverage of prefill DBO in test_dbo.py (#48628) Sage Moore 2026-08-18 12:00:04 -07:00
6948a43fbb [Bugfix] Detect all attention-spelling variants in ModelConfig.is_hybrid (#52161) Michał GanczarenkoandWentao Ye 2026-08-18 20:43:51 +02:00
90984ddbed [CI] Upgrade huggingface-hub to 1.28.0 (#52797) Andreas KaratzasandOpenAI Codex 2026-08-18 13:35:44 -05:00
d75136c030 [Rust Frontend] Wait for all utility calls to finish (#52671) Connor CarpenterandBugen Zhao 2026-08-18 11:23:34 -07:00
01e56caaf2 [Bugfix][MLA] Do not use Dense MHA for GLM-5.2 (#52512) Woosuk KwonandOpenAI Codex 2026-08-18 11:20:48 -07:00
ad5e71b276 [ROCm][Perf] Enable fused KDA decode on gfx942 (MI325X) (#52293) Matvei PashkovskiiandClaude 2026-08-18 21:01:00 +03:00
ddbf826bee [ROCm] Gate Torch FP8 scaled-MM on architecture support (#51021) Strahinja Stamenkovicandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-18 19:54:51 +02:00
bca7bea240 Remove VLLM_TEST_FORCE_FP8_MARLIN to replace with linear_backend/moe_backend (#52182) Misha GoinandOpenAI Codex 2026-08-18 12:58:04 -04:00
88b2bff2c6 [MOE] Standardize and abstract fused shared expert optimization selection (#51695) fxmarty-amd 2026-08-18 18:56:56 +02:00
f9f066d195 [Bugfix][PaliGemma] Remove stale image embedding scaling (#52692)
Jikui Xie OpenAI Codex Harry Mellor
2026-08-19 00:00:55 +08:00
241ff8c443 [Model] Enable LoRA support for tower and connector in LlavaNextForConditionalGeneration (#49788) karthik 2026-08-18 23:53:49 +08:00
3bb9c18f0c [Multimodal] Reorganize video decoder backends (#49155) Isotr0py 2026-08-18 23:06:18 +08:00
689be2bcd3 Upgrade Flashinfer version to 0.6.17 (#52681) Wei Zhao 2026-08-18 11:04:36 -04:00
d29dc3ab87 [Bugfix][Gemma4] Align parser enable_thinking default with template (#52430)
Alexander Lee Chaunceymergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-18 08:41:26 -04:00
be3f614ff1 [CI] Register CPU CI "VLLM_CPU_CI_ENV" environment variable (#52633) Taneem Ibrahim 2026-08-18 07:28:00 -05:00
41f179b57a [Rust Frontend] Simplify data-parallel size ownership (#52575) Bugen ZhaoandNick Hill 2026-08-18 19:06:05 +10:00
eab1cff5b0 Harden DeepSeek V3.2 fused kernel grids (#52381) yimdevandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-18 16:43:12 +08:00
e8ad2855e7 [Bugfix][ROCm] Fix a few int4/int8 quantization errors (#52112) qli88 2026-08-18 03:31:51 -05:00
5c9ff5366b [Bugfix] Accept logprobs=-1 in the Completion API (#46175) Yufeng He 2026-08-18 16:22:13 +08:00
b01728b088 [Bugfix] Return 4xx for client-caused errors in /detokenize (#52622) Rajath Pai 2026-08-18 09:07:15 +02:00
2687fec6ef Replicated embedding and norm fusion for DSV3 flat model (#48484) Jee Jee Li 2026-08-18 14:53:21 +08:00
5fa8ca971a [ROCm][CI] Move ROCm AITER quantization tests (#40938) Andreas KaratzasandOpenAI Codex 2026-08-18 01:50:35 -05:00
d785eb51ce [Test] Add pause/resume E2E tests (#52144) flb_ 2026-08-18 14:46:44 +08:00
c89d692cb5 [Build] Propagate vLLM version to Rust binaries (#52593)
Bugen Zhao Nick Hill Andreas Karatzas
2026-08-18 16:37:37 +10:00
b0e9cff5e7 [XPU] update xpu-manager to v2.1.0 (#52569) Yan MaandKunshang Ji 2026-08-18 14:27:41 +08:00
aa9903490c [Cohere] Misc changes to cohere model definitions (#50156) Komal Kumar TeruandCursor 2026-08-18 00:59:57 -04:00
101c4477dd [Bugfix] Handle DeepseekV4ForCausalLM in benchmark_moe get_model_params (#52044) Do_it_now_!andDo_it_now_! 2026-08-18 12:46:08 +08:00
e0e5a7fb28 [Rust Frontend] Fix GLM-5.2 chat template rendering parity (#51426)
Woosuk Kwon Bugen Zhao OpenAI Codex
2026-08-17 20:16:54 -07:00
d5f5de7a7d [Bugfix] Fix DeepSeek V4 mHC broadcast buffer for weight sync (#52626) ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 2026-08-18 05:55:37 +03:00
f4b161d7fc [XPU][UT] Fix OOM and skip graph case (#49287) Qiming ZhangandKunshang Ji 2026-08-17 18:47:35 -07:00
69d3335066 [Bugfix][Quantization] Guard the MXFP8 FlashInfer path on FlashInfer availability (#52648)
Han Claude Opus 5 Misha Goin
2026-08-18 03:38:42 +02:00
cdb8545a91 [Kernel][Perf] Support Qwen head ratios in fused GDN MTP (#52539) kiroxu 2026-08-18 09:17:40 +08:00
c296851a7d [MoE] Refine FlashInfer one-sided All2All integration (#51924) Bo Liandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-18 08:18:14 +08:00
0db502c8d8 [Kimi-K3] support DCP partial prefix cache hit (#50493)
Summer Yangmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Yifan Qiao
2026-08-17 17:05:51 -07:00
0e8989b416 [ROCm] gaurd on_gfx1250 call with rocm platform (#52625) Kunshang Ji 2026-08-18 08:05:48 +08:00
58aa1e3d26 [Bugfix][SM120][MLA] Disable dense prefill for FlashInfer sparse MLA (#51395) Tommy AsaiandRoger Wang 2026-08-18 01:58:02 +02:00
c296cf8259 [Bugfix] Add forward_xpu to XDRotaryEmbedding for HunyuanOCR on XPU (#52174) Jakub ByczkowskiandCopilot 2026-08-18 01:54:05 +02:00
49fb2ee348 [ROCm][CI] add Aiter ops tests (#52208) Divakar Verma 2026-08-17 19:33:22 -04:00
5ae2d38821 [Perf][Structured Output] Skip unused request-local reasoners (#52573)
Bugen Zhao OpenAI Codex Nick Hill
2026-08-18 09:30:41 +10:00
60c3a31b12 [CI][AMD] Improve Kubernetes failure diagnostics (#52264) Andreas KaratzasandOpenAI Codex 2026-08-17 18:09:12 -05:00
455edc022b [ModelRunnerV2] Support prompt embeds (#42963) Canlin GuoandNick Hill 2026-08-18 05:58:50 +08:00
e68fb75b25 [ROCm][AMD][Installation] add LMCache kv-connector installation and runtime packages to docker image (#51208) Hongxia YangandAndreas Karatzas 2026-08-17 17:58:48 -04:00
8878ebd8fd [ROCm][CI] Expand AITER W4A4 MoE Coverage (#52647) Micah Williamson 2026-08-17 15:46:22 -05:00
f08a95f8d8 [Rust Frontend][gRPC] Advertise LoRA capabilities (#52031) Connor Carpenterandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-17 13:27:56 -07:00
d1e3eee6fb [Spec decode] Support Kimi-K3 DCP with DSpark (#52188) Wei Zhao 2026-08-17 16:08:45 -04:00
9633933dd8 Relax CuPy constraint to only exclude 14.1.0 (#44284)
Kevin H. Luu khluu Nick Hill
2026-08-17 12:41:09 -07:00
3fc2893909 [Bugfix] Account for local DP workers in startup thread allocation (#52385) crZhao 2026-08-17 12:10:34 -07:00
c1e438728c [ROCm][CI] Restore Torch defaults and type DSV4 scratch buffers (#52566) Andreas Karatzasandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-17 13:32:04 -05:00
75dde08d3f [Perf][MoE] Optimize deepep_v2 receiver CPU Overhead (#51114)
Lucas Wilkinson Claude Opus 5 Robert Shaw
2026-08-17 14:21:32 -04:00
402547d7f0 [Bugfix][CI] Release the shared ColBERT engine before test_colbert_hf_comparison (#52608) stefankoncarevicandAndreas Karatzas 2026-08-17 19:50:26 +02:00

... 31 32 33 34 35 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86