Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

92762edc53 [Bugfix] Treat <tool_call> as implicit reasoning end in Qwen3 parser (#35687) Doug Campos 2026-04-23 21:10:04 -04:00
626daa2076 [Feat] Unified Synthetic Acceptance Rate for V1 and V2 (#40662) Benjamin Chislettandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-04-23 20:48:08 -04:00
fe85a92e86 [Core] Avoid seq_lens_cpu GPU->CPU sync (#40654) Nick Hill 2026-04-23 17:35:55 -07:00
62b1bbe470 [EPLB] Remove asyncio infrastructure from Async EPLB (#40730) Sage Moore 2026-04-23 17:21:15 -07:00
fa4b70555b [ROCm] Cast score correction bias tensor during model construction for DeepSeek/Kimi-K2 (#39999) Hemanth Acharya 2026-04-24 05:32:12 +05:30
447c372ac5 [MoE] Move remaining PrepareAndFinalize to prepare finalize folder (#39009) Jackmin801andRobert Shaw 2026-04-23 17:00:53 -07:00
ff2c2bd80a [Docs]Add documentation for bench serve visualization arguments (#40539) Sophie du Couédicandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-04-24 00:48:29 +02:00
cde8d24710 [Spec Decode] Move SpecDecodeBaseProposer out of eagle.py (#40732) Matthew Bonanni 2026-04-23 18:28:27 -04:00
4a6dd1c3cc [Bugfix] Fix DeepSeek V2-Lite Accuracy drop (#40673) bnellnm 2026-04-23 18:11:37 -04:00
7ff65b1900 [Bugfix] Fix workspace resize leaking reserved GPU memory (#39226) czhu-cohereandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-23 13:50:05 -07:00
7f95a66cbf [NVIDIA] Add sm_110 (Jetson Thor) to CUDA 13.0 build targets (#39233) Johnny 2026-04-23 21:42:14 +02:00
1b1c01de39 [MoE] Move xpu moe to fused_moe/experts/ (#40568)
Jackmin801 Claude Kunshang Ji
2026-04-23 10:38:10 -07:00
e9ba519f45 [DP][Ray] Pin DP control bundle to same node as first GPU bundle (#39167)
shaharmor98 Robert Shawmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-23 20:21:13 +03:00
5ef33ab250 [kv_offload+HMA][10/N]: Support load with multiple KV groups (#39402) Or Ozeri 2026-04-23 20:00:45 +03:00
1c2c1eb8b9 [MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671) bnellnm 2026-04-23 11:22:34 -04:00
8824f50f1f [CI] Split disaggregated tests into own test-area (#40623) Nicolò Lucchesi 2026-04-23 17:20:12 +02:00
0098db9ec1 [ROCm] Implement GPU-to-NUMA-node detection (#40015) pschlan-amdandTJian 2026-04-23 17:08:48 +02:00
53ecc807c0 [XPU] Upgrade torch 2.11 for xpu (#37947) Kunshang Ji 2026-04-23 23:07:35 +08:00
b7a2605020 [Bugfix] Make Attention Backend Auto-Selection Batch-Invariance-Aware (#40193)
Srreyansh Sethi Matthew Bonannimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-23 07:57:03 -07:00
d0009ddb0b [Model] Support Hy3 preview (#40681) stevenkuangandJee Jee Li 2026-04-23 22:08:26 +08:00
424033f4fc [Bugfix] Include inductor and functorch configs in compilation cache key (#40627) Richard Zou 2026-04-23 09:52:59 -04:00
da1e7311ca [Misc] use model arch converter for bidi models identification (#40701) Isotr0py 2026-04-23 21:42:52 +08:00
01cb41dcf5 [XPU][CI]Temporary disable 3 cases on Intel GPU in CI (#40683) xiangdong 2026-04-23 21:42:22 +08:00
2f314bc5e6 [CPU] Added faster exp routine for lower precision data types. (#38112)
almayne Fadi ArafehLi, Jiang <jiang1.li@intel.com>
2026-04-23 14:14:44 +01:00
2196bac135 [Compilation] Refactor SiluMul activation+quant Fusion Pass (#39684) BadrBasowid 2026-04-23 21:10:36 +08:00
4b7869d6bc [ROCm] Add gfx1102/gfx1103 support (#40037) Matthias Gehre 2026-04-23 10:32:04 +02:00
4a79262e0f [UT][Hardware] let torchrun example tests use the default backend (#39879) liuzhenwei 2026-04-23 16:22:28 +08:00
3ed5231c6a [Build] Switch default CUDA to 13.0, update CUDA architecture lists, clean up stale build-args (#39878) Shengqi ChenandClaude Opus 4.6 2026-04-23 15:51:28 +08:00
9c2492e501 [Misc] Support Human-readable (k/K/m/M..) json cli arg (#40473) Nicolò Lucchesi 2026-04-23 09:42:23 +02:00
fe57be7809 [MM][CG] Support --enable-vit-cuda-graph option for VLM examples (#40580) Shanshan Shen 2026-04-23 13:46:14 +08:00
8317cedc77 [Responses] Add tool_choice/tools validation to match OpenAI behavior (#40399) Flora Fengandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-23 01:46:10 -04:00
98a242ff61 [compile] Skip FX graph deserialiaztion on loading, further reducing warm compile time. (#40151) Zhengxu Chen 2026-04-23 01:43:18 -04:00
e4ee48da2d [MoE refactor] refactor GPTQMarlinMoEMethod with MK (#37990)
Kunshang Ji Robert Shaw Robert Shaw
2026-04-23 13:21:47 +08:00
342c58bc54 [BugFix]fix Qwen3 MoE call gate twice (#40664) Kunshang Ji 2026-04-23 13:04:41 +08:00
fe9c3d6c5f [TurboQuant] enable FA3/FA4 for prefill paths (#40092)
Julian Huang 墨楼 Claude Opus 4.6 Codex
2026-04-23 12:35:24 +08:00
ccaf5ffaa3 [XPU] disable fusion pattern support on XPU platform (#39789) Chaojun ZhangandKunshang Ji 2026-04-23 10:07:45 +08:00
0283f303d8 [BE] Fix compile time message to be consistent (use monitoring) (#40641) Lucas Kabela 2026-04-22 17:12:08 -07:00
ac58e2a170 [Fix][MoRI] Align MoRI-IO message format with P2pNcclConnector and vllm-router (#39565) Simon DanielssonandMatvei Pashkovskii 2026-04-23 01:06:31 +02:00
b8401a9bf4 [Bugfix] Fix RMS norm + quant fusion on DeepGEMM UE8M0 path for B200 (#40552) Lucas Kabela 2026-04-22 15:04:42 -07:00
9c271f9403 [gRPC] Add standard gRPC health checking (grpc.health.v1) for Kubernetes native probes (#38016) Honglin Cao 2026-04-22 17:31:00 -04:00
22fa63cfe8 [Bugfix][Torch 2.12] Fix batch_invariant test with allow_override for torch 2.12 upgrade (#40562) Lucas Kabelaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-22 13:48:55 -07:00
8f87eb4622 [Refactor] Clean up log once scope="local" (#40540) Wentao Yeandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-04-22 16:42:43 -04:00
cfa49213d7 [Bugfix][Parser] Fix Mistral pre-v11 tool parser failing on trailing model output (#40531)
Doug Smith Claude Opus 4.6 Flora Feng
2026-04-22 16:35:00 -04:00
29f64c5f5e FlexAttention non-causal support (#40394) Fynn Schmitt-Ulmsandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-22 16:22:57 -04:00
eb6661d522 Fix test_startup.py for torch 2.12 (#40636) Angela Yi 2026-04-22 12:31:41 -07:00
d622e27d2b [NVFP4] NVFP4 MOE emulation fallback for H100/MI300/MI350, standardize TritonExperts usage for OCP MX emulation (#35737) fxmarty-amdandKyle Sayers 2026-04-22 17:58:54 +02:00
5f76b3fb30 [MoE] Convert CT W8A8 To Oracle Structure (#39187) Robert ShawandClaude 2026-04-22 10:53:30 -04:00
809d83c2dc [MoE Refactor] Combine MoERunnerBase + DefaultMoERunner (#40560) bnellnm 2026-04-22 10:43:17 -04:00
33ef1941e2 [Bugfix][CI] Fix v1/kv_connector/unit/test_nixl_connector_hma.py::test_fewer_blocks_with_hma (#40597) Nicolò Lucchesi 2026-04-22 15:21:02 +02:00
a4905133f3 [xpu][rocm] Update current_platform.supports_fp8() for TritonExperts (#40132) Hank_ 2026-04-22 19:39:40 +08:00
ecbe42e991 [Doc] Clarify supported keys for --speculative-config (#40455) xiaoandWangxiaoxiaoa 2026-04-22 19:36:17 +08:00
a250f1bd5f [Bugfix] LoRA for DeepSeek V3.2 (#35077)
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Jee Jee Li
2026-04-22 14:33:50 +03:00
04eac6ba24 [Bugfix][CPU][RISC-V] Clamp exp() input to prevent NaN (#40428) lyd1992 2026-04-22 17:38:18 +08:00
9047288b68 support hotwords for FunASR model (#39674) AllenDouandzixiao 2026-04-22 17:25:06 +08:00
ed6d30377d upgrade tpu-inference to v0.18.0 (#40395) Johnny Yang 2026-04-22 01:33:45 -07:00
6aa057c9d7 [Multimodal] Support custom video metadata for pre-extracted frame sequences (#40133) storyiconandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-22 15:50:04 +08:00
a2bd09c960 [Bugfix] [Reasoning] Add reasoning_start_str/reasoning_end_str properties to reasoning parsers (#40566) Chauncey 2026-04-22 15:27:44 +08:00
123674879e [Model] Add block-local attention and YaRN for local layers to Gemma3 (#39823) philip-essential 2026-04-21 23:34:50 -07:00
4254aeb56f [fix] flaky test_mla_attn_quant_fusion.py (#40530) Carl Y 2026-04-21 23:29:58 -07:00
aad88f8486 [kv_offload+HMA][8/N]: Support multi-group worker transfer (#38453) Or Ozeriandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-22 08:44:00 +03:00
0210024ae7 [Bugfix] Pass effective chat template kwargs to reasoning parsers (#40460) Bugen Zhao 2026-04-22 13:17:51 +08:00
4eafc72928 [Audio] Bundle get_generation_prompt() params into SpeechToTextParams (#36268)
Ekagra Ranjangemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
2026-04-22 00:24:18 -04:00
6d09769700 [ROCm] Support non-causal attention in ROCM_ATTN (#40176) Micah Williamson 2026-04-21 22:57:12 -05:00
4506319a28 [compile] mla + group fp8 fusion (#38877) Carl Yandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-21 20:16:58 -07:00
9b60e2ffaa [Bugfix] Fix quantized model initialization failure with prefetch offloading (#40432) Rishapveer Singhandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-22 05:15:58 +02:00
3951d3eacd [MyPy] Enable mypy for vllm/model_executor/layers/ (#40159) Martin Hickey 2026-04-22 04:15:02 +01:00
6f2c71be8f [Multimodal] Add PyAV video backend for concurrent video decoding (#39986)
Jaseel Muhammad Isotr0py Isotr0py
2026-04-22 07:14:57 +04:00
2463f00fb6 [AMD][CI][BugFix] Override normalize_e4m3fn_to_e4m3fnuz for fnuz machines in test_moe_layer_no_parallel (#40550) rasmith 2026-04-21 21:21:02 -05:00
f946659fff [Bugfix] Fix W4A8_FP8 MoE tp>1 correctness and view() TypeError (#40310)
EdalatiAligemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-21 21:58:33 -04:00
f90aa44662 [NIXL][XPU]Fix nixl import on XPU (#40430) Soila Kavulya 2026-04-21 18:26:33 -07:00
cefa5281a7 [ROCm][P/D][MORI][BugFix] Ensure correct api is used when making requests to prefill / decode nodes (#39835) rasmith 2026-04-21 19:48:25 -05:00
46794958f0 test: add nan/inf clamp regression test for fused_topk_bias (#40553) Jhao-Ting Chen 2026-04-21 17:46:53 -07:00
6ff8dea075 [Bugfix] avoid warmup if text only expectation in multi_modal run (#40409) Khushali Desai 2026-04-21 17:19:50 -07:00
583e6f2226 [ROCm] [Wheel] [Bugfix] [Critical] Remove any packages installed from github from rocm.txt e.g fastsafetensors as it is incompatible with uv pip (#40461) TJian 2026-04-22 09:18:07 +09:00
96a85c5750 [Startup][UX] Enable CUDAGraph memory profiling by default (#38284) Matthew BonanniandTyler Michael Smith 2026-04-21 18:16:59 -04:00
9db4650e5e [MoE Refactor] Add more MoE layer tests (#39349) bnellnmandRobert Shaw 2026-04-21 18:12:36 -04:00
5e584ce9ec [MoE Refactor] Remove SharedFusedMoE class (#35782) bnellnm 2026-04-21 18:12:12 -04:00
1842447c09 [Refactor] Remove unused param (#39750) Wentao Ye 2026-04-21 17:59:20 -04:00
16688b26a6 [Perf] Optimize batch invariant with fused rms norm, 2.1% E2E latency improvement (#40413) Wentao Ye 2026-04-21 15:51:03 -04:00
6fbec8ed47 [Bugfix][Kernel] nvfp4 cutlass MoE: fix nvfp4 experts quant out-of-bounds read for expert counts not divisible by 4 or 16 (#40351) Jakub Zakrzewski 2026-04-21 21:06:09 +02:00
5544f8c18b [Performance] Add is_reasoning_end_streaming() override to GptOssReasoningParser (#35745) FergusandChauncey 2026-04-21 19:31:27 +01:00
9f39b380d0 [Bugfix] Fix spec decode test failures on Blackwell (SM100+) (#39546)
Rishi Puri Claude Opus 4.6 Stefano Castagnetta Matthew Bonanni Benjamin Chislett
2026-04-21 13:21:19 -05:00
9a6a66f3b8 [MRv2]fix: model accuracy regression caused by reusing the stale last_sampled_tokens and draft_tokens (#39833) Zijing Liu 2026-04-21 09:30:32 -07:00
67eb6083e3 Revert "[Misc] Move pyav and soundfile to common requirements" (#40276) Isotr0pyandRoger Wang 2026-04-22 00:08:06 +08:00
6ee081d1d0 Add new tp plan styles to the Transformers modelling backend (#40467) Harry Mellor 2026-04-21 16:51:30 +01:00
66cc3fa559 [Model Runner V2] Multiple prompt logprobs support (#39937) Wentao YeandNick Hill 2026-04-21 11:49:05 -04:00
6d85b36a9f Revert #38730 and #38791 (#40032) Vadim Gimpelson 2026-04-21 19:44:11 +04:00
ab5666eb7c [UX] Bump version in CG memory profiling log message (#40465) Matthew Bonanni 2026-04-21 11:26:06 -04:00
f819265a4a Default to 'align' mamba cache mode for Mamba-based models when speculative decoding is enabled (#40454) roikoren755 2026-04-21 17:51:43 +03:00
936e0b79aa [MM][CG] Optimize default max_frames_per_batch auto-infer for ViT CUDA graph video inference (#40445) Shanshan Shen 2026-04-21 22:47:53 +08:00
b2a5518679 [XPU][CI] Add misc, engine and lora cases on Intel GPU in CI (#39887) xiangdongandKunshang Ji 2026-04-21 22:30:46 +08:00
908a713488 [Bugfix] LoRA: extend expert base_layer loading to Qwen3.5 and Step3.x (#37114) ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 2026-04-21 17:17:03 +03:00
ec5ef0ac73 [Doc] Add Qwen3 AWQ models to documentation (#40034) Yusuf Mohammadandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-21 14:37:41 +01:00
7b1e0b07d0 [Bugfix] Fix dataset name and path argument validation bug in vllm bench serve (#40288) Talor Abramovichandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-04-21 16:14:28 +03:00
d249a9e90e Add Granite 4.1 Vision as built-in multimodal model (#40282)
artem-spector artemspector Claude Opus 4.6
2026-04-21 15:43:39 +03:00
d2e2e856ad [Frontend] Remove frontend pooling multi task support. (#37861)
wang.yuqi Cyrus Leungmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-21 20:27:44 +08:00
766cb65d00 feat(multimodal): support externally processed mm_kwargs with cache injection (#39502) Kris HungandClaude Opus 4.6 2026-04-21 19:31:09 +08:00
28c222157b fix: clamp NaN/Inf in topk_softmax to prevent duplicate expert IDs (#39391) Jhao-Ting Chen 2026-04-21 04:04:41 -07:00
3975eb6de6 Revert "[Startup] Parallelize torch/transformers import + weight prefetch + forkserver prewarm" (#40438) wang.yuqi 2026-04-21 16:47:18 +08:00
5a94a19824 [Bugfix] Normalize malformed dict prompts that carry token IDs in prompt (#40339) Zeyu Zhang 2026-04-21 15:44:36 +08:00

... 72 73 74 75 76 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86