Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

87f05d6880 [Revert] Remove DeepGEMM availability check in DeepseekV32IndexerMetadataBuilder (#38076) Chauncey 2026-03-26 09:43:51 +08:00
36f6aede23 [Misc] Optimized check to encapsulate both CUDA and ROCm platforms (#34549) Andreas Karatzas 2026-03-25 20:43:07 -05:00
9704a5c310 Disable dual stream execution of input projection for Qwen3 (#38152) Xin Yang 2026-03-25 18:20:39 -07:00
74056039b7 Fix minimax m2.5 nvfp4 kv scales weight loading (#37214) Wei Zhao 2026-03-25 20:48:06 -04:00
d7d51a7ee5 [Bugfix] Fix Qwen3.5-FP8 Weight Loading Error on TPU (#37348) Jacob Platin 2026-03-25 17:46:01 -07:00
3c3c084240 Various Transformers v5 fixes (#38127) Harry Mellor 2026-03-26 00:10:08 +00:00
7b54f60db0 [Cohere] Enable Cohere-Transcribe (#38120) Ekagra Ranjan 2026-03-25 19:13:51 -04:00
a0e8c74005 [ROCm]: Update rope+kvcache fusion conditions and disable custom op by default (#36716) Rohan Potdar 2026-03-25 15:58:44 -05:00
70a2152830 [MultiModal] add support for numpy array embeddings (#38119)
Guillaume Guy guillaume_guy Cyrus Leung
2026-03-25 15:13:04 -05:00
978fc18bf0 [ROCm] Utilize persistent MLA kernel from AITER (#36574) Sathish Sanjeevi 2026-03-25 12:00:42 -07:00
7d6917bef5 [ROCm] Fix MoE kernel test failures on gfx950 (#37833)
Andreas Karatzas Woosuk Kwon Matthew Wong
2026-03-25 13:46:40 -05:00
e38817fadb [Core][KV Connector] Remove use of num_cached_tokens in error handling (#38096) Mark McLoughlin 2026-03-25 18:20:48 +00:00
72cad44d3c [Frontend] Move APIServerProcessManager target server fn (#38115) Nick Hill 2026-03-25 11:14:41 -07:00
ba2f0acc2d [Misc] Reorganize inputs (#35182) Cyrus Leung 2026-03-26 01:22:54 +08:00
678b3c99e8 [MoE Kernel] Flashinfer nvfp4 cutedsl moe kernel integration (#38050) Yongye Zhu 2026-03-25 13:16:40 -04:00
bf4cc9ed2d [2/n] Migrate per_token_group_quant to torch stable ABI (#36058) mikaylagawarecki 2026-03-25 13:15:13 -04:00
1ac2ef2e53 [CI/Docs] Improve aarch64/DGX Spark support for dev setup (#38057)
Ben Browninggemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Harry Mellor
2026-03-25 12:24:42 -04:00
6e37c46b35 [compile] Add some more startup tests for top models (#38046) Richard Zou 2026-03-25 12:02:22 -04:00
1bf2ddd0ee [Refactor] Rename WAITING_FOR_FSM to WAITING_FOR_STRUCTURED_OUTPUT_GRAMMAR (#38048) Wentao Ye 2026-03-25 11:41:44 -04:00
e7221180e1 [Kernel] Optimize SM120 CUTLASS blockwise FP8 GEMM (#37970) NecofishandMichael Goin 2026-03-25 23:20:04 +08:00
4a76ad12e0 [Bugfix] Preserve CUDA arch suffix (a/f) for SM12x — fixes NVFP4 NaN on desktop Blackwell (#37725) RobTandandLucas Wilkinson 2026-03-25 11:18:25 -04:00
d7e93e13fb [Feature] EPLB Support for GPU Model Runner v2 (#37488) Wentao YeandWoosuk Kwon 2026-03-25 11:16:39 -04:00
cd7643015e [Feature] Support per-draft-model MoE backend via --speculative-config (#37880) Andrii SkliarandAndrii Skliar 2026-03-25 15:31:52 +01:00
a1a2566447 [Docs] Add guide for editing agent instruction files (#37819) Ben Browning 2026-03-25 09:54:09 -04:00
b745e8b5d3 [KVTransfer][Mooncake] Add heterogeneous TP support for disaggregated P/D in MooncakeConnector (#36869) yjz 2026-03-25 21:24:07 +08:00
d215d1efca [Mypy] Better fixes for the mypy issues in vllm/config (#37902) Harry Mellor 2026-03-25 13:14:43 +00:00
34d317dcec [CPU][UX][Perf] Enable tcmalloc by default (#37607) Fadi Arafeh 2026-03-25 12:39:57 +00:00
7ac48fd357 [Model] Add AutoWeightsLoader support for jais (#38074) grYe99andgrYe99 2026-03-25 20:38:40 +08:00
d6bb2a9d9a Fix Plamo 2/3 & LFM2 for Transformers v5 (#38090) Harry Mellor 2026-03-25 12:29:49 +00:00
1e673a43ce Better weight tying check for multimodal models (#38035) Harry Mellor 2026-03-25 12:07:23 +00:00
04417ecd5f [ROCm][CI] Rename filepath test to point to correct file (#38102) Andreas Karatzas 2026-03-25 07:05:46 -05:00
242c93f744 [Docs] Adds vllm-musa to custom_op.md (#37840) R0CKSTAR 2026-03-25 19:54:36 +08:00
a889b7f584 [Bugfix] Pass drafter quant_config to ParallelLMHead in Eagle3 (#37280) Matthias Gehre 2026-03-25 12:42:58 +01:00
ba2910f73a Fix offline mode test for Transformers v5 (#38095) Harry Mellor 2026-03-25 11:39:48 +00:00
f262a62aa1 [ROCm][CI] Fix flaky Cohere/OpenAI embedding parity test (#37616) Andreas Karatzas 2026-03-25 05:55:51 -05:00
9ac2fcafbb [CI] Fix realtime WebSocket timeout deadlock and unhandled model validation errors (#37483) Andreas Karatzas 2026-03-25 05:24:33 -05:00
e9ae3f8077 [Hardware][XPU] Align memory usage with cuda on xpu (#37029) Kunshang Ji 2026-03-25 18:14:29 +08:00
04cec4f927 [ROCm][CI] Increase OpenAPI schema test timeouts (#38088) Andreas Karatzas 2026-03-25 05:06:58 -05:00
14771f7150 [XPU] support MLA model on Intel GPU (#37143) Kunshang Ji 2026-03-25 17:43:42 +08:00
189ddefbfd [ROCm] Attention selector reordering (#36702) Gregory ShtrasbergandMicah Williamson 2026-03-25 04:42:56 -05:00
09c3dc9186 [Revert] Remove CUDA torch fallbacks for fp8_mqa_logits/fp8_paged_mqa_logits_torch function (#37968) Chauncey 2026-03-25 14:19:37 +08:00
42e9547976 [ROCm][Test] Fix ROCM_AITER_UNIFIED_ATTN attn+quant fusion test (#37640) vllmellm 2026-03-25 13:06:15 +08:00
a32783bb35 [Bugfix] Fix IndexError when accessing prev_tool_call_arr in OpenAIToolParser (#37958) Chauncey 2026-03-25 12:06:21 +08:00
9d0351c91d [Docs] Add Encoder (ViT) CUDA Graphs section to CUDA Graphs design doc (#37914) Baorun (Lauren) Mu 2026-03-24 22:53:24 -04:00
a93a53f8a1 [Performance] Auto-enable prefetch on NFS with RAM guard (#37673) Artem Perevedentsev 2026-03-25 02:31:14 +02:00
679c6a3ecc [Bugfix][ROCm][MoE] Fix mxfp4 oracle regressions from #37128 (#37787) Andreas Karatzas 2026-03-24 19:17:33 -05:00
8bbb7c7f20 [ROCm][CI][PD] Add Hybrid SSM integration tests to CI (#37924) Andreas Karatzas 2026-03-24 18:58:39 -05:00
af945615b5 [release] Move the rest of release jobs to release queue (#38044) Kevin H. Luu 2026-03-24 16:40:58 -07:00
82580b10ac [Perf] Disable inductor runtime asserts by default for serving perfor… (#37485) Terry GaoandTianren Gao 2026-03-24 16:37:51 -07:00
a0d487b2e1 nano_nemotron_vl: suppress readonly torch.from_numpy() warning in image and video resize paths (#37903) Netanel Haber 2026-03-25 01:25:56 +02:00
b73b5b0629 Make microbatch optimization (DBO) work with general models (#37926) Junhao 2026-03-24 17:40:08 -04:00
0f0e03890e [UX] Add flashinfer-cubin as CUDA default dep (#37233) Michael Goin 2026-03-24 22:13:08 +01:00
4b53740d7f [MRV2] Fix for DS v3.2 (#38030) Woosuk Kwon 2026-03-24 14:03:24 -07:00
4e824d1c83 [Model Runner V2][Minor] Simplify PP logic (#38031) Nick Hill 2026-03-24 13:57:17 -07:00
0c1809c806 Add Ubuntu 24.04 support for Docker builds (#35386) amey asgaonkar 2026-03-24 13:34:44 -07:00
8c47fdfdb1 [FlexAttention] allow custom mask mod (#37692) liangel-02 2026-03-24 16:03:24 -04:00
54b0578ada [Bugfix] Pass hf_token through config loading paths for gated model support (#37920) Javier De Jesus 2026-03-24 20:22:05 +01:00
89f572dbc0 [BugFix] fix VLLM_USE_STANDALONE_COMPILE=0 (#38015) Richard Zou 2026-03-24 15:08:26 -04:00
71a4a2fbd0 [BugFix] Fix order of compile logging (#38012) Richard Zou 2026-03-24 14:58:18 -04:00
935c46dd9b [Model] Add Granite 4.0 1B speech to supported models (#38019) Nick Cao 2026-03-24 14:23:41 -04:00
057fc94cbd [Bugfix] Fix structured output crash on CPU due to pin_memory=True (#37706) Willy HardyandClaude Opus 4.6 2026-03-24 13:44:17 -04:00
b58c5f28aa docs: fix broken offline inference paths in documentation (#37998)
Vineeta Tiwari Vineeta Tiwarigemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-24 23:05:14 +05:30
c07e2ca6e0 Fix Mamba state corruption from referencing stale block table entries (#37728) (#37728) (#37728) Ming Yang 2026-03-24 10:29:59 -07:00
4df5fa7439 [Bugfix] Force continuous usage stats when CLI override is enabled (#37923)
Dhruv Singal Your Name OpenCode
2026-03-24 10:29:50 -07:00
a5416bc52e [XPU] Support Intel XPU hardware information collection in usage stats (#37964) sihao_li 2026-03-25 01:29:17 +08:00
b3601da6e7 [Mypy] Fix mypy for vllm/model_executor (except vllm/model_executor/layers) (#37904) Harry Mellor 2026-03-24 17:14:01 +00:00
dc78c2c933 [Core] add option to schedule requests based on full ISL (#37307) Dan BlanaruandClaude 2026-03-24 18:01:12 +01:00
4731884796 [Feature] limit thinking tokens (hard limit) (#20859)
Sungjae Lee Chauncey Harry Mellor
2026-03-25 01:53:07 +09:00
8de5261e69 Update new contributor message (#37999) Harry Mellor 2026-03-24 16:01:41 +00:00
1b6cb920e6 [Deprecate] Deprecate pooling multi task support. (#37956)
wang.yuqigemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
2026-03-24 22:07:47 +08:00
352b90c4a4 [Bugfix] Add replacement of _compute_slot_mapping_kernel on CPU (#37987) Li, Jiang 2026-03-24 22:00:20 +08:00
1c0aabdeb0 [Bugfix] Suppress spurious CPU KV cache warning in launch render (#37911) Sage 2026-03-24 14:36:18 +02:00
14acf429ac [EPLB] Remove main waits in case of slow EPLB (#36271) Ilya Markov 2026-03-24 12:50:44 +01:00
ce57fd5557 [Docs] Fix build (#37991) Harry Mellor 2026-03-24 10:20:49 +00:00
2e67fa756d Fix tool_parser_cls type annotation from Callable to type[ToolParser] (#37957) Flora Feng 2026-03-24 01:58:27 -04:00
e3c6c10cad [KV Offload] Refactor CPU offloading: pluggable CachePolicy, remove Backend abstraction, restructure into cpu/ package (#37874) Ronen Schaffer 2026-03-24 07:02:51 +02:00
16a664df24 [Frontend][Bugfix] Pass default_chat_template_kwargs to AnthropicServingMessages (#37899) jetxa 2026-03-24 13:00:12 +08:00
7281199a8c [release] Move agent queue to Release cluster queues (#37783) Kevin H. Luu 2026-03-23 20:36:47 -07:00
b2dd75eb48 Downsize CPU jobs to use small queue (#37913) Kevin H. Luu 2026-03-23 20:36:37 -07:00
c59a132f96 [V0 Deprecation] Refactor kv cache from list to element (#37487) Wentao Ye 2026-03-23 23:10:11 -04:00
de99d91ece [ROCm][CI] Split Entrypoints Integration (API Server 1) into 3 jobs (#37906) Andreas Karatzas 2026-03-23 20:48:37 -05:00
83c9d525b6 [CI] Add batch invariant test: Block FP8 + small MOE (#37895) Wentao Ye 2026-03-23 21:16:14 -04:00
8f4824b664 [Model Runner V2] Gather multimodal embeddings before draft model postprocess (#37932) Giancarlo Delfin 2026-03-23 18:14:13 -07:00
56777b5c89 [Test] E2E Nemotron-3-Super tests (#36803) roikoren755 2026-03-24 02:49:56 +02:00
2488a82f89 [CI] Split V1 Others into 3 separate jobs (#37016) Kevin H. LuuandClaude Opus 4.6 2026-03-23 15:44:38 -07:00
dc6908ac6a [Bugfix] Register VLLM_BATCH_INVARIANT in envs.py to fix spurious unknown env var warning (#35007) RanranandWentao Ye 2026-03-23 17:31:14 -05:00
e85f8f0932 [Bug][MoE] Strengthen _supports_current_device() checks in the TRTLLM FP8, NVFP4, and FlashInfer CuteDSL MoE experts (#36728) yzong-rh 2026-03-23 17:02:57 -04:00
5bf3c42d4c [Bug][MoE] Fix TRTLLM NVFP4 Routing Kernel Precision (#36725) Robert ShawandRobert Shaw 2026-03-23 21:19:06 +01:00
38364a7e32 [Sparse24] [Deprecation] Remove Sparse24 CT integration and kernels (#36799) Kyle Sayers 2026-03-23 16:03:29 -04:00
fafe76b4af [Async][Spec Decoding] Zero-bubble async scheduling + spec decoding (#32951)
Matthew Bonanni zhuhaoran zhrrr Lucas Wilkinson Benjamin Chislett
2026-03-23 15:37:22 -04:00
ffb5b32b5f [MRV2] Consider spec decoding in warmup (#37812) Woosuk KwonandNick Hill 2026-03-23 10:45:43 -07:00
91fd695b75 [CI] split Entrypoints Integration (API Server 1) into 3 jobs (#37882) Kunshang Ji 2026-03-24 01:37:56 +08:00
1cbbcfe8a3 [CI][PD] Add Hybrid SSM integration tests to CI (#37657) Nicolò Lucchesi 2026-03-23 16:58:19 +01:00
aceadb5ee1 Use lazy graph module during split_module to defer recompile() (#37609) Angela Yi 2026-03-23 08:21:29 -07:00
ec2280611a [Bugfix] Fix RoBERTa position_ids accumulation on CUDA graph padding (#37884) Yufeng He 2026-03-23 23:15:12 +08:00
7151ae6528 [Bugfix] RoBERTa position_id accumulation in CUDA graph padding region (#37873) yanghui1-arch 2026-03-23 22:59:21 +08:00
45bd5c8e75 [Mypy] Fix mypy for vllm/config (#37808) Wentao Ye 2026-03-23 10:33:59 -04:00
10a1018c12 [ROCm] fix sleep mode not releasing GPU memory problem on ROCm (#37533) Zhaodong BingandTJian 2026-03-23 21:07:19 +08:00
aec2dc6c0d [Bugfix][LoRA] Fix incorrect LoRA Log (#37877) Jee Jee Li 2026-03-23 19:42:52 +08:00
7938d12119 [Bugfix] Fix CPU backend crash in KV cache block zeroing (#37550) DorBernsohn 2026-03-23 13:35:45 +02:00

... 80 81 82 83 84 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86