This website requires JavaScript.
f0cd590d62
optimize the compressor 128 split cutedsl kernel (#44230 )
Jie Fang and Yongye Zhu
2026-06-04 11:22:57 +08:00
e6018c644a
[Refactor] Remove dead code in tests and parallel_state (#41471 )
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-03 22:32:39 -04:00
f25952e59b
[MM][Perf][CG] Support ViT full CUDA graph for InternVL (#41759 )
Oğuzhan KIR and Isotr0py
2026-06-04 05:24:25 +03:00
b58e082d95
[KV Connector] Update lmcache kv_offloading_backend to use LMCacheMPConnector (#42865 )
maobaolong
2026-06-04 10:23:55 +08:00
0c1e6f63f5
[Bugfix] Fix VLLMNotFoundError when using LoRA adapter name in poolin… (#44410 )
Ted Mostly
2026-06-04 10:22:03 +08:00
ceb0111a90
[Model Runner V2][Spec Decode] Add Gemma4 MTP support (#43241 )
Giancarlo Delfin
2026-06-03 19:51:06 -05:00
0414d75410
[XPU] skip unapplied UT in test_gpu_model_runner.py (#44289 )
Yan Ma and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-04 08:48:17 +08:00
128adabfe0
[Bugfix] Fix Gemma4 MTP block_table batch_size mismatch under concurrent load (#43982 )
Dima and Dmytro Kuntso
2026-06-04 01:11:10 +01:00
bdbf08fc02
Bump actions/stale from 10.1.1 to 10.2.0 (#35078 )
dependabot[bot] and dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-03 14:14:41 -07:00
6bad553f4e
[Minor] Remove FlashInfer version check in topk_topp_sampler (#44442 )
Woosuk Kwon
2026-06-03 14:06:00 -07:00
91945b6e4a
[Bug Fix][Model Runner V2][Spec Decode] Warmup & capture with different attention states for speculator prefill (#44253 )
Giancarlo Delfin and Woosuk Kwon
2026-06-03 15:32:40 -05:00
2b237c7a41
[Bugfix] Honor tool_choice="none" in Chat Completions streaming (#42752 )
hoobnn and sfeng33
2026-06-04 04:27:45 +08:00
dad95e34d8
[Feature] Support batch invariant rms norm with residual (#42453 )
Wentao Ye
2026-06-03 15:22:01 -04:00
a248b45d05
[Model] Add Gemma4 Unified (encoder-free) support (#44429 )
Luciano Martins and Luciano Martins
2026-06-03 16:01:39 -03:00
271328e256
[LoRA] Fix dedup for post-replacement module aliases (#44413 )
linitra24
2026-06-04 02:23:23 +08:00
2b91012650
[Refactor] Remove dead code fp quant (#44122 )
Wentao Ye
2026-06-03 14:22:23 -04:00
5b2a2beade
[ROCm][CI] Move Model Executor test step from MI250 to MI300 (gfx942) (#44370 )
JartX
2026-06-03 19:23:51 +02:00
59d0236193
[10b/n] Migrate custom all-reduce, DeepSeek V4 fused MLA, MiniMax reduce-RMS, and MXFP8 MoE to libtorch stable ABI (#44365 )
Chris Leonard and Shengqi Chen
2026-06-03 12:29:46 -04:00
0a5cbf633e
Handle spinloop ext load failure gracefully (#43659 )
pschlan-amd and Shengqi Chen
2026-06-03 18:09:52 +02:00
51e0c579b0
fix(config): validate max_num_scheduled_tokens >= 0 on all paths (#44207 )
Willow Lopez
2026-06-04 00:06:45 +08:00
0c6631f02a
[KVCache] Support Pluggable KVCacheSpec (#37505 )
2026-06-04 00:05:16 +08:00
df7252c343
[CI] Align PD tests to HMA on by default (#44174 )
Nicolò Lucchesi
2026-06-03 18:04:30 +02:00
4d1fd13613
[CI/Build] Fix LoRA testing (#44425 )
Jee Jee Li
2026-06-03 23:58:06 +08:00
ec8d60bea8
[Model Runner V2] Use FlashInfer sampler (#42472 )
Nick Hill
2026-06-03 07:59:31 -07:00
27f1d34a23
[Frontend][Responses API] Move developer-to-system conversion into HF renderer (#43590 )
2026-06-03 22:52:24 +08:00
e3e132d2dd
[Refactor] Suppress SyntaxWarning from ast.literal_eval in tool parsers (#44346 )
Flora Feng
2026-06-03 10:42:19 -04:00
e5232679a3
[XPU] Add XPU block-scaled W8A8 fp8 path (#39968 )
2026-06-03 20:16:19 +08:00
309385a359
[Rust Frontend] Add /server_info to Rust frontend (#43942 )
Xunzhuo and Bugen Zhao
2026-06-03 19:30:47 +08:00
3d76f395e3
[SharedOffloadRegion] Align blocks to page-size (#43689 )
Varun Sundar Rabindranath and varun sundar rabindranath
2026-06-03 07:25:57 -04:00
823d271c0d
[Attention][CPU] Standardize kv layout to blocks first (#44393 )
Li, Jiang
2026-06-03 19:03:09 +08:00
95b1615ec9
[Perf] Improve multimodal item handling from O(n) to O(log n) per step (#44212 )
Andy Lo
2026-06-03 12:00:26 +01:00
1fa9ea09f6
[Perf] Triton fast path for small CPU→GPU swap_blocks_batch in the offloading connector (#42212 )
2026-06-03 13:38:17 +03:00
02564b4de0
[XPU]fallback to TRITON_ATTN for vit attn on xpu when use float32 dtype (#43759 )
Yan Ma and Kunshang Ji
2026-06-03 18:20:21 +08:00
209709a8c1
[Bugfix] Fix unstreamed tool call args dropped in Responses API streaming (#44348 )
Flora Feng
2026-06-03 06:19:08 -04:00
ace95c9cf8
[Bugfix] Update TrtLLM MoE routing methods (#44347 )
Wei Zhao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-03 05:56:43 -04:00
0e2b13103b
[Doc] Update ViT CUDA graph interfaces (#44388 )
Shanshan Shen
2026-06-03 16:20:59 +08:00
449be4f934
[Rust Frontend] Fix several hf chat template rendering issues (#44311 )
Bugen Zhao
2026-06-03 16:04:43 +08:00
6550ff12f2
[Rust Frontend] Add dynamic LoRA endpoints (#43778 )
Xunzhuo and Bugen Zhao
2026-06-03 15:55:29 +08:00
4aaed4ca22
[Rust Frontend] Add server router extension hook (#43774 )
2026-06-03 15:45:31 +08:00
7268457999
[KV Offloading] Enable HMA models for Tiering Offloading (#44287 )
Varun Sundar Rabindranath and varun sundar rabindranath
2026-06-03 03:03:00 -04:00
9af53a3c13
[Perf] Add tuned selective_state_update configs for H200 and RTX PRO … (#44251 )
2026-06-02 23:59:01 -07:00
87954eb50e
[ROCm][CI] Optimize ROCm Docker build: registry cache, DeepEP, and ci-bake script (#36949 )
Andreas Karatzas
2026-06-03 01:43:07 -05:00
71df063c49
Enable perf_token_group_quant/_C_stable_libtorch for ROCm (#42758 )
Charlie Fu
2026-06-03 01:23:28 -05:00
e0081ef8cf
[Benchmark] Enable reasoning-model (thinking) benchmarking via --chat-template-kwargs for client-rendered datasets (#44244 )
Albert Cheng
2026-06-02 22:49:51 -07:00
f0204358d9
[Bugfix] fix crash in postprocess for null tool args (#43862 )
William Rom and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-03 07:17:26 +02:00
597bc15936
fix: resolve CUTLASS fmin compatibility for DeepSeek-V4 init (#44236 )
Willow Lopez
2026-06-03 13:07:10 +08:00
3f0a91bb96
Nit Changes in Tiered KV Offload (#44293 )
Rotem Shavitt
2026-06-03 07:53:21 +03:00
e67063826b
[CI] Add missing vllm/parser/ CI trigger and fix test_parse.py (#44352 )
Flora Feng
2026-06-03 00:05:19 -04:00
53b88d1dfc
[CI] Reject out-of-vocabulary before they reach the GPU logprob path (#44042 )
Andreas Karatzas
2026-06-02 22:27:52 -05:00
7b476c8f14
[ROCm][CI] Skip fp8 reload tests on gfx90a (MI250) (#44369 )
JartX
2026-06-03 05:27:14 +02:00
4454a18695
[ROCm][CI] Fix stale wvSplitK GEMM fallback test for N=5 (#44368 )
JartX
2026-06-03 05:00:25 +02:00
02a01496fc
[Platform] Add is_cumem_allocator_available (#43838 )
wangxiyuan and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-03 10:54:50 +08:00
27a93cd426
[docker] Stop using extra-index-url for flashinfer-jit-cache (#44366 )
Kevin H. Luu
2026-06-02 18:58:22 -07:00
969aec4bc8
[Bugfix] Fix Deepseek v4 non-mega-moe model init error (#44356 )
Wei Zhao
2026-06-02 21:26:30 -04:00
ca17b6b17d
[Perf] Apply single-pass min_larger finding and binary search in Triton Top-p path. (#42191 )
Jongseok Park and Nick Hill
2026-06-02 17:57:26 -07:00
b254e0456c
[DSV4] Minor cleanup for DeepseekV4MegaMoEExperts (#44367 )
Woosuk Kwon
2026-06-02 17:54:27 -07:00
bd98e97557
[Misc] Remove dead VLLM_RPC_TIMEOUT env var and fix profiling doc that references it (#44128 )
Daoyuan Li
2026-06-02 17:22:10 -07:00
a4ac746405
[MoE/b12x] Accept W4A16 (kNvfp4Static, None) in FlashInferB12xExperts supports check (#43332 )
Junhao Shen and Vadim Gimpelson
2026-06-03 06:20:37 +08:00
8b3b71ee9d
[CI/Build] Bump flashinfer to v0.6.12 (#44036 )
Vadim Gimpelson and Jee Jee Li
2026-06-03 02:19:05 +04:00
0917a009d3
Fix sparse NCCL weight transfer test construction (#44345 )
Siddharth Bedekar
2026-06-02 14:51:21 -07:00
3099de3617
[Kernel][MoE] Add GELU_TANH to CPU, CUTLASS, and WNA16 MoE backends (#42027 )
SeongJun Lee and lesj0610
2026-06-03 06:12:08 +09:00
e15f20258b
[ModelRunnerV2] Avoid pipeline parallel bubbles (#42187 )
Nick Hill
2026-06-02 14:02:01 -07:00
557781131a
[Misc] Remove stray empty file (#44350 )
Matthew Bonanni and Claude
2026-06-02 15:53:03 -04:00
e9e08c49b9
[Bugfix] Cache the EAGLE/MTP lookahead block in the SWA prefix-cache mask (#44082 )
Yifan Qiao
2026-06-02 12:21:07 -07:00
e4a2e584e5
[MRV2] Remove assignment of graph_pool in cudagraph_utils (#44338 )
Woosuk Kwon
2026-06-02 11:50:27 -07:00
b8b49e2395
Bump actions/github-script from 8.0.0 to 9.0.0 (#39667 )
dependabot[bot] and dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-02 11:26:57 -07:00
da107a59e5
[MRV2] Also enable MRV2 for Llama and Mistral dense models (#43458 )
Nick Hill and yewentao256
2026-06-02 11:18:46 -07:00
ed9a7526b6
[Anthropic] Support system role messages inside messages array (#44283 )
2026-06-03 02:13:54 +08:00
2427094152
[Feature] Support EPLB for DeepSeek v4 Mega Moe (#43339 )
Wei Zhao and Wei Zhao
2026-06-02 13:56:44 -04:00
fe32e7830b
[Bugfix] flashinfer: fail fast when --kv-cache-dtype nvfp4 used on unsupported arch (#43669 )
Kartavya sonar
2026-06-02 23:20:00 +05:30
afcb580715
[BugFix] Fix Humming MoE deploy error (#43100 )
2026-06-02 20:02:50 +03:30
3f3e2702c2
[XPU] Enable rms_norm/act quant fusions (#43963 )
liuzhenwei and Kunshang Ji
2026-06-03 00:14:41 +08:00
478b49ddec
[Refactor] Remove dead code from parser infrastructure (#44279 )
Flora Feng
2026-06-02 12:08:27 -04:00
cab5c9a2a9
[Core] Move max_concurrent_batches to VllmConfig (#44274 )
Nick Hill
2026-06-02 08:57:25 -07:00
774e552397
[compressed-tensors] Asymmetric support for MoE WNA16 marlin (#44025 )
Brian Dellabetta
2026-06-02 11:51:45 -04:00
53fa09d085
[Misc] Support local image encoding in benchmarks (#43843 )
XiaoZ
2026-06-02 23:15:06 +08:00
4d93bc35c9
Migrate header files to torch stable abi (#44013 )
Chris Leonard
2026-06-02 11:09:52 -04:00
586201ebdc
[Rust Frontend] Cover different thinking modes in roundtrip tests (#44320 )
Bugen Zhao
2026-06-02 22:51:25 +08:00
88f172188b
[ROCm] Fix AITER RMSNormQuantFusion for Kimi-Linear (#44308 )
pschlan-amd
2026-06-02 16:50:21 +02:00
880fc032f4
[Rust Frontend] Support recursive tool parameter conversion (#44299 )
Bugen Zhao
2026-06-02 22:45:35 +08:00
6314de8bad
[XPU] [Bug] remove xpuw4a16 output size check (#44168 )
zofia and Kunshang Ji
2026-06-02 22:26:20 +08:00
c91a87f01a
[BugFix] [GDN] Read linear_key_head_dim from hf_text_config for multimodal models (#43978 )
IdoAtadTD
2026-06-02 16:17:55 +02:00
ea0d045a05
[FlashAttention] Sync FA with upstream (#44065 )
Matthew Bonanni
2026-06-02 10:15:37 -04:00
0bdfd5eb84
[Bugfix] Vendor MiniCPMV/MiniCPMO processors to unblock Transformers v5 (#44282 )
2026-06-02 22:14:38 +08:00
0cbc48c4f9
Support ModelOpt MXFP8 non-gated MoE (#42958 )
TomerBN-Nvidia and Michael Goin
2026-06-02 16:56:03 +03:00
2fd0e52252
[Bugfix] Fix Gemma4 startup crash with recent transformers multimodal processor (#44232 )
Luciano Martins and Luciano Martins
2026-06-02 10:42:40 -03:00
654bd2bca4
[Bugfix] Sync block_size from EngineCore to frontend for hybrid Mamba… (#42967 )
2026-06-02 16:41:00 +03:00
b623f7ea95
[Frontend] Consolidate dev entrypoints. (#44170 )
wang.yuqi
2026-06-02 21:30:21 +08:00
0eeba5eec1
Fix DFlash prefix cache corruption due to missing lookahead block (#42971 )
Shreyas Kulkarni
2026-06-02 08:06:33 -04:00
f69ede495b
[XPU][Mamba] Triton-based selective scan forward op for XPU (#43421 )
Marceli Fylcek and Kunshang Ji
2026-06-02 13:50:26 +03:00
2a2b5ca791
[KV Offload] Add on_schedule_end() hook to separate step lifecycle from event draining (#44206 )
Ronen Schaffer
2026-06-02 13:42:52 +03:00
689b0eeb9e
[HARDWARE][POWER] Enable SHM communicator support for PowerPC (#43754 )
2026-06-02 15:36:32 +05:30
f8e9c56d15
[Multimodal] Automatically select registered video loader for VLM (#44126 )
Isotr0py
2026-06-02 17:09:47 +08:00
e30313220c
[Parser] Migrate ResponsesParser to unified Parser interface (#42977 )
alberto
2026-06-02 09:50:05 +01:00
d247a9dc13
[EC Connector] Non blocking EC Connector lookup (#41627 )
omerpaz95 and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-02 11:48:25 +03:00
7c37096620
[Core][Refactor]: thread scheduler_block_size into KVCacheManager and KVCacheCoordinator (#44165 )
Yifan Qiao
2026-06-02 01:14:44 -07:00
b817b23f7b
[Rust Frontend] add --enable-request-id-headers flag support. (#43883 )
Maria Guevara
2026-06-02 09:08:37 +01:00
93da882e73
[kv_offload] Add @override decorators to subclass method implementations (#44177 )
Ronen Schaffer
2026-06-02 11:07:47 +03:00
0b25cf4419
[CPU][Perf] Enable fused kernels for GDN's gated delta rules (#43534 )
Fadi Arafeh and Li, Jiang <jiang1.li@intel.com>
2026-06-02 09:00:48 +01:00
dcdfe66bfa
[Perf] use triton moe backend on hopper by default (#44220 )
Jiangyun Zhu
2026-06-02 15:52:30 +08:00
