Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

bc5fdc1e6a Add NVFP4 all-gather GEMM fusion for AsyncTP (#41882) baonudesifeizhaiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-09 21:13:22 -04:00
006af4b956 [Bugfix] Skip routed-experts hot path when disabled (#42148) aoshen02 2026-05-10 09:01:04 +08:00
ea0e501bb1 [KV Connector] Remove compat support for pre-v0.12.0 constructor signatures without KVCacheConfig (#39832) Wentao Ye 2026-05-09 19:39:46 -04:00
f80aa53c9d [Refactor] Nixl util using lazy init (#41392) Wentao Ye 2026-05-09 17:46:52 -04:00
7a2b596982 [Quantization] Add ModelOpt NVFP4 W4A16 (4-bit weights, fp16/bf16 activations) support (#41769) Juhi Mittal 2026-05-09 14:15:50 -07:00
2ee8c2a56e [SpecDecoding] extend mtp support for mimo 2.5 (#41905) Jiangyun Zhu 2026-05-10 02:22:59 +08:00
cd74911d92 [Model] use AutoWeightsLoader for DeepSeekV2 (#41706) SoluMilken 2026-05-10 01:55:25 +08:00
25abddc1a5 [BugFix] Fix Gemma4 'layers.0.moe.experts.0.down_proj_packed' KeyError issue (#40708) SoluMilken 2026-05-10 01:20:44 +08:00
171d59ae8d [Bugfix][PD] Fix DSv4 Disaggregated (#41957) Nicolò LucchesiandZhanqiuHu 2026-05-09 18:48:24 +02:00
3dda9aeb54 [Bugfix] Remove nested torch.compile in GDN rearrange_mixed_qkv causing CUDA graph capture failure (#42070) Thomas ParnellandJiangyun Zhu 2026-05-09 17:30:55 +02:00
adb6d96516 [Bugfix] Fix GDN KKT precision loss on Hopper GPUs by aligning tl.dot operand layout with WGMMA (#42076) Kermit 2026-05-09 21:08:46 +08:00
530d371302 [DSv4] Improved fused Indexer Q quant kernel (#41428) Thien Tran 2026-05-09 15:20:32 +07:00
34ab4f2565 [ROCm] Upgrade aiter to v0.1.13-rc5 (#42113) Micah Williamson 2026-05-09 03:13:45 -05:00
ecd0b60aad [LoRA] Initial EP support for LoRA (#40867) Jee Jee Li 2026-05-09 15:31:23 +08:00
d6563d693c Require C++20 for compatibility with PyTorch (#40380) Richard BarnesandClaude 2026-05-09 01:04:43 -04:00
f6490a2841 [Bugfix] Preserve leading/trailing whitespace in GLM non-streaming tool parser (#42026) Rishapveer Singh 2026-05-09 06:49:15 +02:00
a2812becd6 [Models] Cohere Eagle + fix to Cohere MoE (#42078) Terrence ZhaoandCursor 2026-05-09 00:46:26 -04:00
e8f9038ebd [ROCm][Bugfix] Re-tag AITER MoE weights as preshuffled after replace_parameter (#42061) Markus HartikainenandTJian 2026-05-09 07:42:07 +03:00
df2636a9d8 [Bugfix] Fix LOGITPROC_SOURCE_ENTRYPOINT test to use spawn-compatible dist-info registration for XPU/ROCm (#42040)
David Zhenggemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Andreas Karatzas Kunshang Ji
2026-05-08 21:32:04 -07:00
97cc7685c4 Add @Harry-Chen in CODEOWNERS (#42130) Shengqi Chen 2026-05-09 12:08:22 +08:00
e934e459e6 [CI][Bugfix] Make test_gpt2_cache_hit observable across V1 EngineCore (#42037) haosdent 2026-05-09 11:53:15 +08:00
845ca327ce [Bugfix] Fix test_whisper distributed test process handling (#42038) David Zheng 2026-05-08 20:37:21 -07:00
4f6fa6341d [XPU] update supported models on XPU (#41911) Yan MaandKunshang Ji 2026-05-09 10:44:03 +08:00
a43bc34baf [Docs] Update server entrypoint examples (#42077) Ethan Feng 2026-05-09 10:03:52 +08:00
236bf9d152 [Docs] Fix RLHF example links (#42073) Ethan Feng 2026-05-09 10:03:42 +08:00
b1728c1e66 [Attention][Cleanup] Remove tree attention (#42121) Lucas Wilkinson 2026-05-08 21:36:19 -04:00
be0dcc29dc [XPU] remove q/k/v force contiguous for flash_attn (#40356) Yan MaandKunshang Ji 2026-05-09 09:19:05 +08:00
e3b65a5ba0 [feat] Add explicit /start_weight_update and /finish_weight_update APIs for weight transfer (#39212) Sumanth R Hegde 2026-05-08 18:03:33 -07:00
30f519e947 Use pre-commit / pre-run-check to gate docs build too (#42053) Harry Mellor 2026-05-09 01:02:51 +01:00
60851b1d22 [Bugfix][KV Transfer] Reject NixlConnector + expandable_segments:True (#41237) Roy Wang 2026-05-09 07:47:33 +08:00
8bcd8a260c [Bugfix] Fix FlashInfer CUTLASS MXFP4-MXFP8 MoE by restoring swizzled scale (#42089) Michael Goin 2026-05-08 18:59:06 -04:00
8a2fc80b84 [CUDA][CUTLASS] Enable cutlass scaled mm for non-compatible sizes (#41868) John Calderon 2026-05-08 18:58:05 -04:00
6881c754e1 use HIP_VERSION variables to guard against duplicate atomicAdd definitions (#41802) pmaybankandTJian 2026-05-08 18:44:37 -04:00
0c2e9d4892 [CI] Narrow misc.yaml source dependencies (#42059) Kevin H. Luu 2026-05-08 15:10:12 -07:00
d2f22dfc9f [CI] Narrow engine.yaml source dependencies (#42055) Kevin H. Luu 2026-05-08 14:55:33 -07:00
f4dd5c116c [CI] Narrow Platform Tests (CUDA) source dependencies (#42054) Kevin H. Luu 2026-05-08 14:54:06 -07:00
f47ccc8b1c [CI] Narrow pytorch.yaml compile job source dependencies (#42057) Kevin H. Luu 2026-05-08 14:43:17 -07:00
dbd86a67e3 [Bugfix][Gemma4] Fix infinite loop and array boundary issues in tool parser (#41991) David OyandClaude 2026-05-08 14:24:37 -07:00
2c6b59b807 [ROCm][Perf] Add Fused Shared Expert (FSE) support for Qwen3-Next (#39280)
Nico Holmberg nholmber Tres Tres Popp Doug Lehr Cursor Douglas Lehr
2026-05-08 12:38:00 -07:00
44e6b44a21 [CI][Elastic EP] Fix Elastic EP Scaling Test Failure (#41792) haosdentandNicolò Lucchesi 2026-05-09 03:17:44 +08:00
90f145aaf7 [Models][Gemma3/Gemma4] Support hidden_act variants in gated MLP (#40588) Hiroaki Mikami 2026-05-09 03:29:11 +09:00
4140faa4a5 [Docs] Fix OpenAI batch model argument examples (#42066) Ethan Feng 2026-05-08 22:02:46 +08:00
f2bbd575e2 [CI][XPU] Skip fork-dependent logits processor test (#42013) liuzhenwei 2026-05-08 21:10:19 +08:00
52458b60a8 [CI][Examples][RLHF] Disable async scheduling in rlhf_async_new_apis (#42042) haosdent 2026-05-08 19:58:48 +08:00
630820a59b Make docs environment deterministic (#41926) Harry Mellor 2026-05-08 11:13:03 +01:00
19df11f5d1 [CI][XPU]Ignore some lora tests from LoRA Intel CI pipeline (#42010) Chaojun Zhang 2026-05-08 17:34:27 +08:00
36b2c79d4b [CI][Bugfix] Drop duplicated examples/ prefix in tensorize_vllm_model command (#42039) haosdent 2026-05-08 17:23:22 +08:00
160858cba4 [CI][Bugfix] Surface subprocess output in spawn_new_process_for_each_test (#41943) haosdent 2026-05-08 16:39:37 +08:00
f9b9bf3bbb [CI][ROCm] Ship RIXL with vllm/vllm-openai-rocm (#41634) Simon Danielsson 2026-05-08 09:05:17 +02:00
445d747434 [Bugifx] Missing Renderer for fastokens mode (#41984) TJianandCyrus Leung 2026-05-08 14:45:14 +08:00
77b13b9602 [Docs] Reorganize examples docs. (#41082) wang.yuqi 2026-05-08 14:23:44 +08:00
ed582b6a4c [Aiter][ROCm] gdn_linear_attn kernel fusion (#40711)
Tres hellozhuo Cursor
2026-05-08 08:11:37 +02:00
1acd67a795 [Bugfix] Fix XPU/ROCm compatibility in spawn_new_process_for_each_test (#41895) David Zheng 2026-05-07 21:47:22 -07:00
0b99971352 [Kernel][Helion] Optimize Helion config parsing latency (#40850) Yanan CaoandClaude Sonnet 4 2026-05-07 20:27:34 -07:00
baf068d8be enable persistent mla for sparse mla backend (#41990) Douglas Lehrandganyi 2026-05-07 22:10:50 -05:00
01b0f3adab fix: default TILELANG_CLEANUP_TEMP_FILES=1 to avoid shared /tmp conflicts (#41486) SamareshSingh 2026-05-07 21:59:00 -05:00
989c176c0a [Perf][3/n] Eliminate GPU<->CPU syncs in attention impls (#41434) Nick Hill 2026-05-07 19:44:24 -07:00
cd58e30872 [Perf] Use numpy zero-copy path for embedding float response serialization (#41681)
Shrinav Loka Claude Opus 4.6 wang.yuqimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-07 19:42:21 -07:00
1d694e78c9 [Examples][last/6] Resettle examples. (#41084) wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-08 10:42:12 +08:00
57c2f724c1 [CI][Bugfix] Fix CI failures for "PyTorch Compilation Unit Tests" (#41940) haosdent 2026-05-08 10:42:00 +08:00
5f6a02812a [CI][Bugfix] Fix failure CI step "PyTorch Fullgraph Smoke Test" (#41953) haosdentandRoger Wang 2026-05-08 10:41:56 +08:00
50f2db2555 add: LFM2/2.5 Tool Parser (#39243) JonathanandChauncey 2026-05-07 18:58:17 -07:00
09a7cc5ba9 [KV Connector] Opt DecodeBenchConnector into SupportsHMA (#41770) Zijing LiuandClaude Opus 4.7 2026-05-07 16:10:00 -07:00
10ebb40d62 [Core] Avoid using extra thread in UniProcExecutor (#40891) Nick Hill 2026-05-07 15:33:00 -07:00
54f548e9e5 [Bugfix] Restore moe_forward output shape invariant on TRTLLM MXFP4 path (#41646) Stefano CastagnettaandYongye Zhu 2026-05-08 00:26:06 +02:00
c1819ca283 [Compressed Tensors] Allow configs with non-explicit ignores (#41965) Kyle Sayers 2026-05-07 17:03:45 -04:00
969fbfb4a9 Laguna xs dflash support (#41880) MeganEFlynnandClaude Opus 4.6 2026-05-07 13:31:16 -07:00
3af561ec0a [ROCm] Fix AITER AR+RMSNorm no-residual fusion (#41972) akii96 2026-05-07 23:14:58 +03:00
c936548ce6 [ROCm][DeepSeek] Enable V3.2 TP4 AITER MLA (#41835) akii96 2026-05-07 23:10:57 +03:00
8189a15914 [Core] Replace routing replay with device cache and async D2H pipeline (#39917) TomerBN-Nvidia 2026-05-07 21:24:56 +03:00
8eb401134e [Refactor] Consolidate required/named tool_choice streaming into DelegatingParser (#41876) Flora Feng 2026-05-07 12:50:59 -04:00
9d6500b89d [Misc] Delay EPLB Nixl import until needed (#41805) Nicolò Lucchesi 2026-05-07 18:43:07 +02:00
7a08b34fbf [Model Runner V2] support qwen35 / mamba hybrid model (#35520) zhrrr 2026-05-08 00:31:05 +08:00
06a60d3dd0 Fix spec decode benchmark metrics (#41916) noobHappylife 2026-05-08 00:23:21 +08:00
2a16ece2d3 tokenizer: Add fastokens support (#41741) AlonKejzmanandRoger Wang 2026-05-07 17:49:42 +03:00
003159d98b [ROCm][CI] Avoid duplicate ROCm AITER norm-quant patterns (#41534) Andreas Karatzas 2026-05-07 08:33:30 -05:00
2a84da3b17 [XPU] Implement out-of-place all-reduce functionality (#41808) Chaojun Zhangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-07 20:58:05 +08:00
805e9f7b77 [XPU] Fix lora bugs & enable UTs under tests/lora (#38206) Chaojun Zhang 2026-05-07 20:58:00 +08:00
75f0d516c4 [Bugfix] Fix GLM4-MoE weight loading for NVFP4 quantized checkpoints (#41755) s-yanev 2026-05-07 15:55:52 +03:00
f650ace6de [MM][Gemma4] Use video profiling hints in encoder budget (#41837) SeongJun Leeandlesj0610 2026-05-07 21:46:04 +09:00
b3945cc316 [CPU] Bump up to the latest CPU kernels (#41924) Li, Jiang 2026-05-07 20:45:59 +08:00
ffee741626 [Model] Use AutoWeightsLoader for AXK1 (#41901) wenyiliandClaude 2026-05-07 20:40:29 +08:00
9c0812ffd0 [Bugfix] Fix FusedMoEWithLoRA has no attribute runner (#41889) Jee Jee Li 2026-05-07 19:53:14 +08:00
b20731d0ae [CI][Arm] skip e2e model tests if HF_TOKEN is not set (#41919) Fadi Arafeh 2026-05-07 12:31:50 +01:00
d4b0048404 Eliminate redundant MoE buffer copies in AITER fused experts (without dependency on AITER changes) (#41713) Mehdi Ghanimifardandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-07 12:46:40 +02:00
6e6d182d18 [Bugfix] Fix OOM in tensorizer LoRA deserialization (#41845) Or OzeriandClaude Opus 4.6 2026-05-07 12:17:47 +03:00
8a4888be21 [ROCm] Profiler api support for ROCm MORI toy proxy server in PD Disaggregation (#40264) tej 2026-05-07 03:58:38 -05:00
713b28bd0b [CPU] Add FP8 W8A16 MoE support (#41314) Yuwen Zhou 2026-05-07 14:17:07 +08:00
51f22dcfd0 [Feat][CPU] Enable Gated DeltaNet Attention (Qwen 3.5 / 3.6) (#41025) Fadi ArafehandVadim Gimpelson 2026-05-07 05:57:09 +01:00
20cac26b19 [ROCm] Enable SimpleCPUOffloadConnector on ROCm (#40549) Hongxia YangandTJian 2026-05-06 23:52:02 -04:00
5a0a8fc1ea [Docs] add cache directory security guidance (#38920) Russell Bryant 2026-05-06 19:54:29 -04:00
7a576e2c72 [ROCm][CI] Remove TORCH_NCCL_BLOCKING_WAIT=1 After Bugfix In ROCm 7.2 (#41840) Micah Williamson 2026-05-06 18:37:11 -05:00
80d5e7d103 [Bugfix] Fix condition to clear persistent topk so that it can be captured regardless (#41665) Yongye Zhu 2026-05-06 19:17:48 -04:00
95582868ef [Bugfix] DeepSeekV32/v4: respect string='true|false' attribute andunwrap arguments/input wrapper (#41801) ChaunceyandQwertyJack 2026-05-07 05:48:01 +08:00
50acdc5b5c Fix Qwen3 streaming content routing (#40820) xy3andsfeng33 2026-05-07 05:22:01 +08:00
deb737e323 [Doc] Add ModernBertForSequenceClassification to scoring.md cross-en… (#41832) JackyLiu 2026-05-06 17:17:56 -04:00
f3f8efa73a [CI] Enable gemma4 parser test on CI (#41857) Flora Feng 2026-05-06 16:25:34 -04:00
ca3e62d336 Upgrade tpu-inference to v0.19.0 (#41844) Johnny Yang 2026-05-06 11:41:37 -07:00
38e16678ba [Bugfix] Align block table for TRTLLM MLA edge-case (#39324) Benjamin Chislett 2026-05-06 14:17:02 -04:00
27702f6d08 [Bugfix] Fix token loss in PP mode which causes degraded accuracy (#41133)
Jing Wang Copilot Wentao Yemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-07 02:07:32 +08:00

... 68 69 70 71 72 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86