Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

69b8956dcd [Model Refactoring] Remove unncessary torch op registration for DSv4 (#43891) Woosuk Kwon 2026-05-28 14:04:55 -07:00
a3ed5ab10c [KV Offload] Add per-request offloading policy via on_new_request lifecycle hook (#43205)
Ronen Schaffer Or Ozerimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-28 23:45:18 +03:00
7e53283b1c [Core] Cleanup KVConnector handling with PP + fix MRV2 (#43732) Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-28 13:12:03 -07:00
9090368b65 [Feat] Add support for per GPU worker RDMA NIC selection (#42083) Raj JoshiandCursor 2026-05-28 15:45:23 -04:00
085ac221a3 Deprecate JAISLMHeadModel (#43784) Harry Mellor 2026-05-28 19:29:12 +01:00
9006204e90 [MM][CG] Avoid over-padding Qwen2.5-VL encoder cudagraph window metadata (#42796) Hua Huang 2026-05-28 11:22:56 -07:00
ed7fe831da [ROCm] Enable the aiter top-k/top-p sampler by default (#43331) JohnQinAMDandTJian 2026-05-28 11:19:59 -07:00
5b115bb8a3 [Attention][AMD] Standardize kv layout to blocks first for AMD (#43660) Nicolò Lucchesi 2026-05-28 19:28:50 +02:00
53a2088675 Allow native KV cache dtype in Triton cache update (#43330) Mike GandMichael Gschwind 2026-05-28 09:51:40 -07:00
099024762c [Rust Frontend] Optimize multimodal prompt expansion (#43670) Chao-Ju Chen 2026-05-29 00:46:18 +08:00
9aa131f944 Add Cosmos3 Reasoner model (#43356)
MaciejBalaNV Isotr0py Isotr0pymergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Roger Wang
2026-05-28 18:43:55 +02:00
1b5437cec8 [ROCm] Bump ROCm to 7.2.3 (#43136) Micah Williamson 2026-05-28 11:42:43 -05:00
3207e7680e [XPU][MoE] Add WNA16 oracle backend for GPTQ sym-int4 (xpu_fused_moe) (#41426)
Jason Elie Bou Kheir Claude Opus 4.7 Kunshang Ji
2026-05-28 09:30:48 -07:00
a9ec46d4b7 [ROCm][Perf] Support N=5 in wvSplitK skinny GEMM kernels for speculative decoding (#40687) Matthias Gehre 2026-05-28 18:28:21 +02:00
4bfa0f2b14 [KV Offload] Rename SecondaryTierManager.get_finished() to get_finished_jobs() (#43870) Ronen Schaffer 2026-05-28 19:00:18 +03:00
5d126dd155 [Bugfix] Exclude Ray DP from #42585's deferred port allocation (#43864) Vadim Gimpelson 2026-05-28 19:55:14 +04:00
c08ebebf30 [Perf] Add do_not_specialize to Mamba SSD chunk kernels (#43803) MajidandMajid Taheri Andani 2026-05-28 08:40:02 -07:00
be4062fd6c [Bug] Fix tests/distributed/test_elastic_ep.py - assert False (#43813) Wentao Ye 2026-05-28 11:00:56 -04:00
577d693838 [rust] fix: aggregate is_sleeping and reset_prefix_cache across DP engines (#43429) Will.houandClaude Opus 4.7 2026-05-28 22:56:56 +08:00
61a1e30473 [Rust Frontend] Reduce Gemma4 tool parser args scan complexity (#43850) Bugen Zhao 2026-05-28 22:52:29 +08:00
3a282230ee [Rust Frontend] Add hy_v3 tool parser (#43872) Bugen Zhao 2026-05-28 22:42:47 +08:00
20d69d100a [CPU] Migrate cpu_awq into awq_marlin (#43841) Li, Jiang 2026-05-28 22:36:31 +08:00
552eb81918 [Bugfix][ROCm] Resolve MoRI connector hangs at high concurrency (#40344) Simon Danielsson 2026-05-28 16:30:21 +02:00
9957e4d240 [Model Refactoring] Remove torch compile dependency in DSv4 (#43746) Woosuk Kwon 2026-05-28 07:26:25 -07:00
864990e8d9 Add token-offset based selective offload in OffloadConnector (#39983) Angelo RuoccoandOr Ozeri 2026-05-28 16:11:02 +02:00
f3b2a819f7 [Perf][KDA] Fuse gate softplus, chunk-local cumsum, and RCP_LN2 scaling (#43667) zexplorerhjandhaojiangzheng 2026-05-28 21:47:08 +08:00
64e1218673 [Perf] Optimize moe permute by pre-allocate buffer, 9~14% kernel performance improvement (#43014) Wentao Ye 2026-05-28 09:18:26 -04:00
02606b0b09 [BUGFIX] Multimodal benchmark with MistralTokenizer (#42965) Julien Denize 2026-05-28 14:36:24 +02:00
19af4e6dd4 Fix OlmoHybridForCausalLM not initialising (#43846) Harry Mellorandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-28 13:33:31 +01:00
811d805195 [EC Connector] Add shutdown API to EC Connector. (#42423) omerpaz95 2026-05-28 15:28:01 +03:00
c1c4db8b4b Log dummy DP step in iteration details (#41406) Vadim Gimpelson 2026-05-28 16:18:39 +04:00
d692b89c2c [Feature] Add structured output and effort support to Anthropic Messages API (#42396) Chauncey 2026-05-28 20:06:48 +08:00
8e0580f4ee [CI] Auto-apply rust label to relevant PRs (#43866) Bugen Zhao 2026-05-28 19:57:22 +08:00
61288b5458 [Bugfix] Fix HyperCLOVAX CI failure after upstream removed remote code (#43860) Kevin H. LuuandClaude Opus 4.6 2026-05-28 03:37:36 -07:00
a583c84e2b [Bugfix][ROCm] Fix Accuracy Drop in Sparse Indexer on gfx950 (#43781)
kliuae tjtanaa vllmellm
2026-05-28 18:37:15 +08:00
4ec2817313 [Model][Bugfix] Rename weight_mapper to hf_to_vllm_mapper in LlamaNemotronVL pooling models (#43581)
Jakub Zakrzewski opencode tomeras91
2026-05-28 12:32:22 +02:00
f2caefe226 [UX] Increase DP Coordinator startup timeout from 30s to 120s (#42343) Wei Zhao 2026-05-28 06:31:45 -04:00
bfb9ebc211 [Feature] Add support for timed trace replay in vllm bench serve to replay Moonshot and Alibaba workload traces (#39795) Animesh Trivedi 2026-05-28 12:31:34 +02:00
a9bc0ad8e4 [ROCm][CI] Move workload from MI300 to MI325 (#43824) Andreas Karatzas 2026-05-28 05:31:29 -05:00
b372ad3e90 [Bugfix] Stream DeepSeek DSML tool-call argument deltas incrementally (#42879)
jack QwertyJack Chauncey
2026-05-28 17:50:23 +08:00
2a781756a1 Restore Literal for WeightTransferConfig.backend (#43183) Harry Mellor 2026-05-28 10:39:41 +01:00
a04afd76aa [DSV4] Remove AMD/XPU path in deepseek_v4/nvidia (#43829) Woosuk Kwon 2026-05-28 01:00:52 -07:00
6cc8577421 [Kernel] Marlin MoE: include SM 12.x in default arch list (#40923)
tonyliu312 Tony Liumergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Shengqi Chen
2026-05-28 15:30:26 +08:00
d6b48f928f [BugFix] Fix hard-coded timeout for multi-API-server startup (#43768) Vadim GimpelsonandNick Hill 2026-05-28 11:09:13 +04:00
1b16f2ddc9 change name of fs_python secondary tier to fs. (#43600) Rotem Shavitt 2026-05-28 10:05:48 +03:00
0ba46d4b11 [ROCm][DSV4] Enable Tilelang MHC replacing torch/triton mhc (#43679) TJian 2026-05-28 15:05:28 +08:00
e1814f822d minor docs: fix incorrect example path (#43830) JINO ROHIT 2026-05-28 11:28:43 +05:30
7909f82a45 [Bugfix][Frontend] streaming tool-call serializer drops first args chunk when name and args share a DeltaMessage (#42683) Ignacio Sicaandsfeng33 2026-05-28 02:20:55 -03:00
626fa9bba5 [BugFix] Fix blocked reasoning parsing with MRV2 (#43808) Nick Hill 2026-05-27 21:59:34 -07:00
e54eff769d [Bugfix] Pass routed_scaling_factor to FlashInfer TRTLLM BF16 MoE (#43769) Thien Tran 2026-05-28 12:29:14 +08:00
05ac829629 fix: parse Qwen3 XML JSON arguments first (#43243) Yufeng HeandFlora Feng 2026-05-28 11:35:59 +08:00
33e94fc3ad [ROCm][CI] Stabilize Cargo cache and pre-test image checks (#43815) Andreas Karatzas 2026-05-27 22:24:44 -05:00
413ac5c070 [Misc][Rocm] Remove redundant AiterUnifiedAttentionBackend block size log (#43664) Nicolò Lucchesiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-28 05:19:11 +02:00
2d2c660104 [MoE] Remove inplace fused experts mechanism (#43727) Yongye Zhu 2026-05-27 23:00:19 -04:00
05eec7120e Fix RunAI streamer tensor buffer reuse during weight loading (#43464) Benjamin Bartels 2026-05-28 03:16:52 +01:00
c87f62ccf8 [Rust Frontend] Introduce mock engine for benchmark baseline (#43469) Bugen Zhao 2026-05-28 09:40:35 +08:00
1223732dda [ModelRunnerV2][Hybrid model] Support kernel block size in hybrid model (#38831) Mengqing CaoandNick Hill 2026-05-28 08:55:55 +08:00
381edde1b9 [Bugfix][Kernel] TRTLLM NVFP4 MoE chunking (#43599) amitz-nv 2026-05-28 03:36:21 +03:00
094124af15 Add @AndreasKaratzas to CODEOWNERS (#43740) Andreas Karatzas 2026-05-27 18:14:50 -05:00
5963c19478 Fix Qwen3-VL and Qwen3-omni-thinker accuracy degradation from deepstack inputs under torch.compile (#43617) Dakai An 2026-05-28 06:34:08 +08:00
7fb9c0197a [Bugfix][DFlash]allocate the proper number of lookahead slots (#43733) Benjamin ChislettandNicolò Lucchesi 2026-05-27 17:45:34 -04:00
2c2c966669 Validate against some config fields being set to 0 (#43794) Harry Mellor 2026-05-27 22:14:49 +01:00
2616f67faa Remove Transformers forward/backward compatibility tests (#43785) Harry Mellor 2026-05-27 20:46:36 +01:00
206b72c982 [Quantization] Fix Humming RoutedExperts import (#43540) Minh VuandWentao Ye 2026-05-27 19:51:56 +02:00
284e6f543d [8/n] Migrate merge_attn_states, mamba, sampler to torch stable ABI (continued) (#43361)
Chris Leonard Mikayla Gawarecki Shengqi Chen
2026-05-27 12:35:24 -04:00
05c50c721e [ROCm] mori: add InterNodeV1LL inter-node kernel selection via VLLM_MORI_INTERNODE_KERNEL (#41751) jatseng-ai 2026-05-27 09:33:32 -07:00
41688e2dc7 Fix early CUDA init (#43791) Harry Mellor 2026-05-27 17:30:11 +01:00
49a3510266 [Docs] Fix the duplicate doc icon issue (#43546) Chunyang Wen 2026-05-28 00:09:58 +08:00
165460941f [BugFix] HFValidationError with cloud storage URIs when HF_HUB_OFFLINE=1 (#39155) Injae Ryou 2026-05-28 00:53:32 +09:00
03d9cc2fe2 [misc] Bump cutedsl version to 4.5.2 (#43745) Yongye Zhu 2026-05-27 11:25:36 -04:00
52a31ccecc [Bugfix] Map reasoning_effort to enable_thinking in chat template kwargs (#43401) Ashwin GiridharanandChauncey 2026-05-27 05:39:49 -07:00
2272062471 [Kernel] Enable TritonW4A16LinearKernel as CUDA fallback for non-Marlin-aligned W4A16 shapes (#43731) Luciano MartinsandLuciano Martins 2026-05-27 07:36:27 -03:00
158289e0fc [Docs] Fix MLA prefill backend default docs (#43697) Mohammad Miadh Angkad 2026-05-27 18:13:22 +08:00
396c8fee50 [Rust Frontend] Align tool parser fallback behavior between streaming & non-streaming paths (#43662) Bugen Zhao 2026-05-27 18:13:12 +08:00
ad464e16c0 [Doc] Add Ascend NPU tab to the quickstart installation guide (#43550)
Aditya Singh Claudemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-27 01:41:29 -07:00
de12f5ca0b [ROCm][GPT-OSS] Avoid repeated compile-time cos_sin_cache.to(bf16) casts in rotary path (#42833) akii96 2026-05-27 11:22:27 +03:00
683033d4ba [Frontend] Add MiniCPM5 XML tool call parser (#43175)
zhangtao2-1 zhangtao Chauncey
2026-05-27 15:39:35 +08:00
8c94938cfb [MRV2][BugFix] Fix KV connector handling in spec decode case (#43719) Nick HillandWentao Ye 2026-05-26 23:37:56 -07:00
7b54690244 [ROCm][Perf] Expose AITER MoE sorting dispatch policy via env var (#39177) Nico Holmberg 2026-05-26 22:11:02 -07:00
1fc2cee50a [KVConnector][Mooncake] Wire reset_cache cascade end-to-end (#42694)
aoshen02 aoshen524 Claude Opus 4.7
2026-05-27 11:52:35 +08:00
0fa3114ae1 Fix test_aot_compile for torch 2.12 (#43695) Angela Yi 2026-05-26 23:12:49 -04:00
adaa5e455a [DSv4] Refactor compressor & Fix ROCm compatibility (#43710) Woosuk Kwon 2026-05-26 19:56:46 -07:00
c02c758ea4 [Deprecation] Deprecate functions as scheduled for v0.21.0 (#43358) Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-26 22:56:21 -04:00
aa6138169f [MLA][Attention] Add OOT MLA prefill backend registration mechanism (#43325) Matthew Bonanni 2026-05-26 21:56:09 -05:00
7e33081cee [Attention] Make FlexAttention and FlashAttention use num-blocks first layouts (#42095)
Lucas Wilkinson Matthew Bonanni Nicolò Lucchesi
2026-05-27 03:55:56 +01:00
d8eebe6d97 [Perf] Optimize Fp8BlockScaledMMLinearKernel input_scale tensor using new_empty() (#43677) Xin Yang 2026-05-26 19:55:52 -07:00
5bdb181df5 [ROCm][CI] Fix ROCm multimodal Qwen2.5-VL activation compile and Phi4MM ragged image mask handling (#43647) Andreas Karatzas 2026-05-26 21:53:34 -05:00
0b68f21e7c [Rust Frontend] Add reasoning/tool parser & renderer roundtrip tests (#43582) Bugen Zhao 2026-05-27 08:49:30 +08:00
dede691c95 [Bugfix] Split attention groups by num_heads_q for spec-decode drafts (#43543) Luciano MartinsandLuciano Martins 2026-05-26 21:11:01 -03:00
e19b9b1045 [ci] Add arm64 ci image (#41303) Kevin H. LuuandClaude Opus 4.6 2026-05-26 14:38:09 -07:00
812e7e7364 [Bugfix][V1] Fix TOCTOU race causing intermittent EADDRINUSE on multi-API-server DP startup (#42585) Vadim Gimpelsonandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-27 01:06:00 +04:00
d98cbf472b [KV Connector] MooncakeStore: drop dead discard_partial_chunks parameter (#43627)
Zhewen Li Zhewen Li Claude Opus 4.7
2026-05-26 13:40:21 -07:00
6e503868ca [Kernel] Porting fuse_minimax_qk_norm to manual fusion (#43410) Jee Jee Li 2026-05-27 04:16:03 +08:00
49b4882779 [CI] Soft-fail AMD entrypoints mirror tests (#43709) Kevin H. LuuandClaude Opus 4.6 2026-05-26 13:08:48 -07:00
193ce8812e [DSv4] Drop _get_compressed_kv_buffer in DeepseekCompressor (#43690) Woosuk Kwon 2026-05-26 10:11:25 -07:00
3aea37d28e [Doc] Add line limit to AGENTS.md (#43635) Woosuk KwonandMark McLoughlin 2026-05-26 09:31:23 -07:00
6f5b533241 Add LM head quantization support for ModelOpt (#42124) Wei-Ming Chen 2026-05-26 09:21:05 -07:00
c8414a8271 [ROCm] Remove MegaMoE integration in deepseek v4 (#43629) Woosuk Kwon 2026-05-26 08:56:04 -07:00
f51bbc694d [MoE Refactor] W4a8 int8 oracle (#42789) bnellnmandRobert Shaw 2026-05-26 11:15:42 -04:00
b226ddacfd [MoE Refactor] Migrate ModelOptMxFp8FusedMoE to oracle (#42768) bnellnmandRobert Shaw 2026-05-26 11:14:14 -04:00

... 62 63 64 65 66 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86