Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

5369f7b7b8 [MXFP8][ROCm] Fix MXFP8 MoE backend selection (#49747) fxmarty-amd 2026-07-29 01:57:20 +02:00
e7f6a39db8 [Test] Make EPD correctness tests configurable for XPU (#50110) liuzhenwei 2026-07-29 07:47:48 +08:00
a07fac758f [Perf] Zero-copy torch.Tensor pickling in shm_broadcast MessageQueue (#48442) Ruinan MaandNick Hill 2026-07-28 16:28:28 -07:00
bb3b61f2fd perf: dispatch non-grouped bias-less topk routing methods to fused path (#49618) Julien Debache 2026-07-28 23:57:22 +02:00
2899dca843 [Model] Add Kimi K3 support: Rust frontend [1/2] (#50104) Bugen Zhao 2026-07-29 05:31:13 +08:00
0b6aa3c47c [Bugfix][Spec Decode] Size DFlash query buffers for cudagraph-padded batches (#50065) Siddhant BhartiandClaude Fable 5 2026-07-28 12:34:22 -07:00
d552a68645 [Rust Frontend] Extract shared tracing setup logic into vllm-tracing (#50129) Bugen Zhao 2026-07-29 03:31:15 +08:00
118bcde449 [BugFix] Fix clang spinloop mwaitx include (#45532) johnnyychiu 2026-07-28 20:00:06 +01:00
6c7e679f04 [ROCm][Bugfix] Sanitize AITER paged-MQA logits before sparse top-k for DeepSeek-V4 (#49714) Shanshan Shen 2026-07-29 02:08:50 +08:00
1db989bbf1 [Bugfix][Multimodal] Fix video temporal padding estimates (#49030) labAxiaoming 2026-07-29 01:57:31 +08:00
8a7b3c2990 [compressed-tensors] update find_matched_target order to prioritize fused name matches over class match (#49483) Brian Dellabetta 2026-07-28 13:03:08 -04:00
05a0814863 [ROCm] Fix and optimize GPT-J-style MRoPE (#49906) Andreas Karatzas 2026-07-28 11:50:10 -05:00
4f56321d7e [ROCm] Cache fp32 upcast of static e8m0 weight scale in AITER scaled_mm (#47773) jiacao-amdandTJian 2026-07-28 09:34:16 -07:00
01661cc57f [Rust][Benchmark] Make vllm bench serve Rust delegation opt-in (#50081) Bugen Zhao 2026-07-29 00:21:12 +08:00
ba702e978e [Attention] Skip sparse indexer scoring for dense short prefills (#48407) Yiliu DongandOpenAI Codex 2026-07-29 00:17:22 +08:00
6453fc0b8c [Bugfix] Don't reuse engine core payload buffer while zmq is sending it (#50053) Nick HillandAndreas Karatzas 2026-07-28 09:09:28 -07:00
4fb483ca86 [Docs] Expand llm-d integration page (#45432)
IBRAHIM IBRAHIM ibrahimibrahim Claude Opus 4.8
2026-07-28 11:47:56 -04:00
30217b0e80 [Bugfix][KV Offload][P2P] Scope serve state to fetch rounds (#49877)
Itay Etelis Itay Etelis Itay Etelis
2026-07-28 18:01:55 +03:00
0d0504b54c [Core] Warm up runner-owned Triton kernels before the first request (#49903) Nick Hill 2026-07-28 07:58:02 -07:00
1e81853afc [Bugfix][KV Offload] Keep Mamba block span unscaled under DCP (#49964) Jonguk CheongandOpenAI Codex 2026-07-28 23:41:55 +09:00
b6cbba8bc8 [Bugfix][Kernel] Fix batch invariance in RMSNorm kernels by pinning block size (#48391)
oops-oom oops-oom Claude Opus 4.8 Wentao Ye Shengqi Chen
2026-07-28 22:24:02 +08:00
94100b5915 [CI] Wire untethered test files into CI jobs (#49340) Nick HillandClaude Opus 4.8 2026-07-28 07:02:31 -07:00
62d8db7c05 [Bugfix] Add missing vllm/models/kimi_k3/__init__.py (#50131) Harry Mellor 2026-07-28 14:47:04 +01:00
601fa9a74e [KV Connector] Support NIXL heterogeneous P/D block sizes for hybrid models (#49612) Nick HillandClaude Fable 5 2026-07-28 06:45:02 -07:00
9b9fc4039c add epilogue hook to flex attention (#45841)
liangel-02mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Matthew Bonanni Michael Goin
2026-07-28 07:26:55 -06:00
98e91a9600 [PD][NixlPush] Skip extra add_remote_agent step in D->P handshake (#49345) Nicolò Lucchesi 2026-07-28 14:57:12 +02:00
948107acf7 [Bugfix] Enhance extra_config handling for layer name suffix matching (#48589) Xin HeandKunshang Ji 2026-07-28 20:37:55 +08:00
35efdf6b34 [Elastic EP] Async preparation (#47288) Itay Alroy 2026-07-28 15:19:13 +03:00
d2bfc6fe20 [Build] Fix DeepEP CUDA driver stub linking (#50103) Kevin H. LuuandOpenAI Codex 2026-07-28 05:09:19 -07:00
912d6b619d [Rust Frontend] Align sampling validation with Python (#47494) Reid 2026-07-28 20:02:00 +08:00
bf9f23003c [Rust Frontend] Fix finish reason for named tool choices (#49496) Reid 2026-07-28 18:53:18 +08:00
25ace8fe5d [CI] Increase Qwen3.5 MTP GSM8K generation length (#49881)
Jiangyun Zhu OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-28 18:04:36 +08:00
247470f23a [CI] Add PyTorch stable ABI audit check (#48164) Chris LeonardandShengqi Chen 2026-07-28 05:37:01 -04:00
88402a41c4 [Test] Skip ROCm AITER MLA prefill tests on non-ROCm platforms (#49945) Liangliang Maandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-07-28 16:26:49 +08:00
5ed3faa43d [Rust Frontend] Add ordinary-text tokenizer encoding (#49992) Bugen ZhaoandOpenAI Codex 2026-07-28 15:46:51 +08:00
61ac368021 [Kimi-K3] Add AttnRes kernels (#50090) Thien Tran 2026-07-28 15:22:09 +08:00
99b57a4823 [CI][ROCm] Soft fail LoRA mirror (#50086) Andreas KaratzasandOpenAI Codex 2026-07-28 02:18:04 -05:00
b09688a6e7 [Bugfix][Spec Decode] Preserve draft buffers across level-2 sleep (#49774)
aoshen02 aoshen02 vx120 Claude Opus 5 Roger Wang
2026-07-28 15:16:21 +08:00
03a2d03367 [Bugfix] Respect cgroup memory limits on all platforms (#49966) ChaunceyandAndreas Karatzas 2026-07-28 14:38:20 +08:00
9069a57139 [Core][Frontend] Add weight version tagging for RL rollouts (#49040) Shuolei Wang 2026-07-28 14:35:29 +08:00
90245f4190 [Bugfix] Fix multi-modal support on CPU MRV2 (#50073) Li, Jiang 2026-07-28 14:09:48 +08:00
f472ab0a4c Remove triton per group quant [ROCm] [Bugfix] (#49621) afriedri 2026-07-28 00:53:46 -05:00
74587939b1 [Build] Fix CUDA arch detection producing kernel-less builds on SM121 (#49904) Ayushman Singh 2026-07-28 01:01:56 -04:00
d223c900d8 [Bugfix] Only pad transformers backend value when it is narrower (#50060) Nick Hill 2026-07-27 21:36:45 -07:00
52c3c4a42f [Bugfix][KV Offload][OBJ] Preserve job completion during cleanup (#49947) MINJUN GILandOpenAI Codex 2026-07-28 13:10:52 +09:00
a8f296083f [KV Offload] Make compact secondary identity TP-independent (#49858)
Chang Guo GPT-5.6 Sol Or Ozeri
2026-07-27 21:10:20 -07:00
fbb1ef6803 [Bugfix] Fix DeepseekV4FP8 Quark MXFP4 crash on list-valued weight (#49634) Colin ZandAndreas Karatzas 2026-07-27 20:32:42 -07:00
33fe71a4d3 [AMD] Revert Mxfp4MoeBackend.TRITON_UNFUSED fallback (#46491)
fxmarty Felix Marty Andreas Karatzas
2026-07-28 05:31:23 +02:00
d18ed2304a [KV-offload][FS] : Batch store/load_block in C (#49152) Varun Sundar RabindranathandVarun Sundar Rabindranath 2026-07-27 23:21:53 -04:00
60915c972c [Feature] Add VidCom2 video token pruning (#47750) nvbfalk 2026-07-28 05:20:17 +02:00
7aea73d83d [ROCm][Quark][6/N] Use MXFP4 linear kernel abstraction for aiter backend (#49348) fxmarty-amdandAndreas Karatzas 2026-07-28 04:35:57 +02:00
73af7a362a [XPU] Add online fp8 quantization test (#44513) Yan Ma 2026-07-28 10:34:33 +08:00
e68bfc2828 [CI][ROCm] Soft-fail Python-only installation mirror (#50041) Andreas Karatzas 2026-07-27 21:22:53 -05:00
02b6ecf07c [ROCm][DSv3.2] Eliminate per-decode FillFunctor launches in sparse-MLA hot loop (#44527) frida-andersson 2026-07-28 04:12:20 +02:00
1206891822 [ROCm][KVConnector][MoRI-IO] Fix WRITE-mode remote-TP rank collapse (#46332 follow-up) (#47764)
avininjamay8 avininjamay8 avininjamay8
2026-07-28 06:56:24 +05:30
60b3d39cd3 [Docs] Remove experimental warning for EP (#50057) Woosuk Kwon 2026-07-27 17:06:28 -07:00
60417b4b74 [Core][PCP] Select MRV2 when PCP is enabled (#50034) Lucas Wilkinson 2026-07-27 20:03:26 -04:00
272abd5f48 [Tests][Spec Decode] Add gemma4 MTP acceptance rates test (#47920) Giancarlo Delfin 2026-07-27 16:15:54 -07:00
1e34a13539 Fix Humming non-gated MoE (#49096) Netanel Haber 2026-07-28 01:56:44 +03:00
ebcef33766 Fix MQA with tensor parallelism on transformers modeling backend (#49987) Miłosz GrunwaldandHarry Mellor 2026-07-28 00:51:17 +02:00
28158b2fc3 [ROCm] [BugFix] Fix Quark GLM-5.2 Checkpoint inference: indexer wk per-channel FP8 dequant + missing sparse-MLA metadata fields (#48886)
Colin Z TJian fanxingran
2026-07-27 15:38:22 -07:00
53f6dd5c6f [CI][ROCm] Fix test_ocp_mx_wikitext_correctness reference value (#49690) fxmarty-amd 2026-07-27 23:47:07 +02:00
99115fcdcd [CI] Initialize DeepEP FP8 test weights (#49912) Andreas Karatzas 2026-07-27 16:29:08 -05:00
1053e248f0 [ROCm][Quantization][5/N] Refactor quark_moe w8a8-int8 w/ oracle (#46765)
amd-sourjya amd-sourjya Cursor Andreas Karatzas
2026-07-27 14:01:34 -07:00
b5bcb3ce88 [Refactor] Remove dead code in multiple files (#49745) Wentao Ye 2026-07-27 15:58:26 -04:00
b2f9e4caa4 [DSv4 Perf] Adaptive topk width, 1.0% E2E throughput improvement (#50004) Wentao Ye 2026-07-27 15:56:52 -04:00
831d3848f1 [Core] Fail fast when /dev/shm is too small for the shm ring buffer (#48879)
Andrea Tassi Dr Andrea Tassi Claude Opus 4.8
2026-07-27 20:48:35 +01:00
fd10e8946d [Test] Regression test for hybrid-Mamba eagle cache-peek in Mooncake connector (#43559) (#48361) Rishi PuriandClaude Opus 4.8 2026-07-27 16:03:58 -03:00
ed13deb376 [Bugfix][CPU] Fall back to torch for unaligned swigluoai on NEON/vec MoE (#49985)
oops-oom oops-oom Claude
2026-07-28 02:35:57 +08:00
99de48e98f Fix MLA padding and grouped topk routing in the Transformers modelling backend (#49982) Harry MellorandClaude Opus 5 2026-07-27 19:32:32 +01:00
bf2b45b5d6 [Attention] Integrate FlashAttention 4 SM100 headdim 256 support (#42669) Matthew Bonanniandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-07-27 14:25:50 -04:00
8112b6c997 [MRV2] Always build attn metadata at capture time (#49364) (#49995) Nick HillandWoosuk Kwon 2026-07-27 10:25:17 -07:00
15d65f8669 [Bugfix] Changed speech to text chunk timestamp to cumulative approach (#41131) TobyJBell 2026-07-27 13:06:46 -04:00
e3c2fc3b3c [Rust Frontend][gRPC] Add server and model discovery (#49491) Connor CarpenterandNick Hill 2026-07-27 09:53:27 -07:00
2b465b2c42 [Misc][PD] Nixl cleanup get_backend_aware_kv_block_len and virtually_split_kv_in_blocks (#49988) Nicolò Lucchesi 2026-07-27 18:51:37 +02:00
3f47a8384d [Bugfix] Fix VLLM_ENFORCE_STRICT_TOOL_CALLING mutation in tests (#49846) yzong-rh 2026-07-27 12:12:14 -04:00
04502deca2 [Perf] Hash videos by source bytes (#49607)
Guan-Ming Chiu Isotr0pymergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-27 23:57:55 +08:00
d2ca3002d9 [MRV2][Performance] Skip no-op FP32 logits materialization (#47711)
Song Zhixin Wentao Ye Jee Jee Li
2026-07-27 23:31:36 +08:00
27d7061ef6 [Bugfix] Restore truncate_prompt_tokens for Jina rerank/score online (#49963) Umut Polat 2026-07-27 17:52:59 +03:00
ef9975d021 [Bugfix] Reject pipeline parallelism for DiffusionGemma (#45828) Guan-Ming Chiu 2026-07-27 22:37:54 +08:00
56c96b0d91 [Perf] Tune LL BF16 Router GEMM (#48774) Roberto L. Castro 2026-07-27 16:25:37 +02:00
59a6b0411d [Core] Fix internal LB load-balancing (#49204) Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-07-27 07:00:46 -07:00
dbccc5ae32 [Model] Enable EVS for Qwen3.5 (#48912) Rui "Garry" Gao 2026-07-27 21:42:35 +08:00
a89015c6df [Perf] Make merge attention context count a runtime argument (#48739) liminfei-amd 2026-07-27 21:24:42 +08:00
96fa3f42c9 [Perf] Skip ll_bf16 router GEMM warmup for non-MoE models (#49659) neweyes 2026-07-27 20:16:42 +08:00
81962bb699 [Bugfix]Reject invalid FlashInfer MNNVL workspaces (#49043) rongfu.lengandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-07-27 20:12:23 +08:00
92e8518d37 Improve Transformers modelling backend fx tracer (#49957) Harry Mellor 2026-07-27 12:51:16 +01:00
77cba0259f [KV Offloading] Per-request tier filtering with TierFilter/TierMatcher (#48123) Ronen Schaffer 2026-07-27 13:29:57 +03:00
30fbd05537 [ROCm] Use backend-default dot precision for ReplaySSM (#49909) Andreas Karatzas 2026-07-27 05:11:06 -05:00
0906123953 [ROCm] [Model] Enable TML inkling (#48841) TJianandOpenAI Codex 2026-07-27 18:05:46 +08:00
bc3629b1c4 [ROCm][CI] Skip three torchao tests of gfx950 until torchao==0.18 is released (#49732)
fxmarty Felix Marty Claude
2026-07-27 11:36:38 +02:00
312ea82e75 [CI][ROCm] Make hf-xet reconstruction safe on shared NFS (#49837) Andreas KaratzasandOpenAI Codex 2026-07-27 04:23:12 -05:00
394beb633b [Bugfix][ROCm] Use batch DMA for CPU KV cache loads (#49843) Andreas KaratzasandOpenAI Codex 2026-07-27 04:18:01 -05:00
7f599d7854 [communication] [bugfix] fix quickreduce acc error in cudagraph mode (#46913)
haoyangli0109 tjtanaamergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Douglas Lehr Andreas Karatzas
2026-07-27 16:38:45 +08:00
eb290ab673 [Bugfix][CPU] Zero-pad MoE intermediate size for grouped-gemm TP alignment (#49591) Li, JiangandClaude Sonnet 5 2026-07-27 16:32:23 +08:00
cbc3a87200 [Tokenizer] Use HF config for HF tokenizers (#49907) Andreas Karatzas 2026-07-27 02:49:11 -05:00
afc94523c9 [XPU][CI] Use platform device in InputBatch V2 test (#49939) liuzhenwei 2026-07-27 15:28:02 +08:00
8061dc26bd [Bugfix] Normalize sparse MLA warmup compression ratios (#49392) Xiaochang WuandCopilot 2026-07-27 15:02:48 +08:00
fd9d2ede6f [Rust Frontend] Keep --max-model-len engine-owned (#49944) Bugen ZhaoandOpenAI Codex 2026-07-27 14:50:08 +08:00
e09900436c [CI][ROCm] Reduce kernel test runtime (#49915) Andreas Karatzas 2026-07-27 01:44:21 -05:00

... 40 41 42 43 44 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86