Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

b216db3ed0 [Bugfix] Reject negative token ids as out-of-vocabulary (#51795)
Junhao Shen Claude Opus 4.8 Vadim Gimpelson
2026-08-14 09:07:29 +08:00
827a2af806 [Kernel] Gemma-4 FA4 FP8 Kernel (#48666)
Jhao-Ting Chen Qixiang Lin Claude Opus 4.8 Yongye Zhu
2026-08-13 21:42:00 -03:00
38f097fab8 [Bugfix] Reject NUL byte in structured_outputs.regex (#51796) Junhao ShenandClaude Opus 4.8 2026-08-14 08:08:36 +08:00
f80b66f548 [Model Runner V2][Spec Decode] Add KV cache support for multi-layer MTP (#50062) Giancarlo DelfinandClaude Fable 5 2026-08-13 16:22:29 -07:00
71b0da7c4e [Bugfix] Fix .../mrope.py::apply_interleaved_rope() when torch.compile is used in torch==2.13 (#52005) bastefaniak 2026-08-14 00:08:40 +02:00
51def7848a [Bugfix] Reapply 50869 (#52223) Benjamin Chislett 2026-08-13 18:08:07 -04:00
63550515b4 [Bugfix] Correct prompt lengths for timed_traces benchmark (#45423) Stan WozniakandThomas Parnell 2026-08-13 23:51:24 +02:00
b652dedd0c [Attention] Fix FlashInfer SM12x prefill with sinks (#52148)
Andrii Skliar Andrii Skliar Michael Goin
2026-08-13 23:25:09 +02:00
2e2ffd104b [CI Failure] Fix CUDA wheel build for the Kimi K3 fused MLA kernel (#52210) Misha Goin 2026-08-13 16:56:36 -04:00
73b83949f7 [Platform] Add check_runner_kv_caches_multi_layer (#51633) wangxiyuan 2026-08-14 03:25:07 +08:00
11c3fa4adc [Bugfix][ROCm][CI] Give the AITER MLA decode metadata stub its MLA dims (#52139) stefankoncarevic 2026-08-13 20:35:00 +02:00
c5b7c069a4 [ROCm] Defer tilelang import through its import from vllm.tilelang_utils import tilelang and relaxed has_tilelang (#51159)
fxmarty-amd Andreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-13 20:12:40 +02:00
e6b2a8ad5e [Bugfix][Structured Output] Mask request stop tokens in xgrammar until grammar terminates (#50595) yzong-rh 2026-08-13 14:07:59 -04:00
48825acc23 [Quantization] Remove dead QuantizationConfig.is_mxfp4_quant (#51793) fxmarty-amd 2026-08-13 19:58:31 +02:00
b96bcd0b47 [ROCm][CI] Solidify entrypoint LLM lifecycle (#51280) Andreas Karatzas 2026-08-13 12:24:02 -05:00
69d4c3a06b Auto-ping Cohere on related issues (#52091) Cyrus Leung 2026-08-14 00:40:38 +08:00
83d4c6196a [Bugfix] Declare SupportsEagle3 on KimiLinearForCausalLM (#52171) Nick Iusiumbeli 2026-08-13 19:22:50 +03:00
b245d8e73f Apply logit softcapping in Transformers modelling backend (#52173) Harry Mellor 2026-08-13 17:21:55 +01:00
6014f9e672 [Kimi-K3] Add GEMM-RS for sequence parallelism (#52079) Thien Tran 2026-08-14 00:02:28 +08:00
96acd473a7 [Bugfix][MiniCPM-V] Fix AssertionError in get_dummy_mm_data when passing VideoDummyOptions to _get_dummy_images (#52122) Qiming Zhang 2026-08-13 08:44:03 -07:00
64ca614fe4 [Bugfix] Fix --data-parallel-start-rank 0 being treated as unset in create_engine_config (#47692) Ali JaseemandClaude 2026-08-13 10:12:26 -05:00
80d6d557f3 Standardise weight tying on ParallelLMHead.tie_weights (#52147) Harry MellorandClaude Opus 5 2026-08-13 16:10:09 +01:00
7553aac77b [Frontend] Add routed-experts prompt offset (#51906)
aoshen02 OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-13 23:09:00 +08:00
f3c1638927 [ROCm] Enable V2 model runner for Kimi-K3 on ROCm (#51653) vllmellm 2026-08-13 22:34:53 +08:00
2d24355eb8 [Bugfix] Fix packed GDN decode launch for large batch-head grids (#52030) Michael Goin 2026-08-13 10:19:08 -04:00
170592a931 [Bugfix] Disable sequence parallelism for Dots3 NOTE (#52172) 范裕达 2026-08-13 22:13:00 +08:00
95c9144424 [LoRA][Gemma4] Support vision tower LoRA (#42662)
linitra24 Jee Jee Li linitra24
2026-08-13 22:07:43 +08:00
8e1131e1d1 [Model] [Quantization] Add Ling hybrid MXFP4 routed experts support (#52114) zexplorerhj 2026-08-13 21:46:53 +08:00
152c913a63 [Frontend] Log output token IDs at DEBUG level (#52098) yang rui 2026-08-13 21:11:51 +08:00
015660da91 [Misc] Add missing return type annotations in outputs.py (#52145) Vineeta TiwariandVineeta Tiwari 2026-08-13 18:28:45 +05:30
c4e969294e [XPU] Add tuned Mamba SSU configs for Intel Arc Pro B70 (#50534) pmanczak 2026-08-13 14:36:55 +02:00
d0ae25e2f2 [Bugfix] Preserve Anthropic disable_parallel_tool_use (#52021) Taneem Ibrahim 2026-08-13 14:25:55 +02:00
5fee0a872d chore: Upstream Cohere parser fixes + tests (#51998) JasonCohere 2026-08-13 13:05:36 +01:00
443fa5aca0 [Doc] Fix stale rejection_sample_method and synthetic_acceptance_rate (#51611) QWERQWERQWE86 2026-08-13 18:56:27 +08:00
37c3bdf5a7 fix(security): enforce audio decode duration limit in NanoNemotronVL (#50221) Juan Pérez de Algaba 2026-08-13 12:31:55 +02:00
b8baa31a28 Hardware-agnostic model definition via HF transformer backend (1/N) (#49458) Thomas Ortner 2026-08-13 12:21:50 +02:00
903da602f3 [Docs] Fix WhisperEncoderLayer.forward docstring in dots3_note (#52134) Harry Mellor 2026-08-13 10:32:14 +01:00
1c3633acaf [CI/Build][CPU] Shrink triton-cpu-build layer by dropping build artifacts (#52127) Li, Jiang 2026-08-13 17:17:00 +08:00
f96261637c [ROCm][Perf] Kimi-K3 Remove prefill pipeline stall in chunk KDA (#51862) kliuae 2026-08-13 17:11:45 +08:00
50ba4bc6b2 [Feature] Mask Replay (#49577)
vx120 aoshen02 Codex Claude Opus 4.6 OpenAI Codex
2026-08-13 16:44:31 +08:00
10bcad2523 Update CODEOWNERS (#52123) Cyrus Leung 2026-08-13 16:43:07 +08:00
7bbbf7c8e5 [Core] Configure custom encoder cache managers from VllmConfig (#51251) hotTeaandHMCCMH 2026-08-13 16:08:01 +08:00
399f97424e [Bugfix][R3] Size monolithic routing replay buffer for DP (#50874)
TomerBN-Nvidia Claude Opus 5 zjy0516
2026-08-13 10:18:08 +03:00
373592ef57 [CPU] Ship triton-cpu wheel and fix several hardcoded pin_memory=True (#52092) Li, Jiang 2026-08-13 14:40:53 +08:00
89c8401c8a [Model] Skip unused Jina V5 output layers (#52037) kiroxuandCodex 2026-08-13 14:14:37 +08:00
7bc1660bd6 [Misc] Use VLLMValidationError in pooling input validation (#51931) Frank 2026-08-12 23:13:25 -07:00
b908a21f9a [Model][LoRA] Add tower/connector LoRA support for Ultravox (#48215)
Yangyi Gao Claude Fable 5 高杨懿 OpenAI Codex OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-13 13:57:59 +08:00
5658391af7 Remove NIXL reinstall step (#51882) ovidiusm 2026-08-13 07:44:05 +02:00
903d2efe7e [Attention][MLA] Fuse Kimi-K3 chunked-context K/V packing (#51772)
Yongye Zhu Claude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-12 22:42:57 -07:00
63344913a6 [XPU] update UMD to 26.27 (#50513) Yan MaandKunshang Ji 2026-08-13 13:15:25 +08:00
3d204dfdaa Revert "[Perf][ROCm] Dual-stream decode with hipgraphs" (#52024) Simon DanielssonandAndreas Karatzas 2026-08-13 06:38:34 +02:00
2ac1f683f1 [BugFix] Reserve the bonus query slot in DFlash scheduling budget (#51256) kx 2026-08-13 11:50:23 +08:00
79f3183f86 [Bugfix] Bound KV block zeroing launch geometry (#52058) Lucas WilkinsonandOpenAI Codex 2026-08-12 23:42:26 -04:00
f9538af28c [Core] Clearer comments in BlockPool.free_blocks() (#52076) Nick Hill 2026-08-12 20:36:08 -07:00
9a276d6375 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[cC][dD]" (#52003) Wentao Ye 2026-08-12 23:33:55 -04:00
15227b934a [Bugfix][KV Offload] Emit self-describing CPU events at KV-group block granularity (#51614) Ziqi Fan 2026-08-12 20:28:15 -07:00
f9a0f629bd [KV Offload] Expose data-parallel topology to offloading backends (#51879) Ziqi Fan 2026-08-12 20:27:26 -07:00
6e502b62cb fix and test EPLB balancedness calculation (#51813) Julien Debache 2026-08-13 05:27:22 +02:00
61826c1c6f [Hardware][Power] Unqualized MoE Backend for Power (VSX) (#51624)
Akash kaothalkar Akash kaothalkar Antigravity
2026-08-13 08:26:09 +05:30
8eb35c5217 [CI] Mirror external test assets in vLLM S3 (#52064) Kevin H. LuuandOpenAI Codex 2026-08-12 19:08:59 -07:00
b369f10d5c [Bugfix][ROCm][CI] Restore the DeepSeek-V4 input GEMM override point (#51821) stefankoncarevicandAndreas Karatzas 2026-08-13 03:33:52 +02:00
5936bac7d7 [Bugfix] Report FULL_ATTENTION for uniform-base UniformTypeKVCacheSpecs groups instead of UNKNOWN (#51218)
Yifan Jiangmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Yifan Qiao
2026-08-12 18:25:23 -07:00
caf9e8f7d5 [CI] Force source builds for hybrid dependencies (#52043) Andreas KaratzasandOpenAI Codex 2026-08-12 20:18:17 -05:00
98f86b9c02 [ROCm] [bugfix] Chunked prefill paged decode masked load perf (#50017) afriedri 2026-08-12 18:04:01 -05:00
23f360edaa [CI Bug] Fix ci moe test (#52009) Wentao Ye 2026-08-12 17:28:53 -04:00
025d56a11e [Build] Update DeepGEMM pin to deepseek-ai nv_dev tip (#52035) Yongye ZhuandKimi Code 2026-08-12 13:49:04 -07:00
34735aceda [Bugfix] Pin DeepEP by its full commit hash (#52028) Tyler Michael Smith 2026-08-12 16:01:32 -04:00
e62abc37d4 [Bugfix][Kernel] Fix persistent top-k histogram reuse after short rows (#49139) xiao fengandMichael Goin 2026-08-13 03:54:45 +08:00
7f7a32cfec [Spec Decode] DSpark confidence-scheduled verification (#47808)
Lucas Wilkinson OpenAI Codex Claude Opus 5 Benjamin Chislettmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick Hill OpenAI Codex
2026-08-12 15:27:48 -04:00
fe889ac925 [K3 Perf] Flash kda out kernel for prefill, 1.1~1.4x kernel performance improvement (#51311) Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-12 14:51:48 -04:00
19d61b1273 [Bugfix] Initialize DeepGemmQuantScaleFMT oracle lazily; bound QuantFP8 UE8M0 packed path to group_size 128 (#51359)
kiroxu Copilot Wentao Yemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-13 02:27:50 +08:00
d20a031d33 [Bugfix][ROCm][MoE] Update AITER MXFP4 W4A16 tests to the renamed expert_mask (#51980) stefankoncarevic 2026-08-12 19:45:45 +02:00
0fb168e6ee [CI Bug] Fix ci qwen3.5 (#52007) Wentao Ye 2026-08-12 13:22:09 -04:00
b745d08de1 [ROCm][K3] Dequantize the fp8 decode query for MLA backends without quant-query support - TRITON_MLA (#51860) Hongxia YangandClaude Fable 5 2026-08-12 12:40:41 -04:00
324f452f64 [ROCm] update triton in base docker for gluon compatibility (#51464)
Hongxia Yangmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Andreas Karatzas
2026-08-12 12:26:17 -04:00
6ea6c42659 [Bugfix] Bound Anthropic stop sequences (#51997) Taneem Ibrahim 2026-08-12 18:05:12 +02:00
8151f2ad43 [Docs] Warn that --api-key does not gate all endpoints (#51999) Russell BryantandClaude Opus 4.8 2026-08-12 11:16:35 -04:00
9035151d6c [Model] Add native Dots3 NOTE multimodal support (#51255)
范裕达 OpenAI Codex Jee Jee Li Jee Jee Li
2026-08-12 23:13:20 +08:00
b1b752042f Avoid long-blocking H2D copies in ViT (#51841) Max HuandNick Hill 2026-08-12 10:54:04 -04:00
7aa248fcfe [XPU][CI/Release][2/N] add triton shim in xpu requirements (#51935) Kunshang Ji 2026-08-12 20:10:10 +08:00
8e958902ee [Bugfix] Disable fine-grained prefix-cache hits for incompatible hybrid KV layouts (#51843) Michael Goin 2026-08-12 07:08:40 -04:00
4c51ceb11b [Bugfix][Multimodal] Invalidate retained PyNvVideoCodec decoder after failure (#51139) dmai-afk 2026-08-12 18:02:10 +07:00
86e2ab50aa [Bugfix][Frontend] Return 400 for invalid PyNvVideoCodec video input (#51120) dmai-afk 2026-08-12 17:58:46 +07:00
e60f3c4c13 [Bugfix] Fix MiniMax-M3 compressed-tensors FP8 MoE SwiGLU params (#46845) Tan Pin Siang 2026-08-12 03:52:30 -07:00
6accb779a3 [CI] Add registry layer cache to x86 CPU image build (#51911) Li, Jiang 2026-08-12 17:58:01 +08:00
8c011da6d0 [XPU] Route block-quantized FP8 weights to the W8A8 kernel (#50787) Chaojun Zhang 2026-08-12 16:57:22 +08:00
aeece10c06 [Bugfix][XPU] Run GDN attention as eager break under breakable cudagraph (#51928) HuanxingandCopilot 2026-08-12 16:23:39 +08:00
4eef91c03c [Model] Support R3 capture with DeepGEMM MegaMoE (#51831) aoshen02 2026-08-12 16:07:07 +08:00
10b7766a90 [Attention] Move context_lens_tensor compute into GDN prefill path (#51913) Xin Yang 2026-08-12 00:35:44 -07:00
10f9b5d74f [CI/Release][XPU] fix workdir path for triton shim job (#51923) Kunshang Ji 2026-08-12 14:41:29 +08:00
3ac9525507 [Bugfix][Quantization] Reuse online NVFP4 MoE kernel across reloads (#50074) Matej SirovatkaandJee Jee Li 2026-08-11 23:06:53 -07:00
02ac17851d [Refactor][MRV2] Unify uniform decode token count helper (#51917) Lucas WilkinsonandOpenAI Codex 2026-08-12 01:58:43 -04:00
f2906b492e [Bugfix] Avoid repeated layerwise reload warning scans (#49505) aoshen02andCodex 2026-08-12 13:53:50 +08:00
3ee2df3033 [Frontend] Make model optional on all /derender request classes (#51463) Vinay R DamodaranandClaude Opus 5 2026-08-11 21:48:58 -07:00
a53ad85913 [XPU][CI]Change to use global VLLM_DISABLE_COMPILE_CACHE=1 in Intel GPU CI (#51905) xiangdong 2026-08-12 12:30:41 +08:00
3962042304 [Bugfix][V1][Multimodal] Recover from P0/P1 processor cache drift (#46747) (#46747)
WillZZZy Cyrus Leung Claude
2026-08-12 00:06:57 -04:00
f4e9bb01de [CI/Release][1/N][XPU] Publish XPU Triton shim index (#51759) Kunshang JiandGitHub Copilot 2026-08-12 11:45:06 +08:00
7f9173dfa2 [ROCm][Perf] Kimi-K3 Fused kernel for KDA decode (#50654)
kliuae Claude TJian
2026-08-12 11:39:34 +08:00
d462dee37d [Bugfix][Triton] Make fp8_min/fp8_max constexpr in _quantize_pad_fp8_kernel (#51872) Max HuandMax Hu 2026-08-11 23:09:54 -04:00
3fa89225bc [CI/Build] Use file rendezvous for local distributed tests (#51652) Stellar鱼 2026-08-12 10:48:12 +08:00

... 33 34 35 36 37 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86