Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

b8bd773fe4 [XPU] Fix Triton attn fp8/bf16 check failing (#45758) liuzhenwei 2026-06-16 12:31:20 +08:00
2addbb9cc9 [BugFix] Support async scheduling with prompt embeds for multimodal models (#45673) Ruinan Ma 2026-06-15 21:12:54 -07:00
e3cfea2e1b [Multimodal] Add Qwen3-VL video loader (#44412) Isotr0py 2026-06-16 11:45:34 +08:00
f99260d2aa [Rust Frontend] Lower out-of-vocab validation to text layer (#45685) Bugen Zhao 2026-06-16 11:37:58 +08:00
3f65e21e32 [Rust Frontend] Support max_logprobs validation (#45674) Bugen Zhao 2026-06-16 10:57:56 +08:00
b00e76ff72 [Misc][Model] add io processor for query/document embeddings from ColBERT (jinaai/jina-colbert-v2) (#45210) xx-thomas 2026-06-15 20:32:32 -05:00
f4359a70f9 [DSV4][Minor] Fix supported KV cache dtypes (#44892) Woosuk Kwon 2026-06-15 17:14:51 -07:00
3afe659b6b [EP] Enable DBO with NIXL EP (#45275) Itay Alroy 2026-06-16 02:37:22 +03:00
16e91176cf [EP] Query NIXL EP top-k index dtype (#45298) Itay Alroy 2026-06-16 01:50:18 +03:00
ab8b0fe338 nixl_ep: Skip post-receive quantization for NVFP4 (#45606) Itay Alroy 2026-06-16 01:42:05 +03:00
d467a2a7f2 [Bugfix] Defer block freeing until in-flight steps finish under async scheduling + PD KV consumer (#45357)
llx Nick Hill Jiangyun Zhu
2026-06-16 05:36:09 +08:00
76a373eff4 [Frontend] Replace legacy Gemma4 parsers with engine-based implementation (#45588) Ben BrowningandFlora Feng 2026-06-15 17:34:07 -04:00
25ee659db0 Fix parallel_tool_calls: null treated as false instead of default true (#44955) Zang Peiyu 2026-06-16 05:14:10 +08:00
eacff17c8d [Model Runner V2][Bugfix] Fix MRV2 LoRA warmup (#35536)
Jee Jee Li Nick Hill Woosuk Kwon
2026-06-16 04:17:23 +08:00
cd9078fe59 [Frontend] Skip structural tags for auto tool_choice without strict mode (#45600) Flora Feng 2026-06-15 15:55:31 -04:00
e18fe932ca [Perf] Optimize DSv4 prefill chunk planning, 4.0% E2E Throughput Improvement (#45061) Wentao Ye 2026-06-15 15:50:21 -04:00
51ec5cf08f [Bugfix] Chat Completions Harmony Refactor Clean up (#45464) yzong-rhandBen Browning 2026-06-15 14:45:19 -04:00
7e612a0f06 [KV Offloading] Implement reset_cache for TieringOffloadingManager (#44541) Ronen Schafferandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-15 21:42:53 +03:00
0a1c5034f5 [Model] Add MiniMax M3 support (#45381)
youkaichao OpenAI Codex Isotr0py Thien Tran Bugen Zhao Jee Jee Li Roger Wang functionstackx Yongye Zhu Jee Jee Li
2026-06-16 01:01:25 +08:00
a3195fab7b [AMD][Bugfix][Quantization] Honor fused-name match in is_layer_skipped (#43981) RoyWang 2026-06-16 00:37:52 +08:00
0d80979644 [Chore] Consolidate reasoning/tool parser attributes into unified Parser in chat serving (#45548) Flora Feng 2026-06-15 11:16:45 -04:00
588db18362 [Bugfix] Two-phase KV allocation for cross-group prefix cache hits (supersedes #33775) (#44409) Saddss 2026-06-15 22:39:59 +08:00
fa63bb9db6 Remove redundant Triton KV cache dtype asserts and enforce architectural support (fp8 >= sm89) (#43914) Mike GandMichael Gschwind 2026-06-15 06:49:57 -07:00
5ed15f42b9 Fix the E8M0 scale computation in the MXFP4 (W4A4) MOE CUTLASS kernel (#43557)
Xin Hemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Kunshang Ji
2026-06-15 21:04:54 +08:00
b997071ec4 (security) Enforce audio upload size limit before full file materialization (#45510) Juan Pérez de Algaba 2026-06-15 12:25:24 +02:00
6c5872efc5 [Bugfix] Unset HF's default max_new_tokens for DiffusionGemma (#45417) Martin Kukla 2026-06-15 10:31:57 +01:00
1d88c4dadd [Docs] Update the online serving docs. (#45676) wang.yuqi 2026-06-15 17:23:36 +08:00
25c53d1293 [ROCm][Doc] Add installation notes about python version requirement (#45671) vllmellm 2026-06-15 17:22:55 +08:00
9872921c5f [XPU] skip UT test_with_ngram_gpu_spec_decoding (#44423) Yejing Lai 2026-06-15 16:46:30 +08:00
c17e2f7c84 [Bugfix][Rust Frontend] Make metrics respect --served-model-name (#45465) Reid 2026-06-15 16:05:10 +08:00
40eac9a9d9 [Rust Frontend] Support parallel_tool_calls = false (#44760) FAUST 2026-06-15 15:50:48 +08:00
b5adb027ad [Models] Fix MiMo v2.x QKV TP sharding + FP4 support (#45200) Giancarlo Delfinandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-15 02:13:34 -05:00
64833f8158 [Rust Frontend] Add external→internal request-id map for abort() (#45137) Sahil Singh 2026-06-15 12:21:24 +05:30
ddad5dbda2 [Bugfix][Rust] Sync EngineCoreReadyResponse with the Python dataclass (#45557) Will EatonandBugen Zhao 2026-06-15 02:49:42 -04:00
ebb0a71ad0 [Bugfix] Reject out-of-range temperature values in SamplingParams (#44965) Peter Pan 2026-06-15 14:12:44 +08:00
48df95c43e [Feature][Frontend] Report multimodal token counts in usage.prompt_tokens_details (#45458) Ting SUN 2026-06-15 13:20:58 +08:00
7df4fe1bd7 [Model] Remove XverseForCausalLM (#45638) Tiezhen WANGandClaude 2026-06-15 13:09:00 +08:00
b8336c3c7c [Bugfix][V1] Split V2 model-runner attention groups on num_heads_q (#45564)
Roger Wang Claude Opus 4.8 Nick Hill
2026-06-14 21:49:46 -07:00
e8d3e22c88 Fix included router missing path for FastAPI >=0.137 (#45629) Roger WangandClaude Opus 4.8 2026-06-14 21:28:52 -07:00
c4a3f9d137 [Frontend] Add Streaming Parser Engine and new Qwen3 Parser (#45413) Ben BrowningandFlora Feng 2026-06-14 23:59:05 -04:00
e3e3cd5458 [Bugfix][CI] Update Dockerfile dependency graph PNG (#45602) Flora Feng 2026-06-14 22:35:24 -04:00
8760f972ca [CPU] Refine CPU attention frontend (#45391) Li, Jiang 2026-06-15 10:26:54 +08:00
b675cb7d0f [Bugfix][CPU] Honor cgroup memory limit when computing KV cache size (#45086) maobaolongandLi, Jiang <jiang1.li@intel.com> 2026-06-15 10:26:50 +08:00
2725c84aae [XPU] Enable sequence parallel support for XPU (#38608) Chaojun Zhang 2026-06-15 10:26:46 +08:00
1801fad0ba [Bugfix] Stream Llama4 weight loading to avoid host-OOM with copy-returning loaders (#44645) Noa Neria 2026-06-15 05:23:44 +03:00
3d6ce816f0 [Bugfix][Model] Validate runai_streamer model_loader_extra_config (#45291) Ting SUN 2026-06-15 10:23:30 +08:00
2c764c089a Added real /v1/embeddings support for messages + chat_template_kw (#45173) Taneem Ibrahim 2026-06-14 20:08:10 -05:00
c621af1690 [BugFix] Fix prompt_embeds for multimodal models (#45383) Michael Ma 2026-06-14 01:44:56 -07:00
e2bf2b3d84 [Perf] Use bisect for mm feature lookup in model runner v2 (#45566) Roger Wang 2026-06-14 00:22:53 -07:00
725c3bc808 [ROCm][Perf] Enable W4A16 FlyDSL MoE (#44400) Amanzhol Salykov 2026-06-14 09:14:39 +02:00
9548a1887f [XPU] Support int4 group_size=32 W4A16 MoE (#45136) Marceli FylcekandKunshang Ji 2026-06-14 10:14:35 +03:00
9fd737badc [Bugfix][DCP] Fix illegal memory access in DCP a2a decode under full CUDA graphs (#45487) Jeff (Junze) Ma 2026-06-14 00:14:31 -07:00
4ef4492e9b [V1][Spec Decode] Add Dynamic SD (#32374)
Ekagra Ranjangemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Harry Mellor Benjamin Chislett
2026-06-14 03:14:27 -04:00
78e7293bb1 [Build] Fix CUDA arch build coverage gaps (#45277)
Shengqi Chen Xin Li ShawRong Change72
2026-06-14 13:09:20 +08:00
54bbf51668 [Bugfix] nightly Docker images crash with ImportError: AnthropicOutputConfig since May 28 (#44795) achyuthan.sandShengqi Chen 2026-06-14 08:45:29 +04:00
cf027b86af [Core] Simplify MRV2 async output handling (#45442) Nick Hill 2026-06-13 18:15:36 -07:00
71b961dd35 [Perf] SM90 cutlass fp8 mm supports odd M by swap_ab, 180~290% kernel performance improvement (#44572) Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-13 15:05:45 -04:00
521b88c29e [Bugfix] Reject structured outputs for diffusion decoders with a clear error (#45468) WEI CHENG CHIUandClaude 2026-06-14 03:04:01 +08:00
b3f0a0a0df Fix docs build on main (#45536) Harry Mellor 2026-06-13 16:53:23 +01:00
470229c37e [Security] Fix DoS via prompt_embeds on M-RoPE models (#45252) Juan Pérez de Algaba 2026-06-13 12:17:38 +02:00
2b3006076c [Security] Add timeout guard for regex compilation in structured outp… (#45118) Juan Pérez de AlgabaandHarry Mellor 2026-06-13 11:52:56 +02:00
96fa5cdd9e [CI Bug] Fix ValueError: There is no module or parameter named 'model.vision_tower.vision_model' (#45478) Wentao Ye 2026-06-13 05:38:37 -04:00
9261dbbc55 Treat null completion max_tokens like the default (#45491) Andreas Karatzas 2026-06-13 04:34:09 -05:00
2ecf7d0eb4 [Model Runner V2] Fix openai.InternalServerError: Error code: 500 - 'list index out of range' (#45467) Wentao Ye 2026-06-13 04:44:16 -04:00
0d29612292 [Doc] Fix uv dependency resolution failure for setuptools during CPU source builds (x86 & ARM) (#45412) midas 2026-06-13 11:48:58 +05:30
5b2943f5a6 [Bugfix] Return the tokenizer from maybe_make_thread_pool so it survives pickling (#45460) WEI CHENG CHIU 2026-06-13 14:01:35 +08:00
43f0e024bc [Render] Add /derender endpoints for disaggregated postprocessing (#43606) Martin HickeyandIsotr0py 2026-06-13 06:55:33 +01:00
1033ffac2e [CI] Wait for SSL cert refresher events in the test (#45489) Andreas Karatzas 2026-06-12 23:57:18 -05:00
ff5a30cfac [Bugfix] Replace deprecated Qwen2VLImageProcessorFast with Qwen2VLImageProcessor (#42700)
longguo Claude Roger Wang
2026-06-13 12:04:31 +08:00
17ee5b1ac5 [Bugfix] Set type/role explicitly in streaming message_start event (#45376) WEI CHENG CHIU 2026-06-13 09:40:50 +08:00
1a369783e9 [BugFix] Avoid prematurely freeing cached mm encoder outputs (#45347) Nick Hill 2026-06-12 15:39:40 -07:00
e3e31e54b0 [Bugfix][CPU] Don't build triton-cpu on arm64 release image (#45401) Kevin H. Luu 2026-06-12 14:51:45 -07:00
badddd254f [ROCm][DSV4][Perf] Fuse inverse-RoPE and cache bf16 wo_a in o-projection (#45103) Fangzhou AiandClaude Fable 5 2026-06-12 16:57:09 -04:00
c90650088d Add the QuantizedActivation linear-kernel contract (#44260) Michael GoinandClaude 2026-06-12 21:48:15 +01:00
9eaacb23ec [Kernel] Consolidate Marlin thread-tile padding across all dense Marlin paths (#45295) Michael Goin 2026-06-12 21:46:21 +01:00
78739c1946 [Model Runner v2] Migration from v1 to v2, with Qwen and DSv2 MOE models [3/N] (#42667) Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-12 16:44:52 -04:00
cf567cbc71 [Attention] Improve attention benchmarks: configs and profiling (#39336) Matthew Bonanni 2026-06-12 16:24:25 -04:00
39cb9bf292 [ROCm] Bump Torch to 2.11 (#45362) Micah Williamson 2026-06-12 15:22:26 -05:00
6e4a547176 [Refactor] Deprecate ResponsesParser wrapper, inline parsing into ParsableContext (#45431) Flora Feng 2026-06-12 16:15:41 -04:00
aab639c705 [Core][AMD] Propagate shutdown timeout to MultiprocExecutor (#43154) Ryan Rockandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-06-12 15:13:31 -05:00
efe7adb5e1 [Perf] Use native DSA indexer decode path for next_n > 2 on SM100 (#45322)
qizixi Claude Fable 5 Yongye Zhu
2026-06-12 12:54:00 -07:00
6635279d8a [Migration] Migrate GGUF quantization support to plugin (#39612) Isotr0py 2026-06-13 03:02:21 +08:00
d6fd7ce8da [Model][Dflash] Enable Dflash support for Qwen3NextForCausalLM targets (#45319) Jonas I. Liechti 2026-06-12 19:30:09 +02:00
272c16953e [Kernel][Helion][1/N] Add Helion kernel for dynamic_per_token_scaled_fp8_quant (#33790) Xiaohong (Sean) ChenandYanan Cao 2026-06-12 12:50:06 -04:00
053e7daa79 [Model] Add encoder CUDA graph support to Lfm2VL (#44930) Yi Zhong 2026-06-12 09:17:26 -07:00
5af4aec141 [Rust Frontend] Add standalone granite4 tool parser (#45216)
Tahsin Tunan Bugen Zhaomergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-12 22:16:36 +06:00
a30addc754 [Docs][KV Connector][NIXL] document KV Transfer stat logging and Prometheus metrics (#44055) Sai Sridhar Tarra 2026-06-12 21:09:11 +05:30
3b8fc3fe6d [Frontend] Support strict mode for tool calling with ResponsesAPI (#45396) Chauncey 2026-06-12 22:59:59 +08:00
9ff278b1d2 [Core][KV Connector] fix scheduler KV connector stats aggregation (#43877) Srinivas Krovvidiandsrinivas_oo7 2026-06-12 09:51:55 -05:00
c7aa3d2630 [Core] Support structured outputs for beam search (#35022) Guan-Ming (Wesley) Chiu 2026-06-12 21:56:25 +08:00
fbc3a1907a [Bug] Migrate Reset cache for both v2 and v1 model runner (#42759) Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-12 09:38:12 -04:00
4171ae406c [V1][Metrics] Add MLA attention metrics for DeepSeek MFU estimation (#39457) Thillai ChithambaramandMark McLoughlin 2026-06-12 09:28:40 -04:00
b7f9b6ab27 [Metrics] Add group-aware KV cache capacity to vllm:cache_config_info (#42206) Ethan Feng 2026-06-12 19:49:44 +08:00
8af550b399 [BUGFIX][XPU] Update fa interface for compatibility (#45394) liuzhenweiandKunshang Ji 2026-06-12 19:45:01 +08:00
f1e13f7df9 [Model] Remove Mono-InternVL (InternLM2VEForCausalLM) (#45129)
Tiezhen WANG Claude Isotr0py
2026-06-12 18:41:09 +08:00
88ed636218 [KV Connector]: Support KV push from Prefill to Decode node using Nixl KV Connector (#35264) snadampalandNicolò Lucchesi 2026-06-12 05:38:41 -05:00
a014dddbaa [11b/n] Migrate Machete kernels to torch stable ABI (#45304) Chris LeonardandShengqi Chen 2026-06-12 06:36:49 -04:00
a37b4a940e [Doc] AGENTS.md: add section about coding style (#45301) Thomas Parnell 2026-06-12 12:23:04 +02:00
f715f25f29 Fix misleading error for audio duration limit rejection (#45113) Juan Pérez de Algaba 2026-06-12 11:58:08 +02:00
462ef83d58 Update hidden states extraction integration test triggers (#45294) Fynn Schmitt-Ulms 2026-06-12 04:05:19 -04:00

... 56 57 58 59 60 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86