Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

1ae1051b4b [Bugfix][Rust Frontend] Return 400 for prompt-validation submit errors (#45286) JinYan SuandClaude Fable 5 2026-06-12 15:53:11 +08:00
2043258dec [Frontend] Support strict mode for tool calling (#45003)
Chauncey cjackalmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-12 15:51:48 +08:00
bd59c913bc [CI] ci-fetch-log.sh: fetch all failed jobs from a build URL or PR number (#45274) Michael GoinandClaude Fable 5 2026-06-12 08:42:18 +01:00
04cec9e4d8 [XPU][DeepSeek-V4] Fix MTP: sync with upstream fixes #44821 and #43746 (#45240) Ma Jianandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-12 15:41:36 +08:00
87b98d6d6c [Rust Frontend][Bugfix] Forward --shutdown-timeout and --disable-log-stats to the managed Python engine (#45300) Will Eaton 2026-06-12 03:39:27 -04:00
0cd9b7af25 [CPU] Support CPU W4A16 INT4 MoE (#43409) Yuwen Zhou 2026-06-12 15:12:37 +08:00
a2c72d4388 [Bugfix] Fix Dockerfile dependency graph pre-commit error (#45374) Isotr0py 2026-06-12 15:10:18 +08:00
fe04238292 [ROCm][gpt-oss] Pass GateMode.INTERLEAVE for MXFP4 W4A16 fused MoE (#44893) Rohan Potdar 2026-06-12 01:02:04 -05:00
39dee1114a [MM][Perf][CG] Support ViT full cudagraphs for mllama4 (#40660) allgatherandIsotr0py 2026-06-11 22:17:55 -07:00
eb28452b10 [Model] Add DiffusionGemma Support (#45163)
Lucas Wilkinson Martin Kukla Matthew Bonanni Dipika Sikka NickLucche jiahanc Alec Kohlhoff Porras Huang Claude scoootscooob
2026-06-12 01:17:35 -04:00
1ce3cdc5c1 [ROCm][CI] fix fp8 support for test_deepep_moe (#45302) Divakar Verma 2026-06-12 01:16:14 -04:00
6fbfdd1831 [NIXL] Per-region KV transfer classification for mixed full-attn + MLA groups (#44583) Dao007forever 2026-06-11 21:42:41 -07:00
7021be66e8 [11a/n] Migrate Marlin kernels to torch stable ABI (#45176) Chris Leonard 2026-06-12 00:22:37 -04:00
226ba9fc9e [ASR] Add Long Audio benchmark and correctness test (#44587) Ekagra Ranjan 2026-06-12 00:11:16 -04:00
b927004c44 [Bugfix] Mamba CPU Offloading (#44599) Varun Sundar Rabindranathandvarun sundar rabindranath 2026-06-12 00:07:35 -04:00
e0b9fb1290 [ASR] Optimize CPU preproc to get 2.5x RTFx via multi-threading (#44612) Ekagra Ranjanandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-12 00:05:11 -04:00
42ae5e7ac6 [Bugfix] Fix --enable-prompt-tokens-details omitting zero cached tokens (#44383)
sasindharan Sasindharan Sankar Chauncey
2026-06-12 09:07:42 +05:30
2263f8a3de [CI][BugFix] Fix broken test_mamba_prefix_cache.py due to stale mock (#45345) Nick Hill 2026-06-11 20:26:17 -07:00
c1076839c9 [Bugfix][Model] Pass revision by name in Run:ai and bitsandbytes index downloads (#45308) Ting SUN 2026-06-12 11:21:46 +08:00
fcf5115c45 [ROCm][DSv4][Perf] Flash-decode split-K decode attention kernel (#44899) Fangzhou AiandvLLM Contributor 2026-06-11 23:17:52 -04:00
4bc83323f2 [Bugfix] OffloadingConnector: respect skip_reading_prefix_cache flag (#44592)
littlecircle0730 Hsiao-Yuan Chen Or Ozeri
2026-06-11 19:20:39 -07:00
e0871ad225 [Refactor] Chat Completions Streaming Harmony Refactor and Bugfixes (#45104) yzong-rh 2026-06-11 21:09:47 -04:00
6f573f486b [Bugfix] Initialize missing attributes in mistral eagle (#45217) jpwangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-12 08:21:01 +08:00
9bbf42be26 Make mistral_common optional by deferring MistralToolCall import (#45305) Neil Schemenauer 2026-06-11 15:59:11 -07:00
8a91228dbe [Bugfix][KVConnector][Mooncake] Close MooncakeDistributedStore on connector teardown (#45206) Dao007foreverandClaude 2026-06-11 14:33:48 -07:00
f712fd0d7d [Refactor] Chat Completions Harmony Refactor, non-streaming path. (#45171) yzong-rh 2026-06-11 17:18:30 -04:00
5a6c7b7ab5 [Bug] Fix test flashmla for DSv4 (#45052) Wentao Ye 2026-06-11 16:22:26 -04:00
c9340e6f35 [Model] Remove InternLMForCausalLM registry alias (#45128) Tiezhen WANGandClaude 2026-06-12 04:02:51 +08:00
235b63c004 [Bugfix] Fix Anthropic tool_use content handling dropping args (#45287) Ben Browning 2026-06-11 16:01:29 -04:00
3b03a2cf47 [Rust Frontend] Support continuous_usage_stats stream option (#43965) Chao-Ju ChenandBugen Zhao 2026-06-12 01:50:59 +08:00
b8142294b7 [Bugfix] Restrict FlashInfer cuDNN FP8 ViT attention gate to Blackwell (SM 100) (#45251) wentian-byte 2026-06-12 00:39:24 +08:00
2ec6594db9 [Kernel][Helion][1/N] Add Helion kernel for per_token_group_fp8_quant (#36902)
Xiaohong (Sean) Chen Yanan Caomergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-11 11:59:08 -04:00
79f8c5bd8c [Metrics] Scope unregister_vllm_metrics() to strictly "vllm:" metrics (#42331) vraiti 2026-06-11 11:43:14 -04:00
f81daf8880 [Attention] add triton diff-kv backend for mimo (#41797) Jiangyun Zhu 2026-06-11 23:36:31 +08:00
4085ff7cb4 [Core] Add kvcache watermark to reduce preemptions (#44594) Nick HillandClaude Opus 4.8 2026-06-11 08:27:31 -07:00
23eb7c8fbb [Bugfix] Fix NixlEPAll2AllManager's dependency on --enable-elastic-ep to function (#44422) fangyuchuandTyler Michael Smith 2026-06-11 23:14:49 +08:00
c2b4cd39ac [Doc][Attention] Fix MLA top-of-file comments (#37047) wineandchord 2026-06-11 23:14:45 +08:00
f1d8d99717 [Bugfix] CohereModel.load_weights: skip modelopt _quantizer.* keys (#43495) Kai K. 2026-06-11 17:14:21 +02:00
750aab5b8e [Bugfix] Fix CPU memory leak related to not cleaning up old remotes data (#44424) Nicolò Lucchesi 2026-06-11 16:54:52 +02:00
5edf7ff489 [Core] Release cached device memory under pressure on UMA GPUs during weight loading (#45179) Michael GoinandClaude 2026-06-11 15:49:50 +01:00
b78fc47f05 [Docs] Add redirect for moved lmcache examples page (#45218) Natalie LinandHarry Mellor 2026-06-11 22:41:08 +08:00
03878d1c22 Deprecations for v0.23 and v0.24 (#44992) Harry Mellor 2026-06-11 15:35:38 +01:00
55911db580 [PD][Core] Fix Mamba prefix cache hit rate in PD disaggregation (#44243)
zhanqiuhu lHrHenry233 underfituu
2026-06-11 10:10:25 -04:00
cc640ee8bc [Rust Frontend][Metrics] Export vllm:lora_requests_info from frontend (#45030)
Will Eaton Bugen Zhaomergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-11 09:45:03 -04:00
ebc6ef971a Hidden states extraction improvements (#43805) Fynn Schmitt-Ulmsandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-11 09:44:45 -04:00
ab3a1fd2e6 minicpmv4_6: fix ImageSize (W,H) order for placeholder token calculation (#45244) tc-mb 2026-06-11 21:43:56 +08:00
c3662b36ea [KV offload] Parallel-agnostic fs-tier cache for single full-attention group (#44733) Itay EtelisandItay Etelis 2026-06-11 15:48:37 +03:00
e62d00ab73 docs: add fix disclosure policy to SECURITY.md (#45253) Juan Pérez de Algaba 2026-06-11 14:48:00 +02:00
1f60771c74 fix: guard flash-attn rotary import (#42679) Yufeng HeandWentao Ye 2026-06-11 20:43:31 +08:00
05d9848267 [Build] Upgrade CUDA Dockerfiles from GCC 10 to GCC 12 for C++20 compatibility (#44923) Richard BarnesandShengqi Chen 2026-06-11 08:26:52 -04:00
ef67071b21 [Build] Skip spinloop extension on Python < 3.11 (#44783) jasen 2026-06-11 19:23:21 +08:00
3508cb78d4 [Bugfix] Fix broken profile_modular_kernel.py (#43300) x41lakazam 2026-06-11 14:17:23 +03:00
432905d5d6 Only enable PR docs builds manually (#45262) Harry Mellor 2026-06-11 11:14:29 +01:00
1f9dd7900d [Bugfix][Rust Frontend] Validate out-of-vocab token ids in request params (#44680) Ting SUNandBugen Zhao 2026-06-11 18:14:11 +08:00
9492362972 [Security] Apply sanitize_message to Anthropic and STT error paths (#45119) Juan Pérez de Algabaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-11 12:05:34 +02:00
7852e50e4d [docs] Document --scheduler-cls base class requirement (extend AsyncScheduler, not Scheduler) (#43724)
Georgii Kliukovkin Claude Sonnet 4.6 Harry Mellor
2026-06-11 02:49:51 -07:00
0d657e44dc [Rust Frontend] Fix DeepSeek V3.2 continue_final_message rendering (#45155) Reid 2026-06-11 17:34:19 +08:00
aa1df36c53 Fix/minicpmv46 missing version (#44980) 王金旭andCursor 2026-06-11 17:20:45 +08:00
f06aefb4e3 [CPU] Add missing scalar fallback for CPU W4A8 INT4 GEMM (#44523) wcyandlyd1992 2026-06-11 16:52:01 +08:00
1c3a72b8b2 [Bugfix] Add fetch_images to MistralCommonImageProcessor (#45180) Julien Denize 2026-06-11 10:13:01 +02:00
d598d23973 [Security] Reject non-finite temperature and repetition_penalty values (#45116) Juan Pérez de Algaba 2026-06-11 10:12:14 +02:00
f219788f91 [Security] Fix info disclosure via int32 truncation in GGUF dequantize kernels (#44971) Juan Pérez de Algaba 2026-06-11 10:05:14 +02:00
6e64c1bab1 [10c/n] Migrate MoE kernels to torch stable ABI (#44565) Chris LeonardandShengqi Chen 2026-06-11 02:02:26 -04:00
2f2c5cf4f1 [release] Always block release images to dockerhub (#45236) Kevin H. Luu 2026-06-10 22:53:04 -07:00
40e065e86a [Docker] Fix CUTLASS DSL cu13 install order in Dockerfile (#45204) Mohammad Miadh Angkad 2026-06-11 13:19:36 +08:00
0b995f8609 Use std::bit_cast for type punning in CPU kernels (#45089) Yuanyuan ChenandLi, Jiang <jiang1.li@intel.com> 2026-06-11 13:07:44 +08:00
43914dd743 [Rust Frontend] Add Python bridge for Rust tool parsers (#44624) Bugen Zhao 2026-06-11 12:51:06 +08:00
3501324957 [Build] fix self-contradictory precompiled-flag orthogonality test (#44942) Prajjwal ChittoriandShengqi Chen 2026-06-11 10:19:08 +05:30
3a04061701 [Refactor][Parser] Unify Response API to use parser.parse() like Chat Completion API (#45190) Flora Feng 2026-06-11 00:37:51 -04:00
f272dfdce1 [KV Connector] Mooncake store: prefix-cache retention interval for sparse attention (#44774) Yifan Qiao 2026-06-10 21:36:34 -07:00
f31bc2ea60 [CPU][RISC-V] Enable oneDNN W8A8 INT8 to run on RISC-V (#44478) velonica0 2026-06-11 12:09:05 +08:00
248e33c40d [Bugfix][Responses API] Set id on function_call item in streaming done event (#44608) ankrovvandFlora Feng 2026-06-10 20:52:42 -07:00
5d5591d99b [Rust Frontend] Populate cached_token_count in responses (#44887) Bugen Zhao 2026-06-11 11:50:05 +08:00
85a0ffae42 [CI Bug] Remove qwen test ValueError: No example model defined for Qwen/Qwen-7B-Chat (#45194) Wentao Ye 2026-06-10 23:11:00 -04:00
18d87a87dc Deprecate Transformers v4 support (#45161) Harry Mellor 2026-06-11 04:04:01 +01:00
b038a2f73b [CI][Bugfix] Update Dockerfile dependency graph PNG (#45209) Flora Feng 2026-06-10 22:40:25 -04:00
2d481f8a94 [Bugfix][Rust Frontend] Stop unescaping XML-style tool-call parameter values (#45025) Ting SUN 2026-06-11 10:05:23 +08:00
7920ccb97c [Bugfix]: Fix Quark gpt-oss weight loading broken by FusedMoe refactor (#45067) Rohan PotdarandAndreas Karatzas 2026-06-10 20:17:46 -05:00
86111c00c7 [Chore] Add Github notification for MRv2 for @yewentao256 (#45191) Wentao Ye 2026-06-10 21:01:49 -04:00
e2db0222e9 [Perf][Attention] Pin MLA chunked-context metadata tensors so H2D copies are truly non-blocking (#45074) qizixi 2026-06-10 15:56:49 -07:00
82d6b59f04 [CI/Build] Skip test_use_trtllm_attention on non-CUDA platforms (#44687)
Dan Blanaru Wentao Yemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-11 00:18:42 +02:00
16282a9c4e [ROCm][CI] Moving MI300 tests to MI325 until cluster is stabilized (#45170) Andreas Karatzas 2026-06-10 15:26:17 -05:00
5b6b536fdc [ROCm][Bugfix] Make intermediate_pad TP-aware in rocm_aiter_fused_experts (#44679) Rohan PotdarandAndreas Karatzas 2026-06-10 15:10:50 -05:00
12f3f19c19 feat(qwen3-asr): support prompt parameter in v1/audio/transcriptions (#35415)
Nathan Price Cursor Cyrus Leungmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-10 14:54:59 -05:00
6471ec75bd [EPLB] Reject NCCL-based EPLB communicators with async EPLB (#44978) Ilya Markov 2026-06-10 21:51:27 +02:00
3d300aecb1 [Doc] Switch K8S examples to default MP mode (#39400)
Peter Pan Harry Mellor Kyle Sayers Flora Feng
2026-06-11 02:17:11 +08:00
ffce72c041 [Model Runner V2] Fix v2 AttributeError: 'CohereASRDecoder' object has no attribute 'embed_input_ids' (#44568) Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-10 14:06:01 -04:00
bfe1001ab6 [Bugfix] [DSV4] [ROCm] Pin apache-tvm-ffi version to 0.1.10 (#45169) TJian 2026-06-11 01:41:15 +08:00
fa8c868a3c [Bugfix] Fix Llama4 weight loading (#45047) Tyler Michael SmithandClaude Opus 4.6 2026-06-10 13:40:45 -04:00
d1bcb4b44c [Bugfix] Fix tool parsing crash with non-function tool types (e.g. WebSearchTool) (#45147) Ben Browning 2026-06-10 13:17:16 -04:00
29026682cb [Bugfix] Fix nemotron accuracy drop introduced by #41184 (#45037) bnellnm 2026-06-10 13:16:25 -04:00
dc66e01a70 [Hybrid] Marconi-style admission policy for hybrid cache (#37898) Stan Wozniak 2026-06-10 19:03:13 +02:00
2ba68d9bf7 [Test] Fix one-sided MNNVL alltoall test workspace under-reservation (#44946) Yongye Zhu 2026-06-10 12:43:12 -04:00
2131b597b1 [CI] Ping Mistral team for ministral/voxtral/mixtral/pixtral changes (#45153) Julien Denize 2026-06-10 17:48:00 +02:00
0bae1d3848 [MRV2][Spec Decode] DFlash (#44586) Benjamin ChislettandGiancarlo Delfin 2026-06-10 11:47:46 -04:00
4673ca1d78 fix: prefix DeepSeek V4 MTP projections (#44821) Yufeng He 2026-06-10 23:47:04 +08:00
de900fa7e5 fix: AOT compile cache collision for dataclass-based HF configs (#45059) Angela Yi 2026-06-10 08:05:29 -07:00
166d14e9bf [bugfix] skip conch kernel for g_idx reordering (#45072) Divakar Verma 2026-06-10 11:04:19 -04:00
af65e08fc5 KV-Cache multi-tier offloading async batched lookup (#44193) Effi OferandOr Ozeri 2026-06-10 17:59:30 +03:00
3cc9fecd58 Deprecated 1st generation Qwen and QwenVL models (#45131) Harry Mellor 2026-06-10 15:55:33 +01:00

... 57 58 59 60 61 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86