Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

10f4db4dbe [Frontend] Add Support for MM Encoder/Decoder Beam Search (Offline) (#36153) Alex Brooksandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-03-06 02:16:56 -07:00
5b3ba94ab4 [Core][KVConnector] Support HMA+NixlConnector (#35758) Nicolò Lucchesi 2026-03-06 08:51:21 +01:00
90f3c01fa4 [Spec Decode][KV Connector] Fix KV transfer in PD + speculative decoding (#35158) zhanqiuhuandNicolò Lucchesi 2026-03-06 02:50:44 -05:00
807d680337 [ROCm][CI] Fix tool use test stability - disable skinny GEMM, prefix caching, eliminate batch variance (#35553) Andreas Karatzas 2026-03-06 01:15:12 -06:00
5afb387bd4 Change "following fields were present in the request but ignored" log from warn to debug (#36173) Tyler Michael Smith 2026-03-06 01:15:46 -05:00
43e77e59ab [BugFix] avoid infinite loop with VLLM_PORT and get_open_ports_list (#36191) Walter Beller-Morales 2026-03-06 01:15:29 -05:00
00bd08edee [Security] Respect user trust_remote_code setting in NemotronVL and KimiK25 (#36192) Russell Bryant 2026-03-06 01:15:19 -05:00
43f10573c9 [Bugfix] Fix misleading context length error messages (#36197) Ajay AnuboluandClaude Opus 4.6 2026-03-05 22:15:12 -08:00
86e1060b17 [Bugfix] Fix inner_dp_world initialization order for multi-node TP (#35892)
Yongye Zhu Claude Opus 4.5 Nick Hill Nick Hill
2026-03-06 01:04:44 -05:00
27066d1b2b [Frontend][Core] Add shutdown timeout - allowing in-flight requests to finish (#34730) Mark McLoughlinandClaude Sonnet 4.5 2026-03-06 06:04:31 +00:00
57c84ff129 perf: add __slots__ to KVCacheBlock (#36164) cong-or 2026-03-06 06:04:09 +00:00
e68de8adc0 docs: fix wrong cc in int8.md (#36209) Xiang Shi 2026-03-06 14:01:02 +08:00
a1ffa56a1e [CI] Fix bge-m3 similarity reference values after *Defination* typo fix (#36208) Andreas Karatzas 2026-03-05 23:07:29 -06:00
0a208d1f54 [BugFix] Fix engine hanging after KV cache initialization failure (#35478) Shiyan DengandLu Fang 2026-03-05 20:58:09 -08:00
03a49bb8f0 [Feature] Add --distributed-timeout-seconds CLI option (#36047) Shiyan DengandLu Fang 2026-03-05 20:57:51 -08:00
8e87cc57f1 [Bug] Fix a corner case in _process_simple_streaming_events (#34754) Shiyan DengandLu Fang 2026-03-05 20:57:32 -08:00
6dd302653f [Misc] Rename group_mm_kwargs_by_modality -> group_and_batch_mm_kwargs (#36158) Cyrus Leung 2026-03-06 12:32:48 +08:00
de00ebeac4 [Bugfix] Fix simple Mistral-Small example (#36156) Cyrus Leung 2026-03-06 12:25:11 +08:00
639680d220 [ROCm][CI] Adding missing dependencies for Multi-modal models tests (#36177) Andreas Karatzas 2026-03-05 22:23:10 -06:00
c5362c739f Reenable features for ROCm attention backends (#36185) Rohan Potdar 2026-03-05 22:21:06 -06:00
0a49676fb0 cpu: aarch64: Upgrade OneDNN for aarch64 to add support for int8 matmul (#36147) Nikhil Gupta 2026-03-06 03:48:59 +00:00
c012a8c477 Don't fire ray compatibility webhook when PR or branch is not provided (#36088) Jeffrey Wang 2026-03-05 16:42:21 -08:00
ebed80a7c8 [Performance] Extract KV-cache update from TreeAttention backend (#35384) Dor Huri 2026-03-06 02:22:43 +02:00
a73af584fe [Model Runner V2] Fix warmup for very small kvcache and/or blocksizes (#36176) Nick Hill 2026-03-05 14:48:10 -08:00
a97954b6a8 [compile] Consistent compiler config for saved/loaded vllm backends. (#35810) Zhengxu Chen 2026-03-05 15:08:12 -05:00
a911f4dd20 [Model] Add support for OLMo Hybrid (#32550) Yanhong Li 2026-03-05 11:51:06 -08:00
5395471d29 [CI] Add explicit permissions to macOS smoke test workflow (#35775) Russell Bryant 2026-03-05 14:08:48 -05:00
a57c877f18 [BugFix] Fallback from FA4->FA2 for Batch Invariance (#36059) Frank Wang 2026-03-05 11:05:56 -08:00
f917020983 [Perf] Optimize FusedMoEModularKernel output tensor using torch.empty (#35794) Xin Yang 2026-03-05 10:47:53 -08:00
86483ca774 [Bugfix] Disable FlashInfer TRTLLM BF16 path for non-gated MoE (#36146) tomeras91 2026-03-05 19:49:05 +02:00
b93a9e6f6d ParakeetProjection.norm = RMSNorm instead of nn.LayerNorm (#36133) Netanel Haber 2026-03-05 19:29:30 +02:00
d8839ef7d9 [XPU] Enable ModelRunnerV2 on XPU (#36078) Xinyu Chen 2026-03-06 01:19:18 +08:00
e998fa76b9 [BUGFIX]Fix Qwen-Omni models audio max_token_per_item estimation error leading to encoder_cache_size is 0 (#35994) Avery Miao 2026-03-06 01:16:29 +08:00
6a895197fa [Bugfix][CI] fix typos (#34934) Jiayi YanandHarry Mellor 2026-03-06 01:05:46 +08:00
8c760b6ab6 [ROCm] Refactor ROCm attention backend selection logic (#35246) Sage Moore 2026-03-05 08:51:26 -08:00
3ee68590c7 refactor funasr model. (#36108)
AllenDou zixiao Isotr0py
2026-03-06 00:07:37 +08:00
7196348157 [Bugfix] Fix Qwen-VL tokenizer implementation (#36140) Cyrus Leung 2026-03-06 00:07:19 +08:00
176c799f4c [openai api] log exception in exception handler (1/N) (#31164) Ning Xie 2026-03-06 00:00:12 +08:00
612e7729c2 [KVConnector] Scheduler: Fix num_computed_tokens after async KV load (#34616) Or Ozeri 2026-03-05 16:25:15 +02:00
ecde7af9c4 Fix import that was moved in Transformers 5.2.0 (#36120) Harry Mellor 2026-03-05 13:59:44 +00:00
8df523351f [Docs] Only build docs if documentation or ready labels are present (#36135) Harry Mellor 2026-03-05 13:58:16 +00:00
b03ff6a96b [CI] Stabilize test_no_args_tool_call and add ROCm-specific server args (#36107) Andreas Karatzas 2026-03-05 07:52:49 -06:00
ed81d5edd1 [Bugfix] Fix RunAI streamer crash with S3-hosted model paths (#35976) Ajay AnuboluandClaude Opus 4.6 2026-03-05 04:14:20 -08:00
3c23ac840e [Bugfix] Fix mypy errors in hermes_tool_parser.py (#36114) Shiyan Deng 2026-03-05 03:37:47 -08:00
a708ef5944 [Misc] Fix SyntaxWarning - invalid escape sequence '\e' (#36020) cjackal 2026-03-05 19:55:31 +09:00
66a2209645 [Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize (#36085) Kunshang Ji 2026-03-05 18:36:39 +08:00
0bfa229bf1 [Release] Include source distribution (sdist) in PyPI uploads (#35136) Doug SmithandDaniele Trifirò 2026-03-05 04:43:50 -05:00
7493c51c55 [Docs] add Dynamo/aibrix integration and kubeai/aks link (#32767) Paco Xu 2026-03-05 17:39:50 +08:00
ac773bbe80 [Docs] Update docs to include mm processor + encoder benchmarks (#34083) Reagan Lee 2026-03-05 01:38:25 -08:00
48e376a007 qwen3coder tool parser fix anyOf double encoded parameters (#36032) Christian Munley 2026-03-05 01:06:57 -08:00
21eb2c3372 [Chore] Correct MTP models test registry ordering (#36115) Isotr0py 2026-03-05 16:55:04 +08:00
e2b31243c0 [Docs] Update CacheConfig block_size docstring to remove inaccurate limit when using CUDA (#35632) Seiji Eicher 2026-03-04 22:24:08 -08:00
c3598d02fa [Misc] Remove deprecated items that are due for removal (#36006) Martin Hickey 2026-03-05 06:14:50 +00:00
57c629e9c1 [Bugfix] Fix block_size for hybrid model MTP (#36036) Benjamin Chislett 2026-03-05 01:10:54 -05:00
d106bf39f5 [Doc] Add Parallel Draft Models (#35973) zihaoanllmandHarry Mellor 2026-03-05 13:44:07 +08:00
b0651021e5 [Kernel] [Helion] [11/N] Retune configs for silu_mul_fp8 (#36062) Yanan Cao 2026-03-04 21:25:59 -08:00
f600d5192e [Bugfix] Fix score layer quantization for sequence classification models - Qwen3 (VL) Reranker (#35849) Hanjun Choandwang.yuqi 2026-03-05 13:57:20 +09:00
8e7820131e [Perf] Use dummy M for weight prepacking on x86 (#35890) Tianmu Li 2026-03-04 20:56:49 -08:00
0a12cea25f Order config.py in Lexicographical order (#35866) Andrii SkliarandAndrii Skliar 2026-03-05 05:56:47 +01:00
dd6dbd93f8 [compile] Fix extra cache save on warm start. (#35921) Zhengxu Chen 2026-03-04 23:56:30 -05:00
26366009c5 [CI] Don't leave docs preview comment on closed PRs (#36087) Harry Mellor 2026-03-05 04:51:46 +00:00
16c472abe7 [Core] Move ray-specific WorkerWrapperBase methods to RayWorkerWrapper (#35328) Nick Hill 2026-03-04 20:11:59 -08:00
3b23d57c96 [Model] Add LoRA support for Whisper models (#29856) daje0601andClaude Opus 4.5 2026-03-05 11:38:25 +09:00
2f4226fe52 [CI] Fix pre-commit mypy issue in main (#36049) Wentao Ye 2026-03-04 21:13:12 -05:00
792cbd64ca Add platform method to enable custom collective ops registration (#34760) nkm-meta 2026-03-04 16:50:32 -08:00
2ed4722e26 [compile] Reduce log spam from compile. (#36044) Zhengxu Chen 2026-03-04 19:48:36 -05:00
a3299c3d1d [Model Runner V2] Misc code simplification (#35941) Nick Hill 2026-03-04 15:26:35 -08:00
6c21a0c2d7 [ROCm][CI] Added MI325 mirrors (stage C) (#35239) Andreas Karatzas 2026-03-04 16:48:46 -06:00
562339abc3 [Misc] Support OOT linear method registering (#35981) Shanshan Shen 2026-03-05 06:25:56 +08:00
d7adcadb9b [Bugfix] Fix passing of activation_type to trtllm fused MoE NVFP4 and FP8 (#36017) amitz-nv 2026-03-05 00:23:51 +02:00
f678c3f61a [RL] [Weight Sync] Guard IPC update-info pickle deserialization behind insecure serialization flag (#35928) Simon MoandCursor Agent 2026-03-04 14:05:32 -08:00
be0a3f7570 [Bugfix] Fix race in non-blocking num_accepted_tokens GPU->CPU copy (#36013) Thomas ParnellandClaude Opus 4.6 2026-03-04 22:52:44 +01:00
17dc9c7fc9 [CI] Bump mypy version (#34950) Harry Mellor 2026-03-04 20:55:11 +00:00
7eca859110 Add PyTorch profiler schedule support with warmup/active iterations (#35240) fenypatel99 2026-03-04 12:53:38 -08:00
636ee223ac [Docs] Document security risks of GPT-OSS Python tool (#35139) Russell Bryant 2026-03-04 15:27:31 -05:00
b7d59ffce2 [UX] Remove NoOpOffloader log (#35678) Robert ShawandRobert Shaw 2026-03-04 15:13:40 -05:00
5569f5218d [torch.compile] Stop lazily compiling (#35472) Richard Zou 2026-03-04 15:13:17 -05:00
138d891d7f [Docs] Clarify structured outputs configuration for Qwen3 reasoning mode (#32441) Davina ZamanandHarry Mellor 2026-03-04 11:44:39 -08:00
d7166e74c1 [CI] Add Blackwell AsyncTP correctness test (#35871) Stefano Castagnetta 2026-03-04 20:41:21 +01:00
417fd28fb1 [Model Runner V2] Fix pooling (#36019) Nick Hill 2026-03-04 10:53:17 -08:00
7faba503c4 [Kernel][Mamba] Optimize Mamba2 SSD prefill Triton kernels (#35397) tomeras91 2026-03-04 20:47:17 +02:00
bc6be89d16 [Frontend] Add vllm launch command for GPU-less preprocessing serving (#34551) Hyunkyun Moon 2026-03-05 03:41:52 +09:00
32224f568a docs: update CPU Docker images to reference Docker Hub instead of AWS ECR (#34882)
Maxime Grenugemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Harry Mellor
2026-03-04 19:31:35 +01:00
f3dc292e9f docs: add version requirement note for --profiler-config flag (#32454) Abhishek Mathukiya 2026-03-04 13:13:54 -05:00
138c5fa186 [Docs] Add RunPod GPU deployment guide for vLLM (#34531) ChenandHarry Mellor 2026-03-04 12:11:34 -06:00
2f2c1d73a7 [Docs] Upgrade dynamic LoRA warning to admonition block (#35218) Russell Bryant 2026-03-04 13:01:42 -05:00
fb3e78ab09 [Feature][CI]: compare func & no_func outputs in test_functionalization.py (#35481) Bhuminjay SoniandLuka Govedič 2026-03-04 23:31:16 +05:30
fd3bfe74c9 [Docs] Update design/multiprocessing.md (#30677) Michael Yao 2026-03-05 01:58:59 +08:00
bfdb512f11 fix minicpmo4.5: fix attn_mask in vit attn && fix resampler pos_emb i… (#34127) tc-mbandhezhihui 2026-03-05 01:46:17 +08:00
d25c1ec3c9 docs(cpu): Clarify pre-built wheels requirement for CPU Python-only build (#35090) Sage 2026-03-04 19:45:35 +02:00
7cc6058ac6 [Doc] Add MTP docs and update speculative decoding guidance (#35197) Xing Liu 2026-03-05 01:23:34 +08:00
28028dff2f fix(docs): use static rdzv backend in multi-node troubleshooting script (#34784) Manrique Vargas 2026-03-04 12:15:35 -05:00
3417ba5648 docs: add README for logits_processor examples (#35933) Dr Alex Mitre 2026-03-04 11:09:19 -06:00
58cfe0dc44 Fix phi4-mm and remove cuda binding (#35964) Yan Ma 2026-03-05 01:08:05 +08:00
e86221deb6 [Doc] Fix GPU Worker count in Process Count Summary (#36000) simone-dotoloandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-03-04 18:03:14 +01:00
289fc48ab7 Use MMEncoderAttention (=use FlashAttention) instead of torch.sdpa in radio.py (#35653) Netanel Haber 2026-03-04 18:43:13 +02:00
2f2212e6cc Split generic IO Processor plugins tests from Terratorch specific ones (#35756) Christian Pinto 2026-03-04 16:01:03 +00:00
18e01a0a10 [Misc] Add --attention-backend auto option (#35738) Nicolò Lucchesi 2026-03-04 16:12:27 +01:00
6cb901093f [Core] Add All-to-All communication backend for DCP (#34883)
sungsoo ha Claude Opus 4.6 Lucas Wilkinson Harry Mellor
2026-03-04 07:01:57 -08:00
ead7bde1ab [Bugfix] Make kaldi_native_fbank optional (#35996) Cyrus Leung 2026-03-04 22:47:32 +08:00

... 87 88 89 90 91 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86