Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

ff5c60fad8 [Misc] Automatically tag PRs to add new models (#20222) Isotr0py 2025-07-03 22:11:03 +08:00
6f1229f91d [Model][2/N] Automatic conversion of CrossEncoding model (#19978) wang.yuqi 2025-07-03 21:59:23 +08:00
1819fbda63 [Quantization] Bump to use latest bitsandbytes (#20424) Jee Jee Li 2025-07-03 21:58:46 +08:00
7f0367109e [CI/Build][CPU] Enable cross compilation in CPU release pipeline (#20423) Li, Jiang 2025-07-03 20:26:12 +08:00
fb14d53cf6 [Kernel] refactor cpu worker v0 cache dtype (#20080) Ning Xie 2025-07-03 16:39:14 +08:00
b024a42e93 [Core] Move multimodal placeholder from chat utils to model definition (#20355) Cyrus Leung 2025-07-03 16:18:30 +08:00
cb97f2bfc5 [Docs] Replace two list with tables in intel_gaudi.md (#20414) Michael Yao 2025-07-03 15:48:25 +08:00
359200f6ac [doc] fix link (#20417) Reid 2025-07-03 15:21:57 +08:00
220aee902a [Misc] Add rules to label Speculative Decoding Related PRs (#20406) Lifans 2025-07-02 23:56:49 -07:00
67d25eca05 [Tests] Update online DP tests to verify that requests are balanced (#20157) Nick Hill 2025-07-03 07:49:13 +01:00
363528de27 [Feature] Support MiniMax-M1 function calls features (#20297) qscqesze 2025-07-03 14:48:27 +08:00
4ff61ababa [TPU] Add a case to cover RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w8a8 (#20385) QiliangCui 2025-07-02 23:46:41 -07:00
0ec3779df7 [Bugfix][CI/CD][CPU] Fix CPU CI tests (#20383) Li, Jiang 2025-07-03 11:11:36 +08:00
b616f6a53d [Misc] Small: Fix video loader return type annotations. (#20389) Chenheli Hua 2025-07-02 20:10:39 -07:00
2e25bb12a8 [Bugfix] Fix import of CutlassExpertsFp8 in compressed_tensors_moe.py (#20381) bnellnm 2025-07-02 22:07:43 -04:00
9965c47d0d Enable CPU nightly performance benchmark and its Markdown report (#18444) Louie Tsai 2025-07-02 18:50:25 -06:00
059d4cdb49 [BugFix] Fix DP headless mode arg validation (#20398) Nick Hill 2025-07-03 01:15:32 +01:00
bdb84e26b0 [Bugfix] Fixes for FlashInfer's TORCH_CUDA_ARCH_LIST (#20136) Tyler Michael Smith 2025-07-02 20:15:11 -04:00
3dd359147d [Docs] Update EAGLE example (#20375) Nicolò Lucchesi 2025-07-03 02:13:51 +02:00
657f2f301a [DP] Support external DP Load Balancer mode (#19790) Nick Hill 2025-07-02 18:21:52 +01:00
a1aafc827a [ROCm][FEAT] Enable Full Graph Mode in AITER MLA V1 Attn Backend (Decode Phase only) (#20254) vllmellm 2025-07-03 00:25:46 +08:00
139508a418 [Misc] add handler HF_TOKEN is emptry string (#20369) rongfu.leng 2025-07-03 00:14:31 +08:00
d265414dbc [Minor] Clean up incorrect comment in test (#20382) Nick Hill 2025-07-02 17:13:37 +01:00
48fb076cbc [V1] LogitsProcessor programming model (#16728) afeldman-nmandNick Hill 2025-07-02 12:10:42 -04:00
c1909e7e8c [Kernels] MoE refactor (#19636) bnellnmandElizaWszola 2025-07-02 09:08:27 -04:00
b95877509b Documentation update tool_calling: mapping back to function from response (#20373) cronoik-inceptionai 2025-07-02 16:55:49 +04:00
706ff13224 [Model] Adds support for SlimMoE models Phi-tiny-MoE-instruct (#20286)
zichongli5Zichong Li <t-lizichong@microsoft.com@Reasoning-H100-VM3.drbuo4tcjzruhloch3eo0b25ef.cx.internal.cloudapp.net>Isotr0py
2025-07-02 05:54:12 -07:00
ccbfb1d1c9 [Bugfix] Fix the max_seq_len limit of 16384 for DeepSeek models (#20322) WangHuaqiang 2025-07-02 20:53:36 +08:00
9e5552aa13 [NVIDIA] Support Cutlass w8a8 FP8 for Blackwell Geforce GPUs (sm120) (#17280) Joonchen Liauandmgoin 2025-07-02 20:47:19 +08:00
0c600b9ab6 [Build/CI] Automatically tag DeepSeek related PRs (#20370) Lu Fang 2025-07-02 20:02:43 +09:00
e303dcf523 [Model] Add Ernie4.5 and Ernie4.5MoE Model Support (#20220) CSWYF3634076 2025-07-02 18:37:01 +08:00
ae9c4d416f [Docs] Make TPU ref prettier in google_tpu.md (#20356) Michael Yao 2025-07-02 17:04:08 +08:00
d853520b3e [Docs] Fix indentations for 2-level items in deprecation_policy.md (#20352) Michael Yao 2025-07-02 14:50:31 +08:00
ba51aea65e [Bugfix] Keye-VL compatibility with tok_kwargs (#20058) (#20353) Cyrus Leung 2025-07-02 14:46:59 +08:00
8452946c06 [Model][VLM] Support Keye-VL-8B-Preview (#20126) Kwai-Keye 2025-07-02 14:35:04 +08:00
2e7cbf2d7d [Frontend] Support configurable mm placeholder strings & flexible video sampling policies via CLI flags. (#20105) Chenheli Hua 2025-07-01 23:34:03 -07:00
7da296be04 [TPU] kv cache update kernel supports dynamic grid (#20235) Chengji Yao 2025-07-01 23:33:37 -07:00
b205e8467d [Doc][TPU] Add models and features supporting matrix. (#20230) QiliangCui 2025-07-01 23:33:20 -07:00
be0cfb2b68 fix[Docs]: link anchor is incorrect #20309 (#20315) yyzxw 2025-07-02 14:32:34 +08:00
1a03dd496b [Bugfix] Fix dynamic rotary embedding (#20343) Cyrus Leung 2025-07-02 14:31:26 +08:00
27b8017636 [FIX][Intel GPU]fix ipex flash_attn_varlen_func api missing parameter (#20348) Kunshang Ji 2025-07-02 13:26:40 +08:00
9ec1e3065a [Misc][Doc] Add missing comment for LLM (#20285) Lifans 2025-07-01 19:04:24 -07:00
9dae7d46bf [Refactor] Remove Unused Env VLLM_ENABLE_MOE_ALIGN_BLOCK_SIZE_TRITON (#20334) Wentao Ye 2025-07-01 22:03:43 -04:00
7058d7dd5d [Refactor] Remove duplicate find_free_port (#20333) Wentao Ye 2025-07-01 22:03:07 -04:00
a0389e0554 [UT][intel GPU] use current_platform instead of device hardcode in v1 tests (#20169) Liangliang Ma 2025-07-02 09:06:04 +08:00
3be8d312a2 [Kernel][Bugfix] Fixup some warnings in nvfp4_blockwise_moe when CUDA < 12.8 (#20324) Tyler Michael Smith 2025-07-01 21:05:47 -04:00
3abfe22154 Enable group size 64 for Machete (#20290) czhu-cohere 2025-07-01 18:05:44 -07:00
e81fbefe8a [Refactor] Refactor import utils (#20269) Wentao Ye 2025-07-01 21:05:42 -04:00
9290de5667 remove unused variables in marlin_template.h (#20236) 周周周 2025-07-02 08:51:52 +08:00
7f280d69c9 [Optimization] Cache sampled token ids in model runner (#20291) Woosuk Kwon 2025-07-01 11:01:31 -07:00
02cabff207 [V1] [ROCm] Enable EP with AITER Fused MoE (#20270) TJian 2025-07-01 09:48:30 -07:00
3d19d47d91 [Frontend] Expand tools even if tool_choice="none" (#17177) Shintarou Okada 2025-07-02 01:47:38 +09:00
8acb4badee [CUDA graphs] Enable full cuda graphs with FA3 AoT scheduling (#20301) Woosuk Kwon 2025-07-01 09:07:36 -07:00
314af8617c [Docs] Update transcriptions API to use openai client with stream=True (#20271) Nicolò Lucchesi 2025-07-01 17:47:13 +02:00
0e96cc9b7e [Misc] Minor refactoring for scheduler (#20299) Woosuk Kwon 2025-07-01 07:55:32 -07:00
ecad851cbd [Model]Add Tencent HunYuanMoEV1 Model Support (#20114)
aiyiwang2025 quinnrong Jee Jee Li
2025-07-01 22:28:13 +08:00
ed70f3c64f Add GLM4.1V model (Draft) (#19331) Yuxuan ZhangandIsotr0py 2025-07-01 20:48:26 +08:00
650d5dbd04 [Misc] Minor refactor of NIXL background handshake (#20068) Nicolò Lucchesi 2025-07-01 13:40:14 +02:00
9025a9a705 [Quant] [Bugfix] Fix quantization config matching with hf_to_vllm_mapper (#20046) Kyle Sayers 2025-07-01 06:20:34 -04:00
c05596f1a3 [Perf] Validate @config in pre-commit instead of dynamically (#20200) Lionel Villard 2025-07-01 05:10:28 -04:00
787b13389e [doc] fix the incorrect logo in dark mode (#20289) Reid 2025-07-01 16:18:09 +08:00
96453cfa83 [BugFix][V1][ROCm] Triton MLA uses V0 backend on V1 engine (#19067) TY-AMD 2025-07-01 16:12:19 +08:00
b1c1fe35a5 [Misc] remove redundant char (#20287) Kebe 2025-07-01 15:33:22 +08:00
08d81f1014 [Bugfix] Fix deepep tests (#20288) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-07-01 03:29:08 -04:00
6cc1e7d96d [CPU] Update custom ops for the CPU backend (#20255) Li, Jiang 2025-07-01 15:25:03 +08:00
9909726d2a Enable ZP Support for Machete (#20268) czhu-cohere 2025-07-01 00:12:20 -07:00
22e9d42040 [Misc] add xgrammar for arm64 (#18359) Prashant Gupta 2025-07-01 00:02:20 -07:00
86debab54c Fix numel() downcast in vllm/csrc/moe/moe_align_sum_kernels.cu +2 (#17082) Richard Barnesandmgoin 2025-07-01 00:48:10 -06:00
be250bbc67 [V1] Only print cudagraph tqdm on rank 0 with is_global_first_rank (#19516) Michael Goin 2025-07-01 15:02:09 +09:00
27949354fa [Feature] A calibration-free RTN-based quantization for accurate and accelerated INT4/INT8 inference (#18768) Alex KoganandMichael Goin 2025-07-01 01:44:38 -04:00
bd5038af07 [Doc] add config and troubleshooting guide for NCCL & GPUDirect RDMA (#15897) Ernest Wong 2025-06-30 21:44:39 -07:00
a2f14dc8f9 [CI][Intel Gaudi][vllm-Plugin]Add CI for hpu-plugin-v1-test (#20196) Chendi.Xue 2025-06-30 23:17:07 -05:00
92ee7baaf9 [Example] add one-click runnable example for P2P NCCL XpYd (#20246) Kuntai Du 2025-06-30 21:03:55 -07:00
7151f92241 [Misc] Fix spec decode example (#20296) Woosuk Kwon 2025-06-30 21:01:48 -07:00
e28533a16f [Bugfix] Fix include prompt in stream response when echo=true (#15233) fyuan1316 2025-07-01 09:30:14 +08:00
6d42ce8315 [CLI] Improve CLI arg parsing for -O/--compilation-config (#20156) Luka Govedič 2025-06-30 21:03:13 -04:00
ded1fb635b [Bugfix][V1][P/D]Fix the issue of occasional garbled output for P2pNcclConnector (#20263) Zhonghua Deng 2025-07-01 07:45:14 +08:00
97d9524fe9 [Refactor] Remove useless pdb comment (#20266) Wentao Ye 2025-06-30 14:15:24 -04:00
d8cf819a9a [Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058) Kyle Sayers 2025-06-30 13:26:49 -04:00
551ef1631a [Unit Test] Add unit test for deep gemm (#20090) Wentao Yeandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-06-30 12:26:42 -04:00
2863befce3 [Optimization] Use Shared CachedRequestData Instance Across All Requests (#20232) Woosuk Kwon 2025-06-30 09:07:50 -07:00
2965c99c86 [Spec Decode] Clean up spec decode example (#20240) Woosuk Kwon 2025-06-30 08:28:13 -07:00
2062c0723d [Spec Decode] Refactor spec decoding into a separate function (#20238) Woosuk Kwon 2025-06-30 08:13:50 -07:00
1c50e100a9 [Bugfix] fix quark ptpc (#20251) li haoyangandHaoyang Li 2025-06-30 21:24:50 +08:00
3ee56e26be [Docs] Fix 1-2-3 list in v1/prefix_caching.md (#20243) Michael Yao 2025-06-30 19:20:51 +08:00
8fe7fc8634 [Quantization] Improve BitsAndBytesModelLoader (#20242) Jee Jee Li 2025-06-30 18:22:09 +08:00
e936e401de [Bugfix] Fix processor initialization in transformers 4.53.0 (#20244) Isotr0py 2025-06-30 18:16:16 +08:00
f5dfa07531 [Bugfix] Skip loading extra parameters for modelopt Qwen3 MoE model (#19598) noiji 2025-06-30 18:21:56 +09:00
022c58b80f [doc] Add Slack and Forum to the top navigation (#20208) Reid 2025-06-30 15:53:45 +08:00
19108ef311 [Misc] Fix import (#20233) Woosuk Kwon 2025-06-29 20:34:54 -07:00
5a52f389dd [BUGFIX][DEEPSEEK][MODEL_LOAD] fix w13, w2 weight not initialized assert (#20202) Chendi.Xue 2025-06-29 21:46:19 -05:00
65b1cbb138 [Model] support dots1 (#18254) redmoe-moutain 2025-06-30 10:34:36 +08:00
6c9837a761 Fix cuda_archs_loose_intersection when handling sm_*a (#20207) Huy Do 2025-06-29 16:52:34 -07:00
6f2f53a82d [Quantization] Add compressed-tensors NVFP4 MoE Support (#19990) Dipika Sikka 2025-06-30 00:05:40 +02:00
7b1895e6ce [CI Fix] Try fixing eagle e2e test OOM by reducing block allocation (#20213) Michael Goin 2025-06-29 11:31:37 +09:00
4d36693687 [Refactor] Create a function util and cache the results for has_deepgemm, has_deepep, has_pplx (#20187) Wentao Ye 2025-06-28 18:06:38 -04:00
daec9dea6e [Bugfix] Correct behavior of GraniteMoeHybrid for TensorParallel execution (#20137) Stan Wozniak 2025-06-28 17:16:41 +02:00
daceac57c7 [Frontend] Generalize v1/audio/transcriptions endpoint (#20179) Nicolò Lucchesi 2025-06-28 17:15:26 +02:00
8615d9776f [CI/Build] Add new CI job to validate Hybrid Models for every PR (#20147) Thomas Parnell 2025-06-28 08:00:25 +02:00
7b460c25f9 [BugFix] Fix the incorrect func name in the comments. (config.py) (#20185) Jiayi Yan 2025-06-28 13:51:16 +08:00

... 158 159 160 161 162 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86