iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,837 Commits 3 Branches 0 Tags
143e4dccdfd8293c70c76f8d32a60ce23ecc23ea
Commit Graph
10555 Commits
Author SHA1 Message Date
Isotr0py 143e4dccdf [Misc] Add online audio_in_video test (#36775) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-15 00:14:11 -07:00
Isotr0py 6590a3ecda [Frontend] Remove torchcodec from audio dependency (#37061) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-15 05:15:59 +00:00
Nick Hill 458c1a4b2d [Frontend] Reduce chat template warmup logging levels (#37062) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-03-14 13:48:59 -07:00
Karan BansalandInokinoki 821fde2df4 [Bugfix] Fix xgrammar dtype mismatch on macOS CPU inference (#32384) 
Signed-off-by: Karan Bansal <karanb192@gmail.com>
Co-authored-by: Inokinoki <inoki@inoki.cc>
 2026-03-14 17:29:06 +00:00
arlo 8c29042bb9 [Feature] Add InstantTensor weight loader (#36139) 2026-03-14 18:05:23 +01:00
Cyrus Leung 5467d137b3 [Frontend] Avoid startup error log for models without chat template (#37040) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-14 09:36:11 -07:00
Santino Ramos 3ed46f374b [Model Runner V2] Add Support for XD-RoPE (#36817) 
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com>
 2026-03-14 09:27:55 -07:00
seanmamasde 84868e4793 [Bugfix][Frontend] Fix audio transcription for MP4, M4A, and WebM formats (#35109) 
Signed-off-by: seanmamasde <seanmamasde@gmail.com>
 2026-03-14 08:44:03 -07:00
Isotr0py a8e8d62dd8 [Misc] Clean up Kimi-audio whisper encoder loading (#36903) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-14 23:37:52 +08:00
Julien Denizerootgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
 		e42b49bd69 Mistral common v10 (#36971) 
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
Co-authored-by: root <root@h200-bar-196-227.slurm-bar-compute.tenant-slurm.svc.cluster.local>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-03-14 07:26:43 -07:00
Sergey Zinchenko 4a718e770d [Bug] Fix Failure in /v1/chat/completions/render for Multimodal Requests (https://github.com/vllm-project/vllm/issues/35665) (#35684) 2026-03-14 14:10:11 +00:00
Harry Mellor ffa5d74f15 Enable loading of fused expert weights in the Transformers modelling backend (#36997) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-14 07:01:06 +00:00
sbeurnier a116f96930 [V1] Remove pin_memory() in async_copy_to_gpu to fix sporadic stalls (#37006) 
Signed-off-by: Sebastien Beurnier <sbeurnier@together.ai>
 2026-03-14 01:37:32 +00:00
Li, Jiangandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 092ace9e3a [UX] Improve UX of CPU backend (#36968) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Li, Jiang <bigpyj64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-14 09:27:29 +08:00
Andrew Xiagemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Andrew Xia
 		f680dc1b39 [responsesAPI] prioritize content over summary in reasoning item input (#36516) 
Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: Andrew Xia <mitandrewxia@gmail.com>
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2026-03-14 09:20:30 +08:00
Giulio Leonegiulio-leoneCopilot
 		b41aa264f9 fix: resolve chat template names before kwargs detection (#36937) 
Co-authored-by: giulio-leone <giulio.leone@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-03-14 00:20:16 +00:00
Dimitrios BariamisandDimitrios Bariamis 367cf5cd3e [Feat][Bugfix] Enable additional dimension for Flashinfer MLA and fix routing dtype (#36931) 
Signed-off-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Co-authored-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
 2026-03-13 16:41:16 -07:00
haosdent 6d53efd2a5 [Bugfix] Fix MLA attention crash with AWQ/GPTQ quantized models (#34695) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-03-13 23:25:41 +00:00
Benjamin Chislett 8b346309a5 [Refactor] Consolidate SupportsEagle (#36063) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-03-13 23:22:40 +00:00
Nick Hill 54a6db827f [BugFix] Fix "DP Coordinator receives unexpected..." messages (#37008) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-03-13 23:18:05 +00:00
Matthew Bonanni 9efc4db965 [Bugfix] Fix DeepSeek-V3.2 tokenizer stripping spaces (#37004) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-13 22:55:36 +00:00
Harry Mellor 0005d2a3c9 Use Transformers v5 WeightRenaming for Transformers modeling backend (#31545) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-13 20:49:08 +00:00
Ekagra Ranjan d0b402974f [Bugfix][Spec Decode] Avoid double call of Ngram CPU (#36952) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-03-13 20:33:19 +00:00
Divakar Verma 6341d43043 [ROCm][Quantization] add quark w4a8 mxfp4_fp8 for LinearLayer (#35316) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-03-13 19:44:24 +00:00
Mark McLoughlinClaude Sonnet 4.5Nick Hill
 		7afe0faab1 [Frontend][Core] Re-add shutdown timeout - allowing in-flight requests to finish (#36666) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-03-13 12:10:06 -07:00
Harry Mellor 5a3f1eb62f [Misc] Set default kv_buffer_device in a better way (#36862) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-13 19:07:33 +00:00
yugong333 b3ce711b93 Fp8 lora dense kernel (#35242) 
Signed-off-by: Yu Gong <yu3.gong@gmail.com>
 2026-03-13 19:05:08 +00:00
Isotr0py abf61aaa8e [Bugfix] Fix Qwen2.5-omni/Qwen3-omni mm_processor cache for audio_in_video request (#36800) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-13 18:16:05 +00:00
bigmoyanwangzhengtaogemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		4508532fbd [Bugfix] fix paddleocr crash on some image shape (#36959) 
Signed-off-by: wangzhengtao <wangzhengtao@msh.team>
Signed-off-by: bigmoyan <moyan_work@foxmail.com>
Co-authored-by: wangzhengtao <wangzhengtao@msh.team>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-13 13:46:55 +00:00
Itay AlroyYongji WuRon Tourgeman
 		d5af196c18 [2/N] Elastic EP Milestone 2: Integrating NIXL-EP (#35627) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Yongji Wu <wuyongji317@gmail.com>
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com>
 2026-03-13 09:25:33 -04:00
Chaojun Zhang 82f836d976 [XPU] Support LoRA via torch.compile on XPU platform (#36962) 
Signed-off-by: chzhang <chaojun.zhang@intel.com>
 2026-03-13 10:34:59 +00:00
Or Ozeri cfaf4668f7 [kv_offload+HMA][1/N]: Support multiple KV groups in OffloadingSpec (#36610) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-03-13 08:04:21 +00:00
Sage a2268617cf [Frontend] Delegate preprocessing to OpenAIServingRender (#36483) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-03-13 00:39:43 -07:00
Rohan Potdar a4ad9db541 Enable RoPE+KV cache fusion for ROCm AITER FA (non-shuffle layout) (#35786) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-03-13 07:33:22 +00:00
Thomas Parnell f296a1966d [Bugfix] Fix FlashInfer GDN warmup ValueError on SM90 GPUs (#36876) 2026-03-13 07:09:39 +01:00
Csrayz bc2c0c86ef [Frontend] Fix usage incorrectly returned with empty stream_options` (#36379) 
Signed-off-by: Csrayz <33659823+Csrayz@users.noreply.github.com>
 2026-03-13 03:33:04 +00:00
jaime campos salas 891c60dcd5 fix(kv-cache): increase hybrid attention grouping threshold from 1.25 to 1.5 (#36684) 
Signed-off-by: Jaime Campos Salas <jaime.campos.salas@gmail.com>
 2026-03-12 23:28:27 -04:00
whyiugandwang.yuqi 1ce13cf992 [Model] Add support for BERT-like Chinese ERNIE pooling models (#36385) 
Signed-off-by: whyiug <whyiug@hotmail.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-03-13 03:23:53 +00:00
NikitaandCyrus Leung 10f08dedfa [Model] Add ColPali late interaction model for multi-modal retrieval (#36818) 
Signed-off-by: Nikita Sukharev <kaonael@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-03-13 02:18:57 +00:00
Aaron Hao 5e1a373d2e [BUG] Fix rank calculation in NCCLWeightTransferEngine (#36940) 
Signed-off-by: hao-aaron <ahao@anyscale.com>
 2026-03-13 01:56:51 +00:00
Yifan Qiao 55d8073d06 [Bugfix] ep_scatter kernel store-load race condition (#34991) 
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
 2026-03-13 01:07:59 +00:00
Nick Hill cd32d6f586 [Model Runner V2] Some code simplification (#36929) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-03-13 00:59:23 +00:00
Jaewon aaa3092f51 [MoE] Add routing simulation override for MXFP4 quantized MoE (#33595) 
Signed-off-by: Jaewon Lee <jaewon@meta.com>
 2026-03-13 00:30:44 +00:00
Shubhra PanditBenjamin ChislettBenjamin Chislett
 		87985077a4 [Speculative Decoding] Add norm_before_fc for gpt-oss draft models (#36545) 
Signed-off-by: Shubhra Pandit <shubhra.pandit@gmail.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-03-12 23:03:32 +00:00
Xinan MiaoandSouthWest7 2cdf92228c [Feature]: Remove Chunking From FusedMoE (#34086) 
Signed-off-by: SouthWest7 <am1ao@qq.com>
Signed-off-by: Southwest <1403572259@qq.com>
Signed-off-by: southwest <am1ao@qq.com>
Signed-off-by: Xinan Miao <1403572259@qq.com>
Co-authored-by: SouthWest7 <am1ao@qq.com>
 2026-03-12 14:24:38 -04:00
Eunkwang Jeon bdc2343454 [Bugfix] Fix KeyError in parse_response_input for reasoning items with optional content (#34499) 
Signed-off-by: jeonsworld <jeonsworld@gmail.com>
 2026-03-13 00:13:36 +08:00
Matthew Bonanni f444c05c32 [Attention] Use FA4 for MLA prefill (#34732) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-12 12:10:17 -04:00
grimulkan a1257fd1ea [Kernel] Add FP8 KV cache support to Triton MLA decode attention (#34597) 
Signed-off-by: grimulkan <grimulkan@gmail.com>
 2026-03-12 08:32:34 -07:00
Thomas ParnellandClaude Opus 4.6 abcffbba8c [CI] Fix mypy pre-commit errors on main (#36882) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-12 08:22:29 -07:00
Kunshang Ji 53ec16a705 [Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-12 07:57:47 -07:00