iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
13,454 Commits 3 Branches 0 Tags
a2ef06e1b3a1c0dc4f7bf6587fbdcf1dad287c31
Commit Graph
760 Commits
Author SHA1 Message Date
Michael Goin 29fba76781 [UX] Use gguf repo_id:quant_type syntax for examples and docs (#33371) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-01-31 12:14:54 +08:00
Isotr0py 9df152bbf6 [Misc] Algin Qwen3-VL-embedding image example outputs with HF repo example (#33419) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-30 19:36:56 -08:00
Patrick von PlatenandNick Hill 10152d2194 [Realtime API] Adds minimal realtime API based on websockets (#33187) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-01-30 18:41:29 +08:00
hujiaxin0andEmilie1001 ba45bedfd1 [model] Add support for openPangu7B-VL (#32449) 
Signed-off-by: hujiaxin <524446785@qq.com>
Signed-off-by: Emilie1001 <79921183+Emilie1001@users.noreply.github.com>
Co-authored-by: Emilie1001 <79921183+Emilie1001@users.noreply.github.com>
 2026-01-30 15:54:27 +08:00
Harry Mellor 9432ed8c7e Explicitly set return_dict for apply_chat_template (#33372) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-01-30 07:27:04 +00:00
Ryan RockandTJian 070c811d6f [CI][AMD] Skip 4 GPUs testgroup ray tests (#33305) 
Signed-off-by: Ryan Rock <ryan.rock@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-01-29 21:39:53 -08:00
Wang Haoyu c46b0cd0af [Model][Multimodal] Add explicit MusicFlamingo adapter (#32696) 
Signed-off-by: WangHaoyuuu <mailwhaoyu@gmail.com>
 2026-01-30 11:01:29 +08:00
Roger Wang 8b3f0a99dd [Models] Qwen3-ASR (#33312) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-01-29 19:27:15 +08:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> abb34ac43a [Bugfix] Fix Qwen3-VL-Reranker load. (#33298) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-29 08:42:53 +00:00
ramosandmayufeng 36d450e3b8 Adds FunAudioChat multimodal audio model support (#2) (#33058) 
Signed-off-by: ramos <49182011+nemoramo@users.noreply.github.com>
Signed-off-by: mayufeng <mayufeng@example.com>
Co-authored-by: mayufeng <mayufeng@example.com>
 2026-01-28 05:18:09 +00:00
Harry Mellor 2eb673a088 Add flake8-implicit-str-concat rules to Ruff (#33191) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-01-28 04:56:10 +00:00
Jared Wen 6ee7f18f33 [Logging] add --disable-access-log-for-endpoints CLI option (#30011) 
Add a new CLI option --disable-access-log-for-endpoints to suppress
uvicorn access logs for specified endpoints (e.g., /health, /metrics, /ping).

This addresses the need to reduce log noise in production environments
where health check endpoints are frequently polled by load balancers or
monitoring systems, generating excessive log entries that obscure
meaningful request logs.

Fixes #29982

Signed-off-by: JaredforReal <w13431838023@gmail.com>
 2026-01-26 21:49:03 +00:00
Yuxuan ZhangandIsotr0py bb17e8f11c [GLM-OCR] GLM-OCR with MTP Support (#33005) 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-26 06:24:43 -08:00
Itay EtelisandItay Etelis 6ca2c91b96 [Model] Use mm_position to compute mrope positions for Qwen3-Omni (#33010) 
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
 2026-01-26 13:48:07 +00:00
ltd0924andluotingdan b40db4dfec [StepVL] add step vl offline example (#33054) 
Signed-off-by: luotingdan <luotingdan@stepfun.com>
Co-authored-by: luotingdan <luotingdan@stepfun.com>
 2026-01-26 01:00:32 -08:00
Cyrus Leung 11b556878b [Refactor] Use data parser for matching data items to multi-modal UUIDs (#32955) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-26 15:00:28 +08:00
Itay EtelisandItay Etelis a698e8e7ad [Model] Use mm_position to compute mrope positions for Qwen2.5-Omni (#32772) 
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
 2026-01-25 20:15:53 +08:00
Mark McLoughlin 1cb4341fbc [ROCm][PD] Remove unused moriio connector proxy code (#32939) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-01-23 15:59:04 +00:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 05f3d714db [Frontend][3/n] Make pooling entrypoints request schema consensus | EmbedRequest & ClassifyRequest (#32905) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-23 12:03:44 +00:00
wang.yuqi 328cbb2773 [Frontend][2/n] Make pooling entrypoints request schema consensus | ChatRequest (#32574) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-01-22 10:32:44 +00:00
Robert ShawandRobert Shaw cea3c754c4 [Quantization][Deprecation] Remove DeepSpeedFp8 (#32679) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-01-21 09:32:12 -05:00
Kim Hee Su 7727ce35c2 [Model] Add Eagle2.5-8B Vision-Language Model support (#32456) 
Signed-off-by: kimheesu <wlskaka4@gmail.com>
 2026-01-21 09:39:53 +00:00
Cyrus Leung 09194b90a5 [Doc] Update docs for MM model development with context usage (#32691) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-20 10:37:35 -08:00
杨朱 · KikiandClaude Haiku 4.5 bb9172030e [Metrics] Complete removal of deprecated vllm:time_per_output_token_seconds metric (#32661) 
This PR completes the removal of the deprecated vllm:time_per_output_token_seconds
metric that was deprecated in v0.11, hidden in v0.12, scheduled for removal in v0.13,
but delayed until v0.15.

Signed-off-by: carlory <baofa.fan@daocloud.io>
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
 2026-01-20 12:28:41 +00:00
Cyrus Leung 4753f3bf69 [Model] Use context managers for encoder- and LM-only mode (#32605) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-20 11:43:38 +08:00
Tomas Ruiz 4a5299c93f feat: spec decode with draft models (#24322) 
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
 2026-01-19 16:05:46 -05:00
wang.yuqi c88860d759 [Frontend] Score entrypoint support data_1 & data_2 and queries & documents as inputs (#32577) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-01-19 14:07:46 +00:00
Isotr0py 38bf2ffb21 [Bugfix] Fix GLM-ASR audio encoder RoPE dim (#32540) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-18 19:17:59 +08:00
wang.yuqi 4ae77dfd42 [Frontend][1/n] Make pooling entrypoints request schema consensus | CompletionRequest (#32395) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-01-16 06:17:04 +00:00
sangho.leeandIsotr0py 7e6f123810 Add Molmo2 multimodal model support (#30997) 
Signed-off-by: sanghol <sanghol@allenai.org>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-14 15:33:09 +08:00
HappyAmazonian 2f4a71daf2 [Misc] Add In-Container restart capability through supervisord for sagemaker entrypoint (#28502) 
Signed-off-by: Shen Teng <sheteng@amazon.com>
Signed-off-by: HappyAmazonian <91216626+HappyAmazonian@users.noreply.github.com>
 2026-01-13 13:06:10 -08:00
Cyrus Leung 232214b2ae [Bugfix] Replace PoolingParams.normalize with use_activation (#32243) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-13 10:45:42 +00:00
Jaehyun An 6bc9c8473e [MODEL] New model support for kakaocorp/kanana-1.5-v-3b-instruct (#29384) 
Signed-off-by: Jaehyun An <steve.ai@kakaocorp.com>
 2026-01-12 16:39:02 +00:00
Isotr0py 9dbe1fe960 [Bugfix] Fix missing scale passing for encoder Triton Attention implementation (#32149) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-12 11:13:41 +00:00
wang.yuqi 60446cd684 [Model] Improve multimodal pooling examples (#32085) 
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-01-12 07:54:09 +00:00
Ning Xie d74132ca3b fix offline inference chat response prompt (#32088) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-01-11 14:01:18 +00:00
Ning Xie 14fc7a68c7 [Bugfix] fix offline chat output prompt (#32076) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-01-10 07:50:57 +00:00
Matthew Bonanni 2612ba9285 [1/N][Attention] Restructure attention: move files (#31916) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-09 13:10:24 -08:00
Isotr0py 2d0c5b630e [Doc] Remove hardcoded Whisper in example openai translation client (#32027) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-09 14:44:52 +00:00
inkcherry 4505849b30 [ROCm][PD] add moriio kv connector. (#29304) 
Signed-off-by: inkcherry <mingzhi.liu@amd.com>
 2026-01-09 14:01:57 +00:00
tianshu-Michael-yuRoger WangCyrus Leung
 		03fd76c570 [Model] Add LFM2-VL model support (#31758) 
Signed-off-by: Tianshu Yu <tianshuyu.formal@gmail.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-01-08 05:00:27 -08:00
Isotr0py eac3b96ec0 [Models] Allow converting Qwen3-VL into Reranker model (#31890) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-08 08:10:15 +00:00
wang.yuqi 96860af655 [Model] rename use_pad_token to use_sep_token (#31784) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-01-06 14:16:04 +00:00
Cyrus Leung da71d44410 [Doc] Show that use_audio_in_video is supported in docs (#30837) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-05 23:27:19 -08:00
baonudesifeizhai 02dbb933cb Fix GLM-4.6v flash tool calling in transformers 5.x (#31622) 
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com>
 2026-01-05 11:32:43 -08:00
wang.yuqiandCyrus Leung 911d38ed99 [Model] Let more models to support the score template. (#31335) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-01-05 11:54:26 +00:00
labAxiaoming a01f2faedf Add multimodal input method in the documentation (#31601) 
Signed-off-by: xiaoming <1259730330@qq.com>
 2026-01-02 12:43:30 +00:00
Ekagra Ranjan adcf682fc7 [Audio] Improve Audio Inference Scripts (offline/online) (#29279) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2025-12-31 23:34:18 +00:00
baonudesifeizhaiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> d722e9e614 Add GLM-ASR multimodal support (#31436) 
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com>
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-31 23:12:24 +08:00
SageandSage 39512aba72 [Prefix Cache] Include lora_name in BlockStored event for deterministic KV-cache reconstruction (#27577) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
Co-authored-by: Sage <80211083+sagiahrac@users.noreply.github.com>
 2025-12-30 00:17:16 +00:00