iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,991 Commits 3 Branches 0 Tags
c19c263ac8ea3fd6e1d2d689f0e4e725855d89cf
Commit Graph
344 Commits
Author SHA1 Message Date
Subhash Polisetti f425a21208 [Bugfix][Frontend] Support scalar priority and wire priority into LLM.chat() (#57243) 
Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
 2026-10-11 10:03:33 +00:00
wang.yuqi c1692c85f1 [Misc] mv chat_utils.py to renderers. (#60522) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-10-10 06:50:34 +00:00
wang.yuqi 459764104d [Frontend] Move offline_utils.py to entrypoints/common. (#58052) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-10-08 09:56:05 +00:00
Harry MellorandClaude Opus 5.5 21d9758459 Remove Python 3.10 support now that it is EOL (#60402) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 02:30:52 -07:00
wang.yuqiaoshen02Claude Opus 5.5
 		4187ade7b8 [Frontend] Consolidate rl entrypoints. (#57849) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 21:37:11 -07:00
czhu-cohereandaoshen02 6fddc9b21e [Profiler] Add per-session profiling controls to Python and Rust frontends (#57875) 
Signed-off-by: czhu <conway.zhu@cohere.com>
Signed-off-by: Conway Zhu <conway.zhu@cohere.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-06 05:21:19 +00:00
Ronaldaoshen02Claude Opus 5.5
 		e5f3d08d72 [Frontend][RL] Align sleep-mode API responses and operation metrics (#52864) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 23:16:38 -07:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
Harry MellorandClaude Opus 5.5 d636456b1f [Frontend] Remove the slow tokenizer mode (#58545) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 08:48:28 -07:00
Dakai AnAlanFokCoaoshen02OpenAI Codex
 		092bdd6d57 [Frontend][Core] Add release_kv_cache_memory() API (#44890) 
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-18 07:12:12 +08:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
vx120aoshen02CodexClaude Opus 4.6OpenAI Codex
 		50ba4bc6b2 [Feature] Mask Replay (#49577) 
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-13 08:44:31 +00:00
wang.yuqi f5ffc59b6a [Renderer] Warm up the renderer properly. (#50408) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-07-31 11:46:53 +00:00
Shuolei Wang 9069a57139 [Core][Frontend] Add weight version tagging for RL rollouts (#49040) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
 2026-07-28 14:35:29 +08:00
wang.yuqi 3e90d015ba [Frontend] Overlap preprocessing and computation for pooling models offline inference (#47699) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-07-16 14:00:20 +00:00
wangxiyuan 94c0ef3001 [Misc] Clean up "swap_space" (#48549) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-07-14 04:43:45 +00:00
vx120crp0128aoshen02Claude Opus 4.8
 		fc1c548093 Runtime Draft Weight Update for Speculative Decoding (#46725) 
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: crp0128 <191679376@qq.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-11 22:51:53 -07:00
Aaron HaoandClaude Sonnet 4.6 77a9c5ae28 Weight sync refactor + move sparse nccl engine (#44353) 
Signed-off-by: hao-aaron <ahao@anyscale.com>
Signed-off-by: haoaaron <ahao@anyscale.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-07-01 01:25:19 -07:00
Sumanth R Hegde 7ff7f5c8eb Revert "Fix Stale Encoder Cache After Weight Update" (#46125) 2026-06-20 07:09:09 -07:00
littlecircle0730 08985351f3 Fix Stale Encoder Cache After Weight Update (#45093) 
Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com>
 2026-06-18 09:32:10 +00:00
Vic Wen ef3af56a97 Fix LLM.wait_for_completion output type docstring (#44617) 
Signed-off-by: viiccwen <viiccwen@gmail.com>
 2026-06-05 00:16:38 -07:00
wang.yuqi d01d0b4646 [Frontend] Consolidate online serving utils. (#44479) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-06-04 06:49:31 +00:00
Siddharth BedekarandOpenAI Codex 266b9d9c64 [Frontend][Core] Add sparse NCCL weight transfer support for in-place updates (#40096) 
Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-06-01 15:37:30 -04:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> d5cf7b4a2c [Frontend] Split the offline inference APIs and utils. (#43553) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-26 05:20:24 +00:00
Daoyuan Li a6682d1d25 [Bugfix] Warn when renderer_num_workers has no effect on offline LLM (#42905) 
Signed-off-by: Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com>
 2026-05-20 21:35:08 -07:00
pmaybankgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		fadf5d332c add enqueue all option to throughput benchmark (#42975) 
Signed-off-by: Philip Maybank <pmaybank@amd.com>
Signed-off-by: pmaybank <113125070+pmaybank@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-19 20:16:02 -07:00
wang.yuqi 301d986473 [Frontend] Consolidate beam search by BeamSearchMixin. (#42946) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-19 07:37:40 +00:00
Michael GoinandClaude 239b5ff30c [Frontend] Add --spec-method/--spec-model/--spec-tokens CLI aliases (#42476) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-18 17:22:27 -07:00
wang.yuqi 75fd68c7a5 [Entrypoints] Split the pooling offline API into PoolingOfflineMixin. (#42267) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-15 08:05:57 +00:00
Michael Goin 8efd508204 [Quantization] Rework quantization_config to use QuantKey and allow for activation override (#41566) 2026-05-13 16:58:32 -04:00
Sumanth R Hegde e3b65a5ba0 [feat] Add explicit /start_weight_update and /finish_weight_update APIs for weight transfer (#39212) 2026-05-08 18:03:33 -07:00
Luis 🚀andAndrew Sansom 14043dfecd feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API (#40720) 
Signed-off-by: Luis Robaina <luis@protopia.ai>
Signed-off-by: Luis Robaina 🚀 <luisfabian1545@gmail.com>
Signed-off-by: LuisRobaina <luis@protopia.ai>
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com>
 2026-05-01 10:05:55 +08:00
wang.yuqi a8208e6a81 [Examples] Resettle features examples. (#40995) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-28 00:33:41 -07:00
wang.yuqigemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
 		9744b699ba [Deprecate] Deprecate LLM.reward offline api, use LLM.encode instead. (#40688) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-04-24 05:37:50 +00:00
Matthew BonanniandTyler Michael Smith 96a85c5750 [Startup][UX] Enable CUDAGraph memory profiling by default (#38284) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-04-21 18:16:59 -04:00
wang.yuqiCyrus Leungmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		d2e2e856ad [Frontend] Remove frontend pooling multi task support. (#37861) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-21 12:27:44 +00:00
wuyingjun 77fd2c8631 [Bugfix] Forward mm_processor_kwargs in offline generate APIs (#40251) 
Signed-off-by: wuyingjun <wuyingjun_yewu@cmss.chinamobile.com>
 2026-04-20 00:56:56 -07:00
Luciano MartinsLuciano MartinsFlora Feng
 		38907e4391 [Frontend] Preserve structured output special tokens in offline LLM.chat (#39352) 
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-04-18 19:46:07 -04:00
daiyu1111 a302a8fd1b [Bugfix] Fix LLM priority normalization for single-string prompts (#40011) 
Signed-off-by: daiyu1111 <2356690121@qq.com>
 2026-04-16 07:56:06 -07:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 4e8c3f1c19 [Frontend][last/5] Improve pooling entrypoints | clean up. (#39675) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-16 07:53:23 -07:00
wang.yuqi 66c079ae83 [Frontend][4/n] Improve pooling entrypoints | pooling. (#39153) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-09 10:09:45 +00:00
Vasiliy Kuznetsov 7b1a7423be [Frontend] new online quantization frontend (#38138) 
Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com>
 2026-04-03 11:58:39 -04:00
wang.yuqi a9b4f07ba2 [Frontend] Re-enable running MaxSim on GPU (#38620) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-03 00:03:13 +08:00
wang.yuqi d9d21eb8e3 [Frontend][3/n] Improve pooling entrypoints | scoring. (#28631) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-03-31 07:52:00 +00:00
Cyrus Leung ba2f0acc2d [Misc] Reorganize inputs (#35182) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-25 10:22:54 -07:00
Harry Mellor d215d1efca [Mypy] Better fixes for the mypy issues in vllm/config (#37902) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-25 06:14:43 -07:00
wang.yuqigemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
 		1b6cb920e6 [Deprecate] Deprecate pooling multi task support. (#37956) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-03-24 14:07:47 +00:00
wang.yuqi ed359c497a [Model] Deprecate the score task (this will not affect users). (#37537) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-03-20 08:07:56 +00:00
wang.yuqi a3189a08b0 [Model] Consolidate score logic by introduce score_type (#36479) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-03-10 13:32:25 +00:00
wang.yuqi fff3711a24 [Frontend][2/n] Improve pooling entrypoints | embed. (#36110) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
 2026-03-09 11:42:19 +08:00