iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,722 Commits 3 Branches 0 Tags
382970ee6ca490aeaaaf4e32c53695b581ff61ba
Commit Graph
145 Commits
Author SHA1 Message Date
Karen Chung 986e217158 [Feature] Add first-class KV hints request envelope for programmatic KV management (#53423) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-09-21 09:56:11 -07:00
Dakai AnAlanFokCoaoshen02OpenAI Codex
 		092bdd6d57 [Frontend][Core] Add release_kv_cache_memory() API (#44890) 
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-18 07:12:12 +08:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Ning XieandCyrus Leung 2a336d8239 [warmup] overlap renderer warmup and engine core initialization (#54557) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-03 19:44:21 +00:00
Oxana KorzhandClaude Opus 5 fb68025138 [Bugfix][MRV2] Release model and KV cache on in-process engine shutdown (#54162) 
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-28 22:09:08 -07:00
Kevin H. Luu d04d3e0a07 [Bugfix] Revert renderer warmup overlap to avoid fork deadlock (#54023) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-08-27 16:36:33 -07:00
Ning Xie ca90b9e7d4 [warmup] overlap renderer warmup and engine core initialization (#52764) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-08-27 11:42:08 +08:00
Jee Jee LiandCyrus Leung 4f686e182a [MISC] Cleanup deprecated parameters (#53559) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-24 10:19:04 -07:00
Rishi PuriClaude Opus 4.8Nick Hill
 		dedbf6be8b [V1][Metrics] Preserve prefix-cache stats on zero-output steps (#48668) 
Signed-off-by: Rishi Puri <riship@nvidia.com>
Signed-off-by: Rishi Puri <puririshi98@berkeley.edu>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-09 01:58:38 +00:00
Karen Chung f57123aa2d feat(frontend): session id plumbing into requests (#48048) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-08-03 22:18:58 +00:00
Shuolei Wang 9069a57139 [Core][Frontend] Add weight version tagging for RL rollouts (#49040) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
 2026-07-28 14:35:29 +08:00
Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 7b5d60cc37 [Bugfix][V1] Clean up compiled-model bytecode hooks on VllmRunner exit (#45195) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-16 20:31:17 -07:00
Tobias Wasner 4658bf882b [Bugfix] Clear P0 mm sender cache on sleep/pause to fix mm_hash desync (#43001) 
Signed-off-by: Tobias Wasner <wasnertobias@gmail.com>
 2026-05-22 03:54:29 -07:00
Wentao Ye 1842447c09 [Refactor] Remove unused param (#39750) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-21 14:59:20 -07:00
wang.yuqi 66c079ae83 [Frontend][4/n] Improve pooling entrypoints | pooling. (#39153) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-09 10:09:45 +00:00
Cyrus Leung ba2f0acc2d [Misc] Reorganize inputs (#35182) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-25 10:22:54 -07:00
Augusto YaoandCyrus Leung 8e75d88554 add io_process_plugin for sparse embedding (#34214) 
Signed-off-by: augusto.yjh <augusto.yjh@antgroup.com>
Signed-off-by: Augusto Yao <augusto.yjh@antgroup.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-02-28 09:16:37 +00:00
Nick Hill dbf0da817a [Core] Cleanup engine pause/sleep logic (#34528) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-02-24 19:33:34 -08:00
Cyrus Leung 761e63e541 [Frontend] Always pass supported_tasks to validation (#35186) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-24 04:16:33 -08:00
Cyrus Leung ac900c89bb [Refactor] Implement output type check in LLM (#34794) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-19 19:57:55 -08:00
Cyrus Leung a766b30349 [Renderer] Deprecate code paths for old input processing (#34775) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-18 00:35:04 -08:00
Cyrus Leung 574fe75245 [Renderer] Move InputPreprocessor into Renderer (2/2) (#34560) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-17 05:29:01 -08:00
Cyrus Leung 73391a1baa [Renderer] Move InputPreprocessor into Renderer (1/2) (#34510) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-02-14 10:14:21 -08:00
Cyrus Leung 2f308214c0 [Refactor] Pass full VllmConfig to Renderer (#34485) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-12 22:48:38 -08:00
JaewonandLu Fang 4453ba8d9e [Core] Profiler improvements and lazy initialization (#33198) 
Signed-off-by: Jaewon Lee <jaewon@meta.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
 2026-02-12 16:16:38 -08:00
JaewonandLu Fang aa181c923b [Core] Add sleep level 0 mode with enqueue/wait pattern (#33195) 
Signed-off-by: Jaewon Lee <jaewon@meta.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
 2026-02-12 16:16:25 -08:00
Cyrus Leung b96f7314b4 [Refactor] Pass Renderer to Input Processor (#34329) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-11 19:38:11 -08:00
Cyrus Leung cd8b405bd0 [Refactor] Consolidate sequence normalization and enc-dec parsing (#33928) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-06 15:43:47 +00:00
emricksini-h 325ab6b0a8 [Feature] OTEL tracing during loading (#31162) 2026-02-05 16:59:28 -08:00
Cyrus Leungandwang.yuqi 038914b7c8 [Refactor] Move task outside of PoolingParams.verify (#33796) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-02-05 09:33:11 +00:00
Cyrus Leung a502831d36 [Chore] Remove redundant input parsing methods (#33542) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-02 10:50:47 +00:00
Cyrus Leung a358e4dffe [Refactor] Make Renderer an abstract class (#33479) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-01 10:36:30 +08:00
jma99_2333andRoger Wang 22d9a056d5 Support clear mm and encoder cache (#33452) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-01-31 15:22:25 +00:00
Cyrus Leung d117a4d1a9 [Frontend] Introduce Renderer for processing chat messages (using ModelConfig) (#30200) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-22 12:44:22 +00:00
Ning Xie 9d7ae3fcdb [code clean] remove duplicate check (#32376) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-01-15 05:29:34 +00:00
Nick Hill bd877162eb [BugFix] Support online dense model DP without overhead (#30739) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: njhill <nickhill123@gmail.com>
 2026-01-02 23:36:38 +08:00
Cyrus Leung 09dc7c690c [Chore][1/2] Drop v0.14 deprecations (#31285) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-24 09:54:01 -08:00
Mark McLoughlinandNick Hill f790068600 [Core] Add a random suffix to frontend-provided request IDs (#27987) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-12-23 13:05:39 -08:00
Cyrus Leung 39cefbdf17 [Refactor] TokenizerRegistry only uses lazy imports (#30609) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-13 23:16:22 +08:00
Cyrus Leung e72d65b959 {Deprecation] Remove tokenizer setter (#30400) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-10 19:10:58 +00:00
Cyrus Leung 253305d5b2 [Chore] Delay recent deprecations (#30398) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-10 17:48:38 +00:00
Cyrus Leung e83b7e379c Revert "[Renderer] Separate out RendererConfig from ModelConfig (#30145)" (#30199) 2025-12-07 00:00:22 -08:00
Cyrus Leung 27f4c2fd46 [Renderer] Separate out RendererConfig from ModelConfig (#30145) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-06 23:15:42 -08:00
Tova Movshovitzandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> adb315060c [KVConnector][Feature] Support KV connector cache reset via /reset_prefix_cache (#27170) 
Signed-off-by: tovam <tovam@pliops.com>
Signed-off-by: Tova Movshovitz <tovam@pliops.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-05 18:33:26 +00:00
Yongtao Huang 2fc5d6e0d7 Fix LLMEngine.del dp_group cleanup condition (#29954) 
Signed-off-by: Yongtao Huang <yongtaoh2022@gmail.com>
 2025-12-03 12:14:44 -08:00
Cyrus Leung 653591d5e7 [Chore] Move tokenizer initialization methods (#29793) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-02 13:33:37 +08:00
Zhuohan LiandNick Hill d0cd728907 [Core] Support reseting all running requests' KV while calling reset_prefix_cache (#28827) 
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-12-02 02:25:05 +00:00
Cyrus Leung 34a984274e [Misc] Refactor tokenizer interface (#29693) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-29 04:02:21 -08:00
Cyrus Leung 8d9338fae4 [Chore] Rename Processor to InputProcessor (#29682) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-28 09:35:41 -08:00
Avishek Goswami 32c40b95e0 [BugFix] bad_words filtering ineffective when n > 1 (#29313) 
Signed-off-by: GOavi101 <1704178@kiit.ac.in>
 2025-11-25 09:36:34 +00:00