iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,722 Commits 3 Branches 0 Tags
382970ee6ca490aeaaaf4e32c53695b581ff61ba
Commit Graph
29 Commits
Author SHA1 Message Date
YukioZzz 9679173788 [KVConnector][MoRIIO] Transfer hybrid mamba/KDA recurrent state in READ mode (#51052) 
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
 2026-09-20 17:30:34 -04:00
Tianyu GuoOpenAI CodexOpenAI Codex
 		70fc7bd695 [Feature][EPD] Support dynamic EPD (EC-connector) proxy (#54176) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-20 12:24:41 +08:00
Tianyu GuoZhou zihengOpenAI Codex
 		0d72627565 [Perf][EPD] Batch image requests per encoder (#57095) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-17 03:45:52 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
jiangkuaixue123 1678b39627 [Bugfix][EPD] Preserve media processing options in encoder requests (#56786) 
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
 2026-09-14 10:05:13 +00:00
Tianyu Guo 319cc5ef19 [Performance][EPD] Reduce Python proxy serialization overhead (#56657) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-09-13 15:07:12 -07:00
junuxyz 372477d2d9 [Bugfix][Examples] Launch prefill and decode concurrently in NixlPushConnector demo (#55088) 
Signed-off-by: junuxyz <216036880+junuxyz@users.noreply.github.com>
 2026-09-12 06:22:34 +00:00
Tianyu Guo 479611c0c6 [Frontend][EPD] Use JSON arrays for multimodal metadata (#56090) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-09-10 05:04:14 +00:00
Teng MaCursorTianyu GuoZhou zihengjiangkuaixue123OpenAI Codex
 		6fbb00b188 [EPD] Add ECMooncakeConnector for encoder cache over Mooncake TransferEngine (#41567) 
Signed-off-by: Teng Ma <sima.mt@alibaba-inc.com>
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-07 06:21:43 +00:00
omerpaz95andClaude Opus 5 8a0a7ee40a [EC Connector] P2P NIXL + CPU EC Connector (#47941) 
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 11:23:56 +08:00
Isotr0py 555ea65e8c [Frontend] Add video embeds input support (#54242) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-31 03:20:47 +00:00
Ankit Nakhawa d66300a1ba [Bugfix][EPD] Fix encoder round-robin fan-out (#52491) 
Signed-off-by: AnkitNakhawa <ankitnakhawa@gmail.com>
 2026-08-20 07:30:40 +00:00
Tianyu Guo b30291cf0c [EPD] Remove duplicate image preprocessing in EPD and enable preprocess on GPU (#50390) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-08-05 15:59:54 -07:00
Tianyu Guo 50c51682a1 [Bugfix][EC Connector] Don't stop an encoder-instance request before its images are encoded (#50275) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-08-05 05:53:58 +00:00
Wentao Ye 4f819f801b [Model Runner v2] E/P/D disaggregation support (#38390) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-08-04 15:52:23 +00:00
shikamd123Douglas Lehrmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f42761204f llmd+vllm+mori-ep(inter node wide-ep)+mori-io(write) for 2p2d with dp=ep=16 tp=1 (#45043) 
Signed-off-by: Shiksha Patel <shikpate@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-04 00:13:15 +00:00
limewardClaudeQinPRPeiran Qin
 		c4cd2bd544 [Bugfix] MoRIIO toy P/D proxy: fix DP-rank index aliasing + harden for high-concurrency bursts (#46115) 
Signed-off-by: Edwin Lim <edwin.lim@mangoboost.io>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: QinPR <1905873179@qq.com>
Co-authored-by: Peiran Qin <66068739+QinPR@users.noreply.github.com>
 2026-07-17 12:35:04 -07:00
chaeminlim-mbEdwin LimJaeyoun KimEdwin Lim
 		520a20ba4e [Bugfix] MoRIIO toy P/D proxy: add /health (#45222) 
Signed-off-by: Chaemin Lim <chaemin.lim@mangoboost.io>
Signed-off-by: Edwin Lim <edwinlim0919@gmail.com>
Co-authored-by: Edwin Lim <edwin.lim@mangoboost.io>
Co-authored-by: Jaeyoun Kim <jaeyoun.kim@mangoboost.io>
Co-authored-by: Edwin Lim <edwinlim0919@gmail.com>
 2026-07-14 22:56:46 +00:00
qizixiandNick Hill 7802c20c4e [KVConnector][NIXL] Support pipeline-parallel prefill in push mode (#45880) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-08 16:49:23 -07:00
Chaitanya Sri Krishna Lolla 5fd442187c [ROCm][P/D] MoRIIO toy proxy: support JSON Content-Type for OpenAI clients. (#46482) 
Signed-off-by: lcskrishna <lollachaitanya@gmail.com>
 2026-07-01 19:17:05 +00:00
wang.yuqi 3483240b7e [Frontend] Consolidate scale out entrypoints (#44512) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-06-29 03:18:53 -07:00
Samuel Shen c9135db27c [Docs] Update stale LMCache examples (#45762) 
Signed-off-by: Samuel Shen <slshen@tensormesh.ai>
 2026-06-19 03:21:36 +00:00
snadampalandNicolò Lucchesi 88ed636218 [KV Connector]: Support KV push from Prefill to Decode node using Nixl KV Connector (#35264) 
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-06-12 10:38:41 +00:00
Change72 320c52b134 [Bench] benchmark_serving_multi_turn: make non-standard conversation_id payload opt-in (#43756) 
Signed-off-by: Change72 <cguo51@asu.edu>
 2026-06-09 19:41:56 -07:00
Nicolò Lucchesi 5add018beb [Connector] Remove P2pNcclConnector (#44854) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-06-08 18:58:29 +08:00
Wentao Ye ea0e501bb1 [KV Connector] Remove compat support for pre-v0.12.0 constructor signatures without KVCacheConfig (#39832) 
The v0.12.0 release contained initial support for HMA in KV Connectors. As part
of these changes, a KVCacheConfig argument was added to KV connector
constructors. Backwards compatibility support for out-of-tree connectors was
included in this change, with a very prominent warning. See #25712 and #27887.

Since the warning has been around for over 5 months, we can safely remove
the support of it.

Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-09 23:39:46 +00:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 1d694e78c9 [Examples][last/6] Resettle examples. (#41084) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-07 19:42:12 -07:00
tej 8a4888be21 [ROCm] Profiler api support for ROCm MORI toy proxy server in PD Disaggregation (#40264) 
Signed-off-by: Tej Kiran <kiran.tej@amd.com>
 2026-05-07 16:58:38 +08:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 51c1ee9b7c [Examples] Resettle Disaggregated examples. (#40759) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-06 01:20:38 -07:00