iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
19,458 Commits 3 Branches 0 Tags
d9dac2b3d4cdd57ff2f7c5311bc8aaf3eef3feec
Commit Graph
100 Commits
Author SHA1 Message Date
+13
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodexWei Zhao
 		aeeb36b1f1 [New model] Kimi K3 (#50000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-07-30 03:49:58 -07:00
Guan-Ming Chiu 8d28b48d01 [Perf] Isolate MM preprocessing on its own executor (#49524) 
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
 2026-07-26 08:04:17 +00:00
fangyuchuzWaNg3a798347923TianZhuo205150940a798347923w00689259zWaNg3yzchang-plusJade Zheng
 		0b0bd2b5f6 [Feature] Add fault tolerance framework (simplified) for DP+EP external LB deployments (#44428) 
Signed-off-by: fangyuchu <fangyuchu@qq.com>
Signed-off-by: a798347923 <2645302020@qq.com>
Signed-off-by: TianZhuo <2770730562@qq.com>
Signed-off-by: a798347923 <39047817+a798347923@users.noreply.github.com>
Signed-off-by: 205150940 <112750056+205150940@users.noreply.github.com>
Signed-off-by: w00689259 <wangzhuo66@huawei.com>
Signed-off-by: zWaNg3 <37772915+zWaNg3@users.noreply.github.com>
Signed-off-by: zWaNg3 <389750525@qq.com>
Signed-off-by: yzchang-plus <1078477584@qq.com>
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: zWaNg3 <37772915+zWaNg3@users.noreply.github.com>
Co-authored-by: a798347923 <2645302020@qq.com>
Co-authored-by: TianZhuo <2770730562@qq.com>
Co-authored-by: 205150940 <112750056+205150940@users.noreply.github.com>
Co-authored-by: a798347923 <39047817+a798347923@users.noreply.github.com>
Co-authored-by: w00689259 <wangzhuo66@huawei.com>
Co-authored-by: zWaNg3 <389750525@qq.com>
Co-authored-by: yzchang-plus <1078477584@qq.com>
Co-authored-by: Jade Zheng <zheng.shoujian@outlook.com>
 2026-07-25 11:49:09 -04:00
Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 7b40fb9645 [UX] Reject incompatible nested runtime overrides (#49247) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-24 07:16:22 -07:00
Woosuk KwonBugen ZhaoGiancarlo DelfinIsotr0pyIsotr0pyJee Jee LiRoger WangYifan QiaoClaude Fable 5OpenAI Codex
 		6570c9800c [Model] Add Inkling model support [1/N] (#48799) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-15 23:40:07 -07:00
Yan XuandJingyi Yang 95ed0feaa5 DCP supports hybrid attention (#40996) 
Signed-off-by: YanXu <yancey.yx@alibaba-inc.com>
Signed-off-by: Jingyi Yang <girasoleyang@gmail.com>
Co-authored-by: Jingyi Yang <girasoleyang@gmail.com>
 2026-07-09 21:34:45 -07:00
Jee Jee Liandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 32ab064621 [UX] Add model_class_overrides for development and debugging (#47148) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-07 11:43:08 +08:00
Ekagra RanjanBenjamin ChislettNick Hill
 		07516fda67 [MRV2][SD] Make Dynamic SD comatible with Full Cuda Graphs (#45953) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-03 23:58:27 -04:00
Wentao YeandNick Hill a2f713002d [ModelRunner V2] Enable by default for all dense models (#44443) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-02 18:48:57 +08:00
Yejing LaiandKunshang Ji ae2c4f3db7 [XPU][UT]Fix xpu pass_config.fuse_norm_quant assert issue (#46804) 
Signed-off-by: Lai, Yejing <yejing.lai@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-06-29 21:13:44 -07:00
Wentao Ye b53b1c7ffe [Model Runner V2] Migration to support quantized model by default [5/N] (#44446) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-18 12:20:44 -07:00
Ting SUN 9d4b87f4f0 [Bugfix][Model] Validate DefaultModelLoader / LoadConfig and fail with clear errors (#45196) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-06-17 21:46:33 +00:00
Wentao Ye 88a9cdd439 [Model Runner V2] Enable GraniteMOE for MRv2 by default (#45461) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-16 09:31:32 -07:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 78739c1946 [Model Runner v2] Migration from v1 to v2, with Qwen and DSv2 MOE models [3/N] (#42667) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-12 20:44:52 +00:00
Nick Hillandyewentao256 da107a59e5 [MRV2] Also enable MRV2 for Llama and Mistral dense models (#43458) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
 2026-06-02 11:18:46 -07:00
Injae Ryou 165460941f [BugFix] HFValidationError with cloud storage URIs when HF_HUB_OFFLINE=1 (#39155) 
Signed-off-by: Injae Ryou <injaeryou@gmail.com>
 2026-05-27 10:53:32 -05:00
Nicolò LucchesiandNick Hill fba010dd74 [Bugfix][MRV2] Fix KVCache tensor explicit kernel_block_size dim (#42766) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-05-18 20:25:41 -07:00
Nicolò Lucchesi 69c91d010a [MRv2] Default to MRv1 when a connector is present (#42955) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-18 20:34:16 +08:00
Wentao YeandNick Hill ae4f59f0ec [Model Runner v2] Oracle for model runner v2 - qwen3 dense model by default [1/N] (#39337) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-05-14 10:02:33 -07:00
Siddharth Bedekar f51f6844f9 [Bugfix][Spec Decode] Wire draft_probs into probabilistic draft_model rejection (#40269) 2026-05-13 21:04:03 -04:00
Wentao Ye 577b9623e6 [Bug] Fix status update address for non-MOE model within external dp mode (#40839) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-04 16:37:16 -07:00
Luka Govedič d58c42e19c [vLLM IR] 2/N fused_add_rms_norm and maybe_inplace overload (#36823) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-05-01 23:41:15 -04:00
wliao2 3abf858443 [Test] Refactor hard coded device string in test files under compile/quantization/models/model_executor folders (#38901) 
Signed-off-by: Liao, Wei <wei.liao@intel.com>
 2026-04-15 11:02:35 +08:00
Tihomir Elek 8d825b87d6 [Bug] Fix TypeError when hf_config.architectures is None during model loading (#38849) 
Signed-off-by: Tihomir Elek <tiho.elek@gmail.com>
 2026-04-13 12:13:21 +01:00
Luka GovedičXinyu ChenChaojun ZhangLuka Govedič
 		40bb175027 [vLLM IR] 1/N Implement IR skeleton and rms_norm op (#33825) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Signed-off-by: chzhang <chaojun.zhang@intel.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Luka Govedič <ProExpertProg@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
 2026-03-31 22:15:05 -04:00
Bvicii bda3eda82d [Bugfix] Disallow renderer_num_workers > 1 with mm processor cache (#38418) 
Signed-off-by: Bvicii <yizhanhuang2002@gmail.com>
 2026-03-28 06:32:52 -07:00
Simon Mo 1fa1e53a73 Revert "[compile] Initialize passes at VllmBackend init" (#37733) 2026-03-20 21:35:49 -07:00
Angela Yi 12fd17eb51 [compile] Initialize passes at VllmBackend init (#35216) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2026-03-20 11:40:33 -07:00
Jiayi YanandHarry Mellor 6a895197fa [Bugfix][CI] fix typos (#34934) 
Signed-off-by: 1195343015 <1195343015@qq.com>
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-05 17:05:46 +00:00
Yanwen LinandHarry Mellor 675ec59aa9 [Bugfix][CPU] Fix basic unit tests failing in CPU platforms (#34677) 
Signed-off-by: Yanwen Lin <lyw1124278064@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-25 08:36:15 +00:00
Luka Govedič 4d9513537d [CI][torch.compile] Reduce e2e fusion test time (#33293) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: ProExpertProg <luka.govedic@gmail.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-02-04 19:09:03 -05:00
Harry Mellor 61e632aea1 Turn @config into a dataclass_transform (#31541) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-03 17:40:59 +00:00
Wentao YeandNick Hill 3e440786af [Feature] Fully support for async scheduling + PP, 30.8% E2E throughput improvement, 31.8% TPOT improvement (#32618) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-01-28 20:30:32 +00:00
Cyrus Leung 232214b2ae [Bugfix] Replace PoolingParams.normalize with use_activation (#32243) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-13 10:45:42 +00:00
Xingyu Liu 80221e1884 [BugFix]Fix eagle draft_model_config and add tests (#31753) 
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
 2026-01-12 23:09:36 -08:00
Cyrus Leung 583a90e005 [Refactor] Separate sequence and token pooling types (#32026) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-10 04:53:24 +00:00
Cyrus Leung d1b6fe007f [Chore] Further cleanup pooler (#31951) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-08 02:16:21 -08:00
Xingyu Liu 0eee877f67 [Core] Parse vLLM engine required fields from hf_config to model_arch_config (#28454) 
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
Signed-off-by: Xingyu Liu <38244988+charlotte12l@users.noreply.github.com>
 2026-01-02 15:13:15 -08:00
Nick Hill bd877162eb [BugFix] Support online dense model DP without overhead (#30739) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: njhill <nickhill123@gmail.com>
 2026-01-02 23:36:38 +08:00
Cyrus Leung 7e24e5d4d6 [Deprecation] Remove deprecated task, seed and MM settings (#30397) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-10 19:59:39 -08:00
Cyrus Leung 5a87d8b9b1 [Deprecation] Remove deprecated plugin and compilation fields for v0.13 release (#30396) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-10 19:59:35 -08:00
wang.yuqi 9e77ffca3f [Model][7/N] Improve all pooling task | Deprecation as_reward_model. Extract hidden states prefer using new multi-vector retrieval API (#26686) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2025-12-08 08:10:09 +00:00
Cyrus Leung e83b7e379c Revert "[Renderer] Separate out RendererConfig from ModelConfig (#30145)" (#30199) 2025-12-07 00:00:22 -08:00
Cyrus Leung 27f4c2fd46 [Renderer] Separate out RendererConfig from ModelConfig (#30145) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-06 23:15:42 -08:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 74c4d80c6c [Model][6/N] Improve all pooling task | Support chunked prefill with ALL pooling (#27145) 
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-04 13:44:15 +00:00
Arpit KhandelwalandLuka Govedič d7284a2604 [Core] Rename PassConfig flags as per RFC #27995 (#29646) 
Signed-off-by: arpitkh101 <arpit5khandelwal@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-12-03 03:38:55 +00:00
Harry Mellor 951445a52d Remove default values from InitVars so that they're not stored (#29859) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-02 12:16:37 +00:00
Boyuan Feng 3b221cb661 [BugFix] respect VLLM_LOGGING_LEVEL in logger (#29761) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-12-02 07:49:16 +00:00
wang.yuqiandCyrus Leung f4b76056ee Improve enable chunked_prefill & prefix_caching logic. (#26623) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-11-27 22:05:48 -08:00
Morrison TurnanskyadabeytaLuka GovedičHarry Mellor
 		0838b52e2e [Frontend][torch.compile] CompilationConfig Overhaul (#20283): Set up -O infrastructure (#26847) 
Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: adabeyta <aabeyta@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Co-authored-by: adabeyta <aabeyta@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-27 01:55:58 -08:00