iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
2340 Commits
Author SHA1 Message Date
Cyrus Leung 6609cdf019 [Doc] Add docs for prompt replacement (#12318) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-22 14:56:29 +00:00
Roger Wang 16366ee8bb [Bugfix][VLM] Fix mixed-modality inference backward compatibility for V0 (#12313) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-01-22 21:06:36 +08:00
zhou fan 528dbcac7d [Model][Bugfix]: correct Aria model output (#12309) 
Signed-off-by: xffxff <1247714429@qq.com>
 2025-01-22 11:39:19 +00:00
Cyrus Leung cd7b6f0857 [VLM] Avoid unnecessary tokenization (#12310) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-22 11:08:31 +00:00
Kevin H. Luu 64ea24d0b3 [ci/lint] Add back default arg for pre-commit (#12279) 
Signed-off-by: kevin <kevin@anyscale.com>
 2025-01-22 01:15:27 +00:00
Cyrus Leung df76e5af26 [VLM] Simplify post-processing of replacement info (#12269) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-21 16:48:13 -08:00
wangxiyuan fa9ee08121 [Misc] Set default backend to SDPA for get_vit_attn_backend (#12235) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-01-21 11:52:11 -08:00
Cyrus Leung f2e9f2a3be [Misc] Remove redundant TypeVar from base model (#12248) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-21 08:40:39 +00:00
Jee Jee Li 1f1542afa9 [Misc]Add BNB quantization for PaliGemmaForConditionalGeneration (#12237) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-01-21 07:49:08 +00:00
Cyrus Leung 96912550c8 [Misc] Rename MultiModalInputsV2 -> MultiModalInputs (#12244) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-21 07:31:19 +00:00
Cyrus Leung b37d82791e [Model] Upgrade Aria to transformers 4.48 (#12203) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-20 17:58:48 +08:00
Isotr0py 83609791d2 [Model] Add Qwen2 PRM model support (#12202) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-01-20 14:59:46 +08:00
Martin Gleizeandmgleize user bbe5f9de7d [Model] Support for fairseq2 Llama (#11442) 
Signed-off-by: Martin Gleize <mgleize@meta.com>
Co-authored-by: mgleize user <mgleize@a100-st-p4de24xlarge-4.fair-a100.hpcaas>
 2025-01-19 10:40:40 -08:00
Roger WangimkeroDarkLight1337
 		81763c58a0 [V1] Add V1 support of Qwen2-VL (#12128) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: imkero <kerorek@outlook.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-19 19:52:13 +08:00
Isotr0py edaae198e7 [Misc] Add BNB support to GLM4-V model (#12184) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-01-19 19:49:22 +08:00
Cyrus Leung 630eb5b5ce [Bugfix] Fix multi-modal processors for transformers 4.48 (#12187) 2025-01-18 19:16:34 -08:00
Isotr0py 02798ecabe [Model] Port deepseek-vl2 processor, remove dependency (#12169) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-01-18 13:59:39 +08:00
Isotr0py 62b06ba23d [Model] Add support for deepseek-vl2-tiny model (#12068) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-01-16 17:14:48 +00:00
Roger Wang 874f7c292a [Bugfix] Fix max image feature size for Llava-one-vision (#12104) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-01-16 14:54:06 +00:00
Isotr0py dd7c9ad870 [Bugfix] Remove hardcoded head_size=256 for Deepseek v2 and v3 (#12067) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-01-16 10:11:54 +00:00
Michael Goin 9aa1519f08 Various cosmetic/comment fixes (#12089) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2025-01-16 09:59:06 +00:00
kewang-xlnxkewang2Michael Goin
 		de0526f668 [Misc][Quark] Upstream Quark format to VLLM (#10765) 
Signed-off-by: kewang-xlnx <kewang@xilinx.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2025-01-15 11:05:15 -05:00
RunningLeon 97eb97b5a4 [Model]: Support internlm3 (#12037) 2025-01-15 11:35:17 +00:00
Jee Jee Li 42f5e7c52a [Kernel] Support MulAndSilu (#11624) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-01-15 02:29:53 +00:00
Jee Jee Li a3a3ee4e6f [Misc] Merge bitsandbytes_stacked_params_mapping and packed_modules_mapping (#11924) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-01-15 07:49:49 +08:00
Concurrensee cf6bbcb493 [Misc] Fix Deepseek V2 fp8 kv-scale remapping (#11947) 
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
 2025-01-12 23:05:06 -08:00
Sungjae Lee 80ea3af1a0 [CI][Spec Decode] fix: broken test for EAGLE model (#11972) 
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
 2025-01-13 06:50:35 +00:00
Siyuan Li 9dd02d85ca [Bug] Fix usage of .transpose() and .view() consecutively. (#11979) 2025-01-13 06:24:10 +00:00
Isotr0pyandCyrus Leung d14e98d924 [Model] Support GGUF models newly added in transformers 4.46.0 (#9685) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-01-13 00:13:44 +00:00
Isotr0pyandCyrus Leung f967e51f38 [Model] Initialize support for Deepseek-VL2 models (#11578) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-01-12 00:17:24 -08:00
sixgodandIsotr0py 4b657d3292 [Model] Add cogagent model support vLLM (#11742) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-01-11 19:05:56 +00:00
Nicolò Lucchesi d697dc01b4 [Bugfix] Fix RobertaModel loading (#11940) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-01-11 14:05:09 +00:00
Sungjae Lee 2118d0565c [Bugfix][SpecDecode] Adjust Eagle model architecture to align with intended design (#11672) 
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
 2025-01-10 20:49:38 -08:00
Travis Johnson d45cbe70f5 [Bugfix] Check that number of images matches number of <|image|> tokens with mllama (#11939) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
 2025-01-10 23:26:00 +00:00
Isotr0py 46fa98ccad [Misc] Clean up debug code in Deepseek-V3 (#11930) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-01-10 19:19:15 +00:00
wangxiyuanandyoukaichao 20410b2fda [platform] support custom torch.compile backend key (#11318) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-01-10 23:46:51 +08:00
Harry Mellor d85c47d6ad Replace "online inference" with "online serving" (#11923) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-10 12:05:56 +00:00
Cyrus Leung b844b99ad3 [VLM] Enable tokenized inputs for merged multi-modal processor (#11900) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-10 03:24:00 +00:00
Cyrus Leung 65097ca0af [Doc] Add model development API Reference (#11884) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-09 09:43:40 +00:00
Cyrus LeungandMaxime Fournioux d848800e88 [Misc] Move print_*_once from utils to logger (#11298) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Co-authored-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
 2025-01-09 12:48:12 +08:00
Cyrus Leung ca47e176af [Misc] Move some model utils into vision file (#11848) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-08 17:04:46 +00:00
Cyrus Leung 2a0596bc48 [VLM] Reorganize profiling/processing-related code (#11812) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-08 18:59:58 +08:00
Cyrus Leung 91445c7bc8 [Bugfix] Fix image input for Pixtral-HF (#11741) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-08 10:17:16 +08:00
jiangjiadiandjiadi.jjd c994223d56 [Bugfix] update the prefix for qwen2 (#11795) 
Co-authored-by: jiadi.jjd <jiadi.jjd@antgroup.com>
 2025-01-07 18:36:34 +00:00
Roger Wang 2de197bdd4 [V1] Support audio language models on V1 (#11733) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-01-07 19:47:36 +08:00
Cyrus Leung 8f37be38eb [Bugfix] Comprehensively test and fix LLaVA-NeXT feature size calculation (#11800) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-07 18:25:02 +08:00
Roger Wang 0f3f3c86ec [Bugfix] Update attention interface in Whisper (#11784) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-01-07 04:36:24 +00:00
Cyrus Leung d0169e1b0f [Model] Future-proof Qwen2-Audio multi-modal processor (#11776) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-07 11:05:17 +08:00
Cyrus Leung 08fb75c72e [Bugfix] Fix LLaVA-NeXT feature size precision error (for real) (#11772) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-07 01:10:54 +00:00
Roger WangandDarkLight1337 91b361ae89 [V1] Extend beyond image modality and support mixed-modality inference with Llava-OneVision (#11685) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-06 19:58:16 +00:00