iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
3,524 Commits 3 Branches 0 Tags
2385b60d8300ce730ae67d9ea945f06de9ec4e21
Commit Graph
2324 Commits
Author SHA1 Message Date
Jee Jee Li 2385b60d83 [Kernel] Register punica ops directly (#10522) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-21 09:18:11 -08:00
Chauncey da7e702c6f [Bug]: When apply continue_final_message for OpenAI server, the "echo":false is ignored (#10180) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2024-11-21 16:24:32 +00:00
Xiaoyu Zhang 4d676f0852 [Bugfix] Embedding model pooling_type equals ALL and multi input's bug (#10494) 2024-11-21 14:40:02 +00:00
Isotr0py d5ec121f95 [Model] Expose dynamic_image_size as mm_processor_kwargs for InternVL2 models (#10518) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-21 14:20:08 +00:00
Alex BrooksandCyrus Leung 1cfde82ffd [Model] Add Support for Multimodal Granite Models (#10291) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-11-21 10:46:20 +00:00
Zhong Qishuai f0e0238016 [Doc] fix a small typo in docstring of llama_tool_parser (#10513) 2024-11-21 09:05:23 +00:00
youkaichao aaddce5d26 [platforms] improve error message for unspecified platforms (#10520) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-20 23:07:56 -08:00
Cyrus Leung 3430857b64 [Misc] Increase default video fetch timeout (#10495) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-20 23:06:42 -08:00
Luka Govedič 8b0fe06c89 [torch.compile] Inductor code caching fix (#10273) 
Signed-off-by: luka <luka@neuralmagic.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
 2024-11-20 21:44:57 -08:00
Mengqing Cao 9d827170a3 [Platforms] Add device_type in Platform (#10508) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2024-11-21 04:44:20 +00:00
Pavani Majety 6c1208d083 [Core] Add Sliding Window Support with Flashinfer (#10462) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2024-11-20 19:56:47 -08:00
youkaichao 388ee3de66 [torch.compile] limit inductor threads and lazy import quant (#10482) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-20 18:36:33 -08:00
Woosuk Kwon 2f77b6cfec [TPU] Implement prefix caching for TPUs (#10307) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2024-11-20 13:54:15 -08:00
Guillaume Calmettes c68f7ede6a [Bugfix]: allow extra fields in requests to openai compatible server (#10463) 
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2024-11-20 16:42:21 -05:00
youkaichao 0cd3d9717e [7/N] torch.compile, reduce compilation time (#10460) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-20 11:20:38 -08:00
youkaichao 772a66732d [platforms] restore xpu check for parallel config (#10479) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-20 17:13:28 +00:00
Li, Jiang 63f1fde277 [Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2024-11-20 10:57:39 +00:00
Mengqing Cao d5b28447e0 [Platforms] Refactor xpu code (#10468) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2024-11-19 22:52:13 -08:00
Cyrus Leung 09dbf9ff16 [Bugfix] Handle conflicts between modern and legacy fields (#10471) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-20 14:45:08 +08:00
Sky Lee 343041c4c4 [model] Reduce medusa weight (#10454) 
Signed-off-by: skylee-01 <497627264@qq.com>
 2024-11-20 06:05:55 +00:00
Cyrus Leung b4be5a8adb [Bugfix] Enforce no chunked prefill for embedding models (#10470) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-20 05:12:51 +00:00
Isotr0py ad44437ba3 [Bugfix] Fix Mamba model initialization and MLP Speculator weights loading (#10456) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-20 05:04:05 +00:00
Yanyi Liu 9e05252b46 [Misc] Add __setitem__ for LazyDict (#10469) 
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
 2024-11-20 04:44:57 +00:00
ElizaWszola b00b33d77e [Model][Quantization] HQQ support through Marlin kernel expansion (#9766) 
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
 2024-11-19 13:31:12 -08:00
Russell Bryant efa9084628 [Core] Avoid metrics log noise when idle (#8868) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-11-19 21:05:25 +00:00
youkaichao 803f37eaaa [6/N] torch.compile rollout to users (#10437) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-19 10:09:03 -08:00
Russell Bryant fd9f124971 [Doc] fix link for page that was renamed (#10455) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-11-19 09:48:30 -08:00
Patrick von Platen 11fd7ea639 [Pixtral-Large] Pixtral actually has no bias in vision-lang adapter (#10449) 2024-11-19 17:33:06 +00:00
COSMOPlat f028dff33d [BugFix] Fix hermes tool parser output error stream arguments in some cases (#10395) (#10398) 
Signed-off-by: xiyuan lee <lixiyuan@haier.com>
 2024-11-19 13:42:50 +00:00
youkaichao 25f9c78961 [misc][plugin] improve plugin loading (#10443) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-19 10:43:21 +00:00
Russell Bryant 5390d6664f [Doc] Add the start of an arch overview page (#10368) 2024-11-19 09:52:11 +00:00
Jee Jee Li 382b6a4852 [Misc] Avoid misleading warning messages (#10438) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-19 08:54:58 +00:00
Travis Johnson 272e31c0bd [Bugfix] Guard for negative counter metrics to prevent crash (#10430) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
 2024-11-19 04:57:10 +00:00
Mengqing Cao 8c1fb50705 [Platform][Refactor] Extract func get_default_attn_backend to Platform (#10358) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2024-11-19 11:22:26 +08:00
Jee Jee Li 7eb719df13 [Bugfix]Fix Phi-3 BNB online quantization (#10417) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-19 03:21:42 +00:00
youkaichao 2298e69b5f [ci][bugfix] fix kernel tests (#10431) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-18 15:29:37 -08:00
youkaichao a03ea40792 [3/N][torch.compile] consolidate custom op logging (#10399) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-18 15:14:59 -08:00
Lucas Wilkinson 96d999fbe8 [Kernel] Initial Machete W4A8 support + Refactors (#9855) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2024-11-18 12:59:29 -07:00
Angus Wang c2170a5b39 [Kernel] Explicitly specify other value in tl.load calls (#9014) 
Signed-off-by: Angus Wang <wangjadehao@gmail.com>
 2024-11-18 11:39:40 -08:00
Yan Ma 6b2d25efc7 [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) 
Signed-off-by: yan ma <yan.ma@intel.com>
 2024-11-18 11:18:05 -07:00
Michael Goin 281cc4b3cd [Model][Bugfix] Support TP for PixtralHF ViT (#10405) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-18 10:04:14 -08:00
youkaichao 7851b45196 [5/N][torch.compile] torch.jit.script --> torch.compile (#10406) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-18 23:20:06 +08:00
Isotr0py e7ebb662d7 [Model] Remove transformers attention porting in VITs (#10414) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-18 21:45:21 +08:00
B-201 5be4e52b65 [Model][LoRA]LoRA support added for glm-4v (#10418) 
Signed-off-by: B-201 <Joy25810@foxmail.com>
 2024-11-18 12:57:10 +00:00
Maybewuss 01aae1cc68 [Model] Remove redundant softmax when using PoolingType.STEP (#10415) 2024-11-18 10:05:36 +00:00
lkchen c7dec926f6 [VLM] Report multi_modal_placeholders in output (#10407) 
Signed-off-by: Linkun Chen <lkchen+anyscale@github.com>
 2024-11-18 16:06:16 +08:00
youkaichao 51bb12d17b [4/N][torch.compile] clean up set_torch_compile_backend (#10401) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-17 23:57:20 -08:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 47826cacf0 [Bugfix] Ignore ray reinit error when current platform is ROCm or XPU (#10375) 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
 2024-11-18 11:29:26 +08:00
Isotr0py c4e464333e [Misc] Add uninitialized params tracking for AutoWeightsLoader (#10327) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-18 09:07:46 +08:00
电脑星人 80d85c5d7b [Bugfix] Fix mrope_position_delta in non-last prefill chunk (#10403) 
Signed-off-by: imkero <kerorek@outlook.com>
 2024-11-17 08:50:24 +00:00