iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
6,768 Commits 3 Branches 0 Tags
6881107948c00a8564bc2fa85308f6fc2f065d64
Commit Graph
4568 Commits
Author SHA1 Message Date
youkaichao 74b529ceee [bugfix] fix chatglm dummy_data_for_glmv (#9955) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-02 08:03:33 -07:00
Robert Shaw d6459b4516 [V1] Fix EngineArgs refactor on V1 (#9954) 2024-11-02 07:44:38 -07:00
youkaichaoandNick Hill e893795443 [2/N] executor pass the complete config to worker/modelrunner (#9938) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2024-11-02 07:35:05 -07:00
youkaichao af7380d83b [torch.compile] fix cpu broken code (#9947) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-01 23:35:47 -07:00
sroy745 a78dd3303e [Encoder Decoder] Add flash_attn kernel support for encoder-decoder models (#9559) 2024-11-01 23:22:49 -07:00
Peter Salas 6c0b7f548d [Core][VLM] Add precise multi-modal placeholder tracking (#8346) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2024-11-01 16:21:10 -07:00
Gene Der Su 27cd36e6e2 [Bugfix] PicklingError on RayTaskError (#9934) 
Signed-off-by: Gene Su <e870252314@gmail.com>
 2024-11-01 22:08:23 +00:00
youkaichao 18bd7587b7 [1/N] pass the complete config from engine to executor (#9933) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-01 13:51:57 -07:00
Pavani Majety 598b6d7b07 [Bugfix/Core] Flashinfer k_scale and v_scale (#9861) 2024-11-01 12:15:05 -07:00
youkaichao aff1fd8188 [torch.compile] use interpreter with stable api from pytorch (#9889) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-01 11:50:37 -07:00
André Jonasson 4581d2cc02 [Core] Refactor: Clean up unused argument in Scheduler._preempt (#9696) 
Signed-off-by: André Jonasson <andre.jonasson@gmail.com>
 2024-11-01 11:41:38 -07:00
Travis JohnsonPrashant GuptaPatrick von Platen
 		1dd4cb2935 [Bugfix] Fix edge cases for MistralTokenizer (#9625) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2024-11-01 10:33:15 -07:00
Cyrus Leung ba0d892074 [Frontend] Use a proper chat template for VLM2Vec (#9912) 2024-11-01 14:09:07 +00:00
Cyrus Leung 06386a64dd [Frontend] Chat-based Embeddings API (#9759) 2024-11-01 08:13:35 +00:00
Yongzaoandyoukaichao 2b5bf20988 [torch.compile] Adding torch compile annotations to some models (#9876) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2024-11-01 00:25:47 -07:00
Michael Goin 93a76dd21d [Model] Support bitsandbytes for MiniCPMV (#9891) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-01 13:31:56 +08:00
youkaichao 566cd27797 [torch.compile] rework test plans (#9866) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-31 22:20:17 -07:00
Michael Goin 37a4947dcd [Bugfix] Fix layer skip logic with bitsandbytes (#9887) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-01 13:12:44 +08:00
youkaichao 96e0c9cbbd [torch.compile] directly register custom op (#9896) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-31 21:56:09 -07:00
Joe Runde 031a7995f3 [Bugfix][Frontend] Reject guided decoding in multistep mode (#9892) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2024-11-01 01:09:46 +00:00
sasha0552 55650c83a0 [Bugfix] Fix illegal memory access error with chunked prefill, prefix caching, block manager v2 and xformers enabled together (#9532) 
Signed-off-by: sasha0552 <admin@sasha0552.org>
 2024-10-31 11:46:36 -07:00
Jee Jee Li 5608e611c2 [Doc] Update Qwen documentation (#9869) 2024-10-31 08:54:18 +00:00
Roger Wang 3ea2dc2ec4 [Misc] Remove deprecated arg for cuda graph capture (#9864) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-10-31 07:22:07 +00:00
Michael Goin d087bf863e [Model] Support quantization of Qwen2VisionTransformer (#9817) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-10-30 22:41:20 -07:00
Kevin H. Luu 890ca36072 Revert "[Bugfix] Use host argument to bind to interface (#9798)" (#9852) 2024-10-31 01:44:51 +00:00
Guillaume Calmettes abbfb6134d [Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens field for chat completion endpoint (#9837) 2024-10-30 18:15:56 -07:00
Joe Runde 3b3f1e7436 [Bugfix][core] replace heartbeat with pid check (#9818) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2024-10-30 09:34:07 -07:00
Elfie Guo 9ff4511e43 [Misc] Add chunked-prefill support on FlashInfer. (#9781) 2024-10-30 09:33:53 -07:00
Went-Liang 81f09cfd80 [Model] Support math-shepherd-mistral-7b-prm model (#9697) 
Signed-off-by: Went-Liang <wenteng_liang@163.com>
 2024-10-30 09:33:42 -07:00
Alex Brooks cc98f1e079 [CI/Build] VLM Test Consolidation (#9372) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2024-10-30 09:32:17 -07:00
Woosuk Kwon 211fe91aa8 [TPU] Correctly profile peak memory usage & Upgrade PyTorch XLA (#9438) 2024-10-30 09:41:38 +00:00
youkaichao ff5ed6e1bc [torch.compile] rework compile control with piecewise cudagraph (#9715) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-29 23:03:49 -07:00
Yan MaandYiSheng5 04a3ae0aca [Bugfix] Fix multi nodes TP+PP for XPU (#8884) 
Signed-off-by: YiSheng5 <syhm@mail.ustc.edu.cn>
Signed-off-by: yan ma <yan.ma@intel.com>
Co-authored-by: YiSheng5 <syhm@mail.ustc.edu.cn>
 2024-10-29 21:34:45 -07:00
Michael Goin 226688bd61 [Bugfix][VLM] Make apply_fp8_linear work with >2D input (#9812) 2024-10-29 19:49:44 -07:00
youkaichao 1ab6f6b4ad [core][distributed] fix custom allreduce in pytorch 2.5 (#9815) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-29 17:06:24 -07:00
Michael Goin bc73e9821c [Bugfix] Fix prefix strings for quantized VLMs (#9772) 2024-10-29 16:02:59 -07:00
Will EatonMax de BayserMaximilien de Bayser
 		882a1ad0de [Model] tool calling support for ibm-granite/granite-20b-functioncalling (#8339) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Maximilien de Bayser <maxdebayser@gmail.com>
 2024-10-29 15:07:37 -07:00
Joe Runde 67bdf8e523 [Bugfix][Frontend] Guard against bad token ids (#9634) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2024-10-29 14:13:20 -07:00
Kunjan 0ad216f575 [MISC] Set label value to timestamp over 0, to keep track of recent history (#9777) 
Signed-off-by: Kunjan Patel <kunjanp@google.com>
 2024-10-29 19:52:19 +00:00
Junichi Sato ac3d748dba [Model] Add LlamaEmbeddingModel as an embedding Implementation of LlamaModel (#9806) 2024-10-29 10:40:35 -07:00
yannicks1 0ce7798f44 [Misc]: Typo fix: Renaming classes (casualLM -> causalLM) (#9801) 
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
 2024-10-29 10:39:20 -07:00
Sven Seeberg 0f43387157 [Bugfix] Use host argument to bind to interface (#9798) 2024-10-29 10:37:59 -07:00
tastelikefeet 08600ddc68 Fix the log to correct guide user to install modelscope (#9793) 
Signed-off-by: yuze.zyz <yuze.zyz@alibaba-inc.com>
 2024-10-29 10:36:59 -07:00
科英 74fc2d77ae [Misc] Add metrics for request queue time, forward time, and execute time (#9659) 2024-10-29 10:32:56 -07:00
wangshuai09 622b7ab955 [Hardware] using current_platform.seed_everything (#9785) 
Signed-off-by: wangshuai09 <391746016@qq.com>
 2024-10-29 14:47:44 +00:00
Isotr0py 09500f7dde [Model] Add BNB quantization support for Mllama (#9720) 2024-10-29 08:20:02 -04:00
Zhong Qishuai ef7865b4f9 [Frontend] re-enable multi-modality input in the new beam search implementation (#9427) 
Signed-off-by: Qishuai Ferdinandzhong@gmail.com
 2024-10-29 11:49:47 +00:00
Cyrus Leung eae3d48181 [Bugfix] Use temporary directory in registry (#9721) 2024-10-28 22:08:20 -07:00
Cyrus Leung e74f2d448c [Doc] Specify async engine args in docs (#9726) 2024-10-28 22:07:57 -07:00
Jee Jee Li 7a4df5f200 [Model][LoRA]LoRA support added for Qwen (#9622) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-10-29 04:14:07 +00:00