 youkaichao
|
18bd7587b7
|
[1/N] pass the complete config from engine to executor (#9933)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-11-01 13:51:57 -07:00 |
|
|
Pavani Majety
|
598b6d7b07
|
[Bugfix/Core] Flashinfer k_scale and v_scale (#9861)
|
2024-11-01 12:15:05 -07:00 |
|
|
youkaichao
|
aff1fd8188
|
[torch.compile] use interpreter with stable api from pytorch (#9889)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-11-01 11:50:37 -07:00 |
|
|
André Jonasson
|
4581d2cc02
|
[Core] Refactor: Clean up unused argument in Scheduler._preempt (#9696)
Signed-off-by: André Jonasson <andre.jonasson@gmail.com>
|
2024-11-01 11:41:38 -07:00 |
|
|
|
1dd4cb2935
|
[Bugfix] Fix edge cases for MistralTokenizer (#9625)
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
|
2024-11-01 10:33:15 -07:00 |
|
|
Cyrus Leung
|
ba0d892074
|
[Frontend] Use a proper chat template for VLM2Vec (#9912)
|
2024-11-01 14:09:07 +00:00 |
|
|
Cyrus Leung
|
06386a64dd
|
[Frontend] Chat-based Embeddings API (#9759)
|
2024-11-01 08:13:35 +00:00 |
|
|
Yongzaoandyoukaichao
|
2b5bf20988
|
[torch.compile] Adding torch compile annotations to some models (#9876)
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
|
2024-11-01 00:25:47 -07:00 |
|
|
Michael Goin
|
93a76dd21d
|
[Model] Support bitsandbytes for MiniCPMV (#9891)
Signed-off-by: mgoin <michael@neuralmagic.com>
|
2024-11-01 13:31:56 +08:00 |
|
|
youkaichao
|
566cd27797
|
[torch.compile] rework test plans (#9866)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-10-31 22:20:17 -07:00 |
|
|
Michael Goin
|
37a4947dcd
|
[Bugfix] Fix layer skip logic with bitsandbytes (#9887)
Signed-off-by: mgoin <michael@neuralmagic.com>
|
2024-11-01 13:12:44 +08:00 |
|
|
youkaichao
|
96e0c9cbbd
|
[torch.compile] directly register custom op (#9896)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-10-31 21:56:09 -07:00 |
|
|
Joe Runde
|
031a7995f3
|
[Bugfix][Frontend] Reject guided decoding in multistep mode (#9892)
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
|
2024-11-01 01:09:46 +00:00 |
|
|
sasha0552
|
55650c83a0
|
[Bugfix] Fix illegal memory access error with chunked prefill, prefix caching, block manager v2 and xformers enabled together (#9532)
Signed-off-by: sasha0552 <admin@sasha0552.org>
|
2024-10-31 11:46:36 -07:00 |
|
|
Jee Jee Li
|
5608e611c2
|
[Doc] Update Qwen documentation (#9869)
|
2024-10-31 08:54:18 +00:00 |
|
|
Roger Wang
|
3ea2dc2ec4
|
[Misc] Remove deprecated arg for cuda graph capture (#9864)
Signed-off-by: Roger Wang <ywang@roblox.com>
|
2024-10-31 07:22:07 +00:00 |
|
|
Michael Goin
|
d087bf863e
|
[Model] Support quantization of Qwen2VisionTransformer (#9817)
Signed-off-by: mgoin <michael@neuralmagic.com>
|
2024-10-30 22:41:20 -07:00 |
|
|
Kevin H. Luu
|
890ca36072
|
Revert "[Bugfix] Use host argument to bind to interface (#9798)" (#9852)
|
2024-10-31 01:44:51 +00:00 |
|
|
Guillaume Calmettes
|
abbfb6134d
|
[Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens field for chat completion endpoint (#9837)
|
2024-10-30 18:15:56 -07:00 |
|
|
Joe Runde
|
3b3f1e7436
|
[Bugfix][core] replace heartbeat with pid check (#9818)
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
|
2024-10-30 09:34:07 -07:00 |
|
|
Elfie Guo
|
9ff4511e43
|
[Misc] Add chunked-prefill support on FlashInfer. (#9781)
|
2024-10-30 09:33:53 -07:00 |
|
|
Went-Liang
|
81f09cfd80
|
[Model] Support math-shepherd-mistral-7b-prm model (#9697)
Signed-off-by: Went-Liang <wenteng_liang@163.com>
|
2024-10-30 09:33:42 -07:00 |
|
|
Alex Brooks
|
cc98f1e079
|
[CI/Build] VLM Test Consolidation (#9372)
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
|
2024-10-30 09:32:17 -07:00 |
|
|
Woosuk Kwon
|
211fe91aa8
|
[TPU] Correctly profile peak memory usage & Upgrade PyTorch XLA (#9438)
|
2024-10-30 09:41:38 +00:00 |
|
|
youkaichao
|
ff5ed6e1bc
|
[torch.compile] rework compile control with piecewise cudagraph (#9715)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-10-29 23:03:49 -07:00 |
|
|
Yan MaandYiSheng5
|
04a3ae0aca
|
[Bugfix] Fix multi nodes TP+PP for XPU (#8884)
Signed-off-by: YiSheng5 <syhm@mail.ustc.edu.cn>
Signed-off-by: yan ma <yan.ma@intel.com>
Co-authored-by: YiSheng5 <syhm@mail.ustc.edu.cn>
|
2024-10-29 21:34:45 -07:00 |
|
|
Michael Goin
|
226688bd61
|
[Bugfix][VLM] Make apply_fp8_linear work with >2D input (#9812)
|
2024-10-29 19:49:44 -07:00 |
|
|
youkaichao
|
1ab6f6b4ad
|
[core][distributed] fix custom allreduce in pytorch 2.5 (#9815)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-10-29 17:06:24 -07:00 |
|
|
Michael Goin
|
bc73e9821c
|
[Bugfix] Fix prefix strings for quantized VLMs (#9772)
|
2024-10-29 16:02:59 -07:00 |
|
|
|
882a1ad0de
|
[Model] tool calling support for ibm-granite/granite-20b-functioncalling (#8339)
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Maximilien de Bayser <maxdebayser@gmail.com>
|
2024-10-29 15:07:37 -07:00 |
|
|
Joe Runde
|
67bdf8e523
|
[Bugfix][Frontend] Guard against bad token ids (#9634)
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
|
2024-10-29 14:13:20 -07:00 |
|
|
Kunjan
|
0ad216f575
|
[MISC] Set label value to timestamp over 0, to keep track of recent history (#9777)
Signed-off-by: Kunjan Patel <kunjanp@google.com>
|
2024-10-29 19:52:19 +00:00 |
|
|
Junichi Sato
|
ac3d748dba
|
[Model] Add LlamaEmbeddingModel as an embedding Implementation of LlamaModel (#9806)
|
2024-10-29 10:40:35 -07:00 |
|
|
yannicks1
|
0ce7798f44
|
[Misc]: Typo fix: Renaming classes (casualLM -> causalLM) (#9801)
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
|
2024-10-29 10:39:20 -07:00 |
|
|
Sven Seeberg
|
0f43387157
|
[Bugfix] Use host argument to bind to interface (#9798)
|
2024-10-29 10:37:59 -07:00 |
|
|
tastelikefeet
|
08600ddc68
|
Fix the log to correct guide user to install modelscope (#9793)
Signed-off-by: yuze.zyz <yuze.zyz@alibaba-inc.com>
|
2024-10-29 10:36:59 -07:00 |
|
|
科英
|
74fc2d77ae
|
[Misc] Add metrics for request queue time, forward time, and execute time (#9659)
|
2024-10-29 10:32:56 -07:00 |
|
|
wangshuai09
|
622b7ab955
|
[Hardware] using current_platform.seed_everything (#9785)
Signed-off-by: wangshuai09 <391746016@qq.com>
|
2024-10-29 14:47:44 +00:00 |
|
|
Isotr0py
|
09500f7dde
|
[Model] Add BNB quantization support for Mllama (#9720)
|
2024-10-29 08:20:02 -04:00 |
|
|
Zhong Qishuai
|
ef7865b4f9
|
[Frontend] re-enable multi-modality input in the new beam search implementation (#9427)
Signed-off-by: Qishuai Ferdinandzhong@gmail.com
|
2024-10-29 11:49:47 +00:00 |
|
|
Cyrus Leung
|
eae3d48181
|
[Bugfix] Use temporary directory in registry (#9721)
|
2024-10-28 22:08:20 -07:00 |
|
|
Cyrus Leung
|
e74f2d448c
|
[Doc] Specify async engine args in docs (#9726)
|
2024-10-28 22:07:57 -07:00 |
|
|
Jee Jee Li
|
7a4df5f200
|
[Model][LoRA]LoRA support added for Qwen (#9622)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2024-10-29 04:14:07 +00:00 |
|
|
youkaichao
|
76ed5340f0
|
[torch.compile] add deepseek v2 compile (#9775)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-10-28 14:35:17 -07:00 |
|
|
youkaichao
|
97b61bfae6
|
[misc] avoid circular import (#9765)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-10-28 20:51:23 +00:00 |
|
|
Yongzao
|
aa0addb397
|
Adding "torch compile" annotations to moe models (#9758)
|
2024-10-28 13:49:56 -07:00 |
|
|
|
5f8d8075f9
|
[Model][VLM] Add multi-video support for LLaVA-Onevision (#8905)
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-10-28 18:04:10 +00:00 |
|
|
Yan Ma
|
2adb4409e0
|
[Bugfix] Fix ray instance detect issue (#9439)
|
2024-10-28 07:13:03 +00:00 |
|
|
Robert Shaw
|
feb92fbe4a
|
Fix beam search eos (#9627)
|
2024-10-28 06:59:37 +00:00 |
|
|
youkaichao
|
32176fee73
|
[torch.compile] support moe models (#9632)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-10-27 21:58:04 -07:00 |
|