 rongfu.leng
|
5a1e1c8353
|
[Model] use AutoWeightsLoader for phimoe,qwen2_moe,qwen3_moe (#16203)
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
|
2025-04-08 04:05:47 -07:00 |
|
|
Alex Brooks
|
69ecaa7c79
|
[Misc] Add warning for multimodal data in LLM.beam_search (#16241)
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
|
2025-04-08 04:05:27 -07:00 |
|
|
Kebe
|
b4ac449a83
|
[Misc] Merge the logs of pp layers partitions (#16225)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2025-04-08 00:18:15 -07:00 |
|
|
Michael Goin
|
8e5314a468
|
[V1] Add disable_chunked_mm_input arg to disable partial mm input prefill (#15837)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2025-04-07 23:24:07 -07:00 |
|
|
Siyuan Liuandmgoin
|
87918e40c4
|
[torch.compile][TPU] Make @support_torch_compile work for XLA backend (#15782)
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2025-04-08 14:23:53 +08:00 |
|
|
Michael Goin
|
b99733d092
|
[Bugfix] Do not skip "empty" parts of chats that are parsable (#16219)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2025-04-08 05:14:15 +00:00 |
|
|
Yong Hoon Shin
|
05a015d6a5
|
Add warning for Attention backends that do not support irope yet (#16212)
|
2025-04-08 03:59:26 +00:00 |
|
|
zxfan-cpu
|
ad971af8c7
|
[Bugfix] fix use-ep bug to enable ep by dp/tp size > 1 (#16161)
|
2025-04-07 20:48:47 -07:00 |
|
|
|
f2ebb6f541
|
[V1] Scatter and gather placeholders in the model runner (#16076)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
|
2025-04-08 10:43:41 +08:00 |
|
|
youkaichao
|
a865bc1ca6
|
[core] do not send error across process (#16174)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2025-04-07 19:09:03 -07:00 |
|
|
Michael Goin
|
21802c4b6d
|
[ROCm][Bugfix][FP8] Make fp8 quant respect fused modules mapping (#16031)
Signed-off-by: mgoin <michael@neuralmagic.com>
|
2025-04-07 21:28:14 -04:00 |
|
|
Driss Guessous
|
652907b354
|
Torchao (#14231)
Signed-off-by: drisspg <drisspguessous@gmail.com>
|
2025-04-07 19:39:28 -04:00 |
|
|
leon-seidel
|
24f1c01e0f
|
[Bugfix][V0] XGrammar structured output supports Enum (#15878)
Signed-off-by: Leon Seidel <leon.seidel@fau.de>
|
2025-04-07 22:38:25 +00:00 |
|
|
Reidandreidliu41
|
fad6e2538e
|
[Misc] add description attribute in CLI (#15921)
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
|
2025-04-07 22:30:35 +00:00 |
|
|
Nick Hill
|
7f6d47c1a2
|
[V1][BugFix] Exit properly if engine core fails during startup (#16137)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-04-07 15:30:15 -07:00 |
|
|
Benjamin Chislett
|
3147586ebd
|
[Bugfix] Fix guidance backend for Qwen models (#16210)
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
|
2025-04-07 22:15:43 +00:00 |
|
|
Roger Wang
|
ed636d99ca
|
[Misc] Move Llama 4 projector call into encoder execution (#16201)
|
2025-04-07 14:02:05 -07:00 |
|
|
Nicolò LucchesiandCyrus Leung
|
090c856d76
|
[Misc] Human-readable max-model-len cli arg (#16181)
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2025-04-07 14:40:58 -04:00 |
|
|
Gregory Shtrasberg
|
ad434d4cfe
|
Print the warning only once (#16193)
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
|
2025-04-07 18:30:06 +00:00 |
|
|
Cyrus Leung
|
66d433b94f
|
[V1] Revert the default max_num_seqs to V0 values for most hardware (#16158)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-04-07 13:54:36 -04:00 |
|
|
Cyrus Leung
|
027b204ff1
|
[Bugfix] Re-enable support for ChatGLMForConditionalGeneration (#16187)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-04-07 23:15:58 +08:00 |
|
|
|
55dcce91df
|
Upstream Llama4 Support to Main (#16113)
Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-04-07 08:06:27 -07:00 |
|
|
YamPengLiandCyrus Leung
|
7699258ef0
|
[Model] Add Qwen3 and Qwen3MoE (#15289)
Signed-off-by: YamPengLi <yampayne.lyp@alibaba-inc.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2025-04-07 04:06:41 -07:00 |
|
|
Shanshan Shen
|
e9ba99f296
|
[V1][Structured Output] Add supports_structured_output() method to Platform (#16148)
Signed-off-by: shen-shanshan <467638484@qq.com>
|
2025-04-07 11:06:24 +00:00 |
|
|
Isotr0py
|
7c80368710
|
[VLM] Florence-2 supports online serving (#16164)
Signed-off-by: Isotr0py <2037008807@qq.com>
|
2025-04-07 04:04:02 -07:00 |
|
|
Isotr0py
|
fc0f87768a
|
[Bugfix] Make dummy encoder prompt padding alternative and add missing warnings (#16129)
Signed-off-by: Isotr0py <2037008807@qq.com>
|
2025-04-07 04:07:15 +00:00 |
|
|
Woosuk Kwon
|
3749e28774
|
[V1][Minor] Minor simplification for get_computed_blocks (#16139)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-04-06 20:38:12 -07:00 |
|
|
Kay Yan
|
86fc2321ff
|
[Metrics] Add bucket for request_latency, time_to_first_token and time_per_output_token (#15202)
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
|
2025-04-06 20:34:51 -07:00 |
|
|
Woosuk Kwon
|
b10e519895
|
[V1][Minor] Optimize get_cached_block (#16135)
|
2025-04-06 20:48:14 +00:00 |
|
|
paolovicandpaolovic
|
da224daaa9
|
[Bugfix] add hf_token to EngineArgs (#16093)
Signed-off-by: paolovic <paul-philipp.luley@uzh.ch>
Co-authored-by: paolovic <paul-philipp.luley@uzh.ch>
|
2025-04-06 14:47:33 +00:00 |
|
|
Varun Sundar RabindranathandVarun Sundar Rabindranath
|
3a100b9278
|
[Bugfix] LoRA : Fix the order in which the kernels process LoRAs (#16040)
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
|
2025-04-06 14:04:50 +00:00 |
|
|
rongfu.leng
|
242a637aea
|
[Model] use AutoWeightsLoader for stablelm,starcoder2,zamba2 (#16103)
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
|
2025-04-06 05:52:01 -07:00 |
|
|
Isotr0py
|
c2a9671510
|
[Misc] Improve model redirect to accept json dictionary (#16119)
Signed-off-by: Isotr0py <2037008807@qq.com>
|
2025-04-06 05:51:45 -07:00 |
|
|
Roger Wang
|
9ca710e525
|
[CI][V1] Fix passing tokenizer as kwarg to validate_guidance_grammar (#16117)
Signed-off-by: Roger Wang <ywang@roblox.com>
|
2025-04-06 16:18:00 +08:00 |
|
|
Lucia Fang
|
620fc2d09e
|
[Model] fix model testing for TeleChat2ForCausalLM and V0 llama4 (#16112)
Signed-off-by: Lu Fang <fanglu@fb.com>
|
2025-04-05 21:23:40 -07:00 |
|
|
Jonghyun Choe
|
29283eaa7e
|
[Model] use AutoWeightsLoader for phi, gemma, deepseek (#16088)
Signed-off-by: Jonghyun Choe <andy.choe729@gmail.com>
|
2025-04-05 20:34:38 -07:00 |
|
|
Jinzhen Lin
|
2fa66ef713
|
[Bugfix] fix use_atomic_add support of marlin kernel when using v1 engine (#15946)
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
|
2025-04-05 20:04:22 -07:00 |
|
|
Chauncey
|
13affc432d
|
[Misc] Remove redundant code (#16098)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2025-04-05 20:03:50 -07:00 |
|
|
Tristan Leclercq
|
4285e423a6
|
[Misc] Auto detect bitsandbytes pre-quantized models (#16027)
Signed-off-by: Tristan Leclercq <tristanleclercq@gmail.com>
|
2025-04-04 23:30:45 -07:00 |
|
|
Woosuk Kwon
|
63375f0cdb
|
[V1][Spec Decode] Update N-gram Proposer Interface (#15750)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-04-04 16:32:54 -07:00 |
|
|
Michael Goin
|
70ad3f9e98
|
[Bugfix][TPU] Fix V1 TPU worker for sliding window (#16059)
Signed-off-by: Michael Goin <mgoin64@gmail.com>
|
2025-04-04 23:31:19 +00:00 |
|
|
bnellnm
|
d6fc629f4d
|
[Kernel][Minor] Re-fuse triton moe weight application (#16071)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2025-04-04 23:27:34 +00:00 |
|
|
Roger Wang
|
af51d80fa1
|
Revert "[V1] Scatter and gather placeholders in the model runner" (#16075)
|
2025-04-04 14:50:57 -07:00 |
|
|
|
f5722a5052
|
[V1] Scatter and gather placeholders in the model runner (#15712)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
|
2025-04-04 21:26:44 +00:00 |
|
|
Nick Hill
|
651cf0fec1
|
[V1] DP scale-out (1/N): Use zmq ROUTER/DEALER sockets for input queue (#15906)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-04-04 12:56:43 -07:00 |
|
|
Michael Goin
|
4708f13a9c
|
[Bugfix] Fix default behavior/fallback for pp in v1 (#16057)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2025-04-04 17:58:08 +00:00 |
|
|
Gregory Shtrasberg
|
a6d042df0a
|
[ROCm][Bugfix] Bring back fallback to eager mode removed in #14917, but for ROCm only (#15413)
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
|
2025-04-04 09:40:37 -07:00 |
|
|
Gregory Shtrasberg
|
40a36ccfeb
|
[ROCm][Bugfix] Use platform specific FP8 dtype (#15717)
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
|
2025-04-04 09:40:20 -07:00 |
|
|
Ilya Markovandilmarkov
|
ef608c37a7
|
[Distributed] [ROCM] Fix custom allreduce enable checks (#16010)
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
|
2025-04-04 09:39:08 -07:00 |
|
|
Li, Jiang
|
2386803f2a
|
[CPU] Change default block_size for CPU backend (#16002)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2025-04-04 09:39:05 -07:00 |
|