iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
6,768 Commits 3 Branches 0 Tags
6881107948c00a8564bc2fa85308f6fc2f065d64
Commit Graph
4568 Commits
Author SHA1 Message Date
Harry Mellor 900edfa8d4 Transformers backend tweaks (#17365) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-29 09:08:03 -07:00
Cyrus Leung 88ad9ec6b2 [Frontend] Support chat_template_kwargs in LLM.chat (#17356) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-29 22:03:35 +08:00
Cyrus Leung 00ee37efa2 [Bugfix] Clean up MiniMax-VL and fix processing (#17354) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-29 20:42:16 +08:00
Ekagra Ranjan 97cc8729f0 [Model] Ignore rotary embed load for Cohere model (#17319) 2025-04-29 00:30:40 -07:00
Hyogeun Oh (오효근) 193e78e35d [Fix] Documentation spacing in compilation config help text (#17342) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
 2025-04-29 00:16:17 -07:00
ponix-j bdb2cddafc [Misc]Use a platform independent interface to obtain the device attributes (#17100) 2025-04-29 06:59:13 +00:00
Cyrus Leung ebb3930d28 [Misc] Move config fields to MultiModalConfig (#17343) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-29 06:37:21 +00:00
qscqesze cde384cd92 [Model] support MiniMax-VL-01 model (#16328) 
Signed-off-by: qingjun <qingjun@minimaxi.com>
 2025-04-29 12:05:50 +08:00
Zhengyuan Su (苏政渊)and苏政渊 17eb306fcc [Bugfix] Add contiguous call inside rope kernel wrapper (#17091) 
Signed-off-by: 苏政渊 <suzhengyuan@moonshot.cn>
Co-authored-by: 苏政渊 <suzhengyuan@moonshot.cn>
 2025-04-28 19:24:07 -07:00
Richard Zou 165cb56329 Ignore '<string>' filepath (#17330) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-04-28 19:23:29 -07:00
Lucia Fang b4ac4fa04d [model] make llama4 compatible with pure dense layers (#17315) 
Signed-off-by: Lucia Fang <fanglu@fb.com>
 2025-04-29 10:22:22 +08:00
Ekagra Ranjan e136000595 [V1][Spec Decode] Make Eagle model arch config driven (#17323) 2025-04-29 10:22:02 +08:00
Michał Moskal 86d9fc29cb implement Structural Tag with Guidance backend (#17333) 
Signed-off-by: Michal Moskal <michal@moskal.me>
 2025-04-29 02:21:32 +00:00
Cyrus Leung 506475de5f [Optim] Compute multimodal hash only once per item (#17314) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-29 09:40:35 +08:00
Michael Goin 8fc88d63f1 [Model] Add tuned triton fused_moe configs for Qwen3Moe (#17328) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-28 15:20:24 -07:00
Alex Wu 6e74fd4945 Support loading transformers models with named parameters (#16868) 
Signed-off-by: Alex <alexwu@character.ai>
 2025-04-28 23:15:58 +01:00
Simon Mo dcbac4cb4b [Model] Qwen3 Dense FP8 Compat Fixes (#17318) 
Signed-off-by: simon-mo <xmo@berkeley.edu>
 2025-04-28 14:12:01 -07:00
Charlie Fu ed2462030f [Bugfix] Fix moe weight losing all extra attrs after process_weights_after_loading. (#16854) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-04-28 21:05:07 +00:00
Lucas Wilkinson cc5befbced [BugFix] Fix cascade attention - RuntimeError: scheduler_metadata must have shape (metadata_size) (#17283) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-04-28 13:55:50 -07:00
Russell Bryant a0304dc504 [Security] Don't bind tcp zmq socket to all interfaces (#17197) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-04-28 10:08:20 -07:00
Harry Mellor c7941cca18 Explicitly explain quant method override ordering and ensure all overrides are ordered (#17256) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-28 16:55:31 +00:00
Harry Mellor b6dd32aa07 Make name of compressed-tensors quant method consistent across vLLM (#17255) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-28 16:28:13 +00:00
Harry Mellor f94886946e Improve conversion from dataclass configs to argparse arguments (#17303) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-28 16:22:12 +00:00
Cyrus Leung 8b464d9660 [Misc] Clean up Qwen2.5-Omni code (#17301) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-28 06:20:45 -07:00
Nicolò Lucchesi 889ebb2638 [Misc] Minor typo/grammar in platforms/interface.py (#17307) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-04-28 05:45:42 -07:00
Cyrus Leung 344e193b7d [Bugfix] Add missing get_language_model to new MLLMs (#17300) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-28 04:09:57 -07:00
Harry Mellor fb1c933ade Add missing class docstring for PromptAdapterConfig (#17302) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-28 04:06:59 -07:00
idouba 72c5b97231 Update tpu_worker.py 's typo (#17288) 2025-04-28 04:01:15 -07:00
Alex Brooks fa93cd9f60 [Model] Add Granite Speech Support (#16246) 
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2025-04-28 10:05:00 +00:00
Cyrus Leung aec9674dbe [Core] Remove legacy input mapper/processor from V0 (#15686) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-28 15:38:48 +08:00
Wanrui Daiandevian 7fcc4223dc [Minor][Models] Pass partial_rotary_factor parameter to rope (#17266) 
Signed-off-by: evian <eviantai@u.nus.edu>
Co-authored-by: evian <eviantai@u.nus.edu>
 2025-04-28 04:28:59 +00:00
Nick Hill 8262a3e23b [Misc] Validate stop_token_ids contents (#17268) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-04-28 03:54:05 +00:00
Michael Goin cb3f2d8d10 [Bugfix] Fix Mistral3 spatial merge error (#17270) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-27 19:40:05 -07:00
Lucas WilkinsonJee Jee LiAaron Pham
 		d8bccde686 [BugFix] Fix vllm_flash_attn install issues (#17267) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
 2025-04-27 17:27:56 -07:00
Lily Liu 20e489eaa1 [V1][Spec Decode] Make eagle compatible with prefix caching. (#17137) 
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
 2025-04-27 09:29:43 -07:00
Cyrus Leung 4213475ec7 [Metrics] Fix minor inconsistencies in bucket progression (#17262) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-27 16:19:39 +00:00
cascadeandTyler Michael Smith 690fe019f0 [Feature] support sequence parallelism using compilation pass (#16155) 
Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-04-27 06:29:35 -07:00
Kaixi Hou ed7a29d9f8 [NVIDIA] Support Cutlass MLA for Blackwell GPUs (#16032) 
Signed-off-by: kaixih <kaixih@nvidia.com>
 2025-04-27 06:29:21 -07:00
Alex BrooksandJee Jee Li 756848e79e [Bugfix] Fix Lora Name Parsing (#17196) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-04-27 20:33:09 +08:00
Flex Wang 18445edd0f [Misc] Change buckets of histogram_iteration_tokens to [1, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8096] to represent number of tokens (#17033) 
Signed-off-by: sfc-gh-zhwang <flex.wang@snowflake.com>
 2025-04-27 12:30:53 +00:00
Jade Zheng 30215ca61f [MISC] Use string annotation types for class definitions (#17244) 
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
 2025-04-27 08:39:57 +00:00
Chen Zhang 838cedade7 [Bugfix] Get a specific type of layer from forward context (#17222) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-04-27 00:58:05 -07:00
Jee Jee Li 4283a28c2f [Bugfix] Fix QWen2 VL multimodal mapping (#17240) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-04-27 05:53:23 +00:00
Cyrus Leung 93a126fbc7 [Misc] Make cached tokenizer pickle-compatible (#17048) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-27 13:05:00 +08:00
rasmith 8e4b351a0c [Kernel][Triton][FP8] Adding fp8 and variable length sequence support to Triton FAv2 kernel (#12591) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-04-27 00:35:08 +00:00
Woosuk Kwon 43eea2953b [Minor] Fix lint error in main branch (#17233) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-04-26 11:10:14 -07:00
Kero Liang de7eb10ce4 [Bugfix] Fix Qwen2.5-Omni M-RoPE position ids generation (#16878) 
Signed-off-by: imkero <kerorek@outlook.com>
 2025-04-26 10:41:35 -07:00
Ning Xie fd11a325b8 [MISC] rename interval to max_recent_requests (#14285) 2025-04-26 16:59:18 +00:00
Lu Fang 4d17e20310 Disable the torch.compile cache checks when VLLM_DISABLE_COMPILE_CACHE=1 (#16573) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-04-26 09:17:58 -07:00
changjun.lee 10fd1d7380 [Bugfix] fix error due to an uninitialized tokenizer when using skip_tokenizer_init with num_scheduler_steps (#9276) 
Signed-off-by: changjun.lee <pord7457@gmail.com>
 2025-04-26 11:51:17 -04:00