iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
4236 Commits
Author SHA1 Message Date
pengyuangeandjiacai.liu de1cb38769 [Model] Support Skywork-R1V (#15397) 
Signed-off-by: jiacai.liu <932997367@qq.com>
Co-authored-by: jiacai.liu <932997367@qq.com>
 2025-03-28 20:39:21 -07:00
Russell Bryant 7329ff5468 [V1] Support disable_any_whtespace for guidance backend (#15584) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-28 23:46:45 +08:00
Cyrus Leung 541d1df486 [Bugfix] embed_is_patch for Idefics3 (#15696) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-28 08:27:52 -07:00
Jee Jee Li 91276c5721 [Model] Adding torch compile annotations to chatglm (#15624) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-28 21:14:09 +08:00
Ce Gao 32b14baf8a [Refactor][Frontend] Keep all logic about reasoning into one class (#14428) 
Signed-off-by: Ce Gao <cegao@tensorchord.ai>
 2025-03-28 00:23:30 -07:00
Cyrus Leung 8693e47e6a [Bugfix] Fix mm_hashes forgetting to be passed (#15668) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-28 05:51:05 +00:00
Gregory ShtrasbergandLuka Govedič 4d0ec37267 [Quantization][FP8] Adding support for fp8 gemm layer input in fp8 (#14578) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-03-28 02:58:16 +00:00
Wes 4ae17bf1e2 Revert "Use Cache Hinting for fused_moe kernel (#15511)" (#15645) 
Signed-off-by: Wes Medford <wryanmedford@gmail.com>
 2025-03-27 19:45:55 -07:00
Jee Jee Li 726efc6a32 [Quantization][V1] BitsAndBytes support V1 (#15611) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-28 10:12:47 +08:00
Cyrus Leung 247181536f [Misc] Replace is_encoder_decoder_inputs with split_enc_dec_inputs (#15620) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-27 17:36:32 +00:00
Cyrus Leung ac5bc615b0 [Model] MiniCPM-V/O supports V1 (#15487) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-27 06:07:29 -07:00
Cyrus Leung e6c9053f9e [Misc] Clean up scatter_patch_features (#15559) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-27 07:45:00 +00:00
Robert ShawElizaWszolaLucas WilkinsonElizaWszola
 		43ed4143c4 [Quantization] Fp8 Channelwise Dynamic Per Token GroupedGEMM (#15587) 
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
 2025-03-27 06:47:25 +00:00
Robert Shaw e1e0fd7543 [TPU] Avoid Triton Import (#15589) 
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
 2025-03-27 06:43:02 +00:00
Jerry Zhang dcf2a590f5 Allow torchao quantization in SiglipMLP (#15575) 2025-03-26 22:45:51 -07:00
Mengqing Cao fb22be5817 [moe][quant] add weight name case for offset (#15515) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-03-27 04:50:29 +00:00
ElizaWszolaandLucas Wilkinson 9239bf718e [Kernel] CUTLASS grouped gemm fp8 MoE kernel (#13972) 
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
 2025-03-27 00:54:44 +00:00
Wes 7a888271f5 Use Cache Hinting for fused_moe kernel (#15511) 2025-03-26 23:21:34 +00:00
Harry Mellor 0af4d764d6 Fix weight loading for some models in Transformers backend (#15544) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-26 10:17:53 -07:00
Alex BrooksandJoe Runde 1711b929b6 [Model] Add Reasoning Parser for Granite Models (#14202) 
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
 2025-03-26 14:28:07 +00:00
Harry Mellor c091c0a588 Improve validation of TP in Transformers backend (#15540) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-26 07:26:48 -07:00
cyyever 1aa162e030 Apply torchfix (#15532) 
Signed-off-by: cyy <cyyever@outlook.com>
 2025-03-26 12:09:06 +00:00
Harry MellorIsotr0pyIsotr0py
 		cf5c8f1686 Separate base model from TransformersModel (#15467) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-03-26 18:13:38 +08:00
vllmellmandtjtanaa 5ebf66748b [FEAT][ROCm] Integrate Fused MoE Kernels from AITER (#14967) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-03-26 16:30:30 +08:00
Bryan LuandCyrus Leung 781d056280 [Feature] Enhance EAGLE Architecture with Proper RMS Norms (#14990) 
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-03-26 08:24:07 +00:00
Cyrus Leung 997c8811d6 [Model] Support multi-image for Molmo (#15438) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-26 11:26:33 +08:00
Harry Mellor e42389f9d7 Transformers backend already supports V1 (#15463) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-25 20:26:16 -07:00
Cyrus Leung 5994430b84 [Misc] Remove redundant num_embeds (#15443) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-25 18:27:57 +08:00
Cyrus Leung a9e879b316 [Misc] Clean up MiniCPM-V/O code (#15337) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-25 10:22:52 +00:00
Tyler Michael Smith b5269db959 Revert "Fix non-contiguous input passed to Marlin kernel (#15319)" (#15398) 2025-03-24 20:43:51 -07:00
Harry MellorandIsotr0py 97cfa65df7 Add pipeline parallel support to TransformersModel (#12832) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-03-25 10:41:45 +08:00
Gregory Shtrasbergandcharlifu f533b5837f [ROCm][Kernel] MoE weights padding (#14454) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: charlifu <charlifu@amd.com>
 2025-03-24 23:45:30 +00:00
Gregory Shtrasberg 8279201ce6 [Build] Cython compilation support fix (#14296) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-03-24 23:37:54 +00:00
liuzhenwei 5eeadc2642 [Hardware][Gaudi][Feature] Enable Dynamic MoE for Mixtral (#12303) 
Signed-off-by: zhenwei <zhenweiliu@habana.ai>
 2025-03-24 09:48:40 -07:00
Manish Sethi 761702fd19 [Core] Integrate fastsafetensors loader for loading model weights (#10647) 
Signed-off-by: Manish Sethi <Manish.sethi1@ibm.com>
 2025-03-24 08:08:02 -07:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 948ab03e7e [Bugfix][V1] Avoid importing PreTrainedModel (#15366) 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
 2025-03-24 10:33:12 +00:00
Jee Jee Li 3892e58ad7 [Misc] Upgrade BNB version (#15183) 2025-03-24 05:51:42 +00:00
Qubitium-ModelCloud d20e261199 Fix non-contiguous input passed to Marlin kernel (#15319) 2025-03-24 03:09:44 +00:00
DefTruth f90d34b498 [Misc] Add tuned R1 w8a8 and MoE configs for NVIDIA L20 (#15322) 
Signed-off-by: DefTruth <qiustudent_r@163.com>
 2025-03-23 01:10:10 -07:00
Russell Bryant b877031d80 Remove openvino support in favor of external plugin (#15339) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-22 14:06:39 -07:00
Naitong YuJason Fangjiangxin
 		2f4bd358f1 [Model] Support Tele-FLM Model (#15023) 
Signed-off-by: Naitong Yu <ntyu@baai.ac.cn>
Signed-off-by: jiangxin <horizon94@outlook.com>
Co-authored-by: Jason Fang <jasonfang3900@gmail.com>
Co-authored-by: jiangxin <horizon94@outlook.com>
 2025-03-22 02:04:44 -07:00
TJian ec870fba9a [FEAT] [ROCm]: Add AITER RMS Norm (Layer Norm) Feature (#14959) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-03-21 22:36:14 -07:00
Mengqing Cao c21b99b912 [Bugfix][VLM] fix llava processor (#15285) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-03-21 05:14:36 -07:00
Isotr0py 84e00adc8a [Bugfix] Fix incorrect resolving order for transformers fallback (#15279) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-21 03:54:08 -07:00
Isotr0py 47c7126213 [Misc] Add attention mask pre-computation optimization back to Qwen2.5-VL (#15273) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-21 10:32:33 +00:00
Chih-Chieh Yang 296f927f24 [Model] RE: Mamba2 Prefill Performance Tweaks: Fixing Flurry of Unnecessary Memory Copies (#14857) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
 2025-03-20 19:21:08 -07:00
Isotr0py 1e508343e1 [Bugfix] Fix incorrect qwen2.5-vl attention mask pre-computation (#15200) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-20 19:18:04 -07:00
Yu Chin Fabian Lim 06dd08256f Enforce that TP > 1 is not supported for Mamba2 if Quantization is Enabled. (#14617) 
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
 2025-03-21 00:44:37 +00:00
Cyrus LeungandIsotr0py 27261e40a6 [Bugfix] Multi-video inference on LLaVA-Onevision (#15082) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-03-20 14:10:45 +00:00
billishyahaoandZhai Feiyue 742369d35a [Frontend][Bugfix] support prefill decode disaggregation on deepseek (#14824) 
Signed-off-by: billishyahao <bill.he@amd.com>
Co-authored-by: Zhai Feiyue <80079571+ZhaiFeiyue@users.noreply.github.com>
 2025-03-20 00:00:33 -07:00