iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
8,868 Commits 3 Branches 0 Tags
8a3cd90af534c39425ebfdfd295eea0a4582d541
Commit Graph
2465 Commits
Author SHA1 Message Date
Xin YangandWentao Ye 8a3cd90af5 [Kernel] Add fused grouped_topk kernel for MoE (#23274) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-08-25 11:47:52 -07:00
Isotr0py a9082a4d14 [Bugfix] Fix Qwen3 MoE GPTQ inference (#23490) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-08-25 06:40:20 -07:00
Cyrus Leung 6879cd80ae [Refactor] Pass tokenizer explicitly instead of binding to prompt update (#23542) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-25 06:31:57 -07:00
Ayush Satyam 5c4b6e66fe [Attention] Unify mamba and attention backend selection (#23171) 
Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
 2025-08-25 09:09:36 +00:00
Cyrus Leung ebafb0936d [Bugfix] Allow dynamic number of patches for llava_onevision (#23525) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-25 08:34:54 +00:00
Cyrus Leung 712d0f88d8 [Refactor] Dynamic target and content for prompt updates (#23411) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-24 23:39:58 -07:00
LIYIFAN_liyifan c9abb10489 [Bugfix] Fix Dense module loading for sentence-transformers embedding models (simplified V2) (#23408) 
Signed-off-by: FFFfff1FFFfff <yifanli0919@gmail.com>
 2025-08-25 05:39:24 +00:00
Benji Beck 787cdb3829 Migrate DonutImagePixelInputs to TensorSchema (#23509) 
Signed-off-by: Benji Beck <benjibeck@meta.com>
 2025-08-25 05:02:15 +00:00
Benji Beck a5203d04df Migrate skyworkr1v inputs to TensorSchema (#23499) 
Signed-off-by: Benji Beck <benjibeck@meta.com>
 2025-08-25 04:43:21 +00:00
Benji Beck 99f8094400 Migrate tarsier inputs to TensorSchema (#23500) 
Signed-off-by: Benji Beck <benjibeck@meta.com>
 2025-08-25 04:42:36 +00:00
Jee Jee Li 170e8ea9ea [Misc] Unified linear print info (#23516) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-24 20:13:51 -07:00
zifeitong a71e4765cc [Bugfix] Fix Qwen2.5-VL quantized model weights loading (#23512) 
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
 2025-08-25 10:40:22 +08:00
Ming Yang 504d914314 [Perf] Add Triton config for DeepSeek V3 FP8 EP32 H200 (#23504) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-08-24 18:06:35 -07:00
Cyrus Leung e2db1164a1 [Model] Enable BLOOM on V1 (#23488) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-24 13:30:47 +00:00
汪志鹏 416f05929a [New Model]Donut model (#23229) 
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2025-08-24 12:52:24 +00:00
czhu-cohere e76e233540 [kernel] Support W4A8 on Hopper (#23198) 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
 2025-08-24 06:18:04 +00:00
Benji Beck a75277285b Migrate Paligemma inputs to TensorSchema (#23470) 
Signed-off-by: Benji Beck <benjibeck@meta.com>
 2025-08-24 04:56:56 +00:00
Benji Beck 053278a5dc Migrate Pixtral inputs to TensorSchema (#23472) 
Signed-off-by: Benji Beck <benjibeck@meta.com>
 2025-08-24 04:55:53 +00:00
Chenxi YangandChenxi Yang 308fa287a8 Add glm4.5v tp2,4 fp8 config on H100_80GB (#23443) 
Co-authored-by: Chenxi Yang <cxyang@meta.com>
 2025-08-23 02:54:19 +00:00
Daifeng LiandMichael Goin fa78de9dc3 Quantization: support FP4 quantized models on AMD CDNA2/CDNA3 GPUs (#22527) 
Signed-off-by: feng <fengli1702@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-22 20:53:21 -06:00
WeiQing Chenandycyaw66 23c939fd30 [Model] Support DP for ViT on MiniCPM-V-4 (#23327) 
Signed-off-by: ycyaw66 <497410282@qq.com>
Co-authored-by: ycyaw66 <497410282@qq.com>
 2025-08-23 02:14:41 +00:00
elvischenvandLuka Govedič 24d0c9e6ed [NVIDIA][torch.compile] Support Flashinfer TRTLLM FP8-q/kv NVFP4-out Attention Kernel (#22703) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-08-22 22:09:05 +00:00
rasmith cc7ae5e7ca [BugFix][AMD][Quantization] Fix torch.compile issue where wvSplitKQ not being called when it should when using quantized FP8 model (#22281) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-08-22 21:47:57 +00:00
Isotr0py 4645024d3a [Quantization] Allow GGUF quantization to skip unquantized layer (#23188) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-08-22 13:04:22 -06:00
Isotr0pyand汪志鹏 cd7a3df26f [Bugfix] Fix broken Florence-2 model (#23426) 
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2025-08-22 17:50:52 +00:00
Isotr0py 32d2b4064f [Model] Add Ovis2.5 PP support (#23405) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-08-22 17:46:34 +00:00
bpppszouyu.zzxbppps
 		424fb7a5d2 [BugFix] Fix the issue where image embeddings were incorrectly split.… (#23366) 
Signed-off-by: bppps <bpppsaka@gmail.com>
Co-authored-by: zouyu.zzx <zouyu.zzx@alibaba-inc.com>
Co-authored-by: bppps <bpppsaka@gmail.com>
 2025-08-22 16:56:46 +00:00
PapaGoose 88491c1b6b [Speculators][Speculative Decoding] Fix Qwen 2 Eagle3 Support (#23337) 2025-08-22 16:39:19 +00:00
杨朱 · Kiki 695e7adcd2 [misc] Remove outdate comment about runai_model_streamer (#23421) 
Signed-off-by: carlory <baofa.fan@daocloud.io>
 2025-08-22 13:08:53 +00:00
Benji Beckandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 998720859c Migrate MiniCPMOAudioInputs to TensorSchema (#21847) 
Signed-off-by: Benji Beck <benjibeck@meta.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-22 16:43:29 +08:00
Chen Zhang 17373dcd93 [Attention] Refactor AttentionMetadata Preparation for Encoder-only Models (#23154) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-08-22 05:05:59 +00:00
Bin JiaandJee Jee Li 5964069367 [New Model] Add Seed-Oss model (#23241) 
Signed-off-by: jiabin.00 <jiabin.00@bytedance.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-22 04:58:10 +00:00
Wentao Ye 394591e343 [Feature] Enable DeepGEMM Linear on B200; 1.5% E2E throughput improvement (#23351) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-08-21 21:01:08 -07:00
Benji BeckandCyrus Leung 0b9cc56fac Migrate MllamaImagePixelInputs to TensorSchema (#22020) 
Signed-off-by: Benji Beck <benjibeck@meta.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-08-22 11:28:49 +08:00
Michael Goin 3bbe11cc13 [Perf] Small optimizations for silu_mul_fp8_quant_deep_gemm (#23265) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-08-21 17:56:15 -04:00
Wentao YeandMichael Goin 48bfb0c9b7 [Bug] Fix R1 Accuracy 0 Bug (#23294) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-21 13:11:28 -04:00
Yi LiuandWentao Ye 0278f1ac3a Fix nvfp4 swizzling (#23140) 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-08-21 16:54:50 +00:00
Benji Beck a482e4e769 Migrate MolmoImageInputs to TensorSchema (#22022) 
Signed-off-by: Benji Beck <benjibeck@meta.com>
 2025-08-21 16:54:08 +00:00
Roger WangandDarkLight1337 79f05e4436 [Multimodal] Always enable hashing mm data (#23308) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-21 07:23:28 -07:00
wang.yuqi d70a16625d [Performance] V1 Pooling Models E2E Performance Optimization (#23162) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-08-21 13:26:09 +00:00
Paul Pak 2e2000f352 [Model] Add LFM2 architecture (#22845) 
Signed-off-by: Paul Pak <paulpak58@gmail.com>
 2025-08-21 09:35:07 +02:00
Cyrus Leung 0c31e28e95 [Bugfix] Fix extra whitespace in strings caused by newline (#23272) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-20 22:03:00 -07:00
杨奇(yann qi)yannqiyanggemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
 		655a09f653 [Model][VLM] Support R-4B Model (#23246) 
Signed-off-by: yannqi <yannqi@qq.com>
Signed-off-by: 杨奇(yann qi) <51905299+yannqi@users.noreply.github.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: yannqiyang <yannqiyang@tencent.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-08-21 04:08:52 +00:00
Asaf Joseph Gardinandasafg 3663870c72 [V1][Mamba1] - Full CUDA and Piecewise CUDA Graphs Support (#23035) 
Signed-off-by: asafg <asafg@ai21.com>
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
Co-authored-by: asafg <asafg@ai21.com>
 2025-08-20 20:08:51 -07:00
Cyrus Leung 2461d9e562 [CI/Build] Split out mm processor tests (#23260) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-20 20:05:20 -07:00
Li, Jiang 7be5d113d8 [CPU] Refactor CPU W8A8 scaled_mm (#23071) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-08-21 09:34:24 +08:00
22quinnandSimon Mo 4b795020ed [EP] Add logging for experts map (#22685) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2025-08-20 23:46:06 +00:00
rongfu.lengandHarry Mellor 4fbda0b20c [Feature] use --eplb_config to set eplb param (#20562) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-20 14:07:28 -07:00
Michael Goin 0cdbf5e61c [Kernel/Quant] Remove the original marlin format and qqq (#23204) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-08-20 15:13:36 -04:00
dongluw ebe56a0064 Small fix for Command-A-Vision (#23268) 
Signed-off-by: donglu <donglu@cohere.com>
 2025-08-20 18:15:18 +00:00