iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
7,797 Commits 3 Branches 0 Tags
4adc66f64d56338489d00d94de6e13d95741c4be
Commit Graph
2118 Commits
Author SHA1 Message Date
ElizaWszola 4adc66f64d [Bugfix] Allocate less memory in non-batched CUTLASS MoE (#21121) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
 2025-07-18 18:55:52 +08:00
wang.yuqi ca4eb82bcb [Model] Re-add the implicit conversion feature for as_seq_cls_model (#21103) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-07-18 07:15:07 +00:00
Shu Wangandmgoin c7d8724e78 [Core] FlashInfer CUTLASS fused MoE backend (NVFP4) (#20037) 
Signed-off-by: shuw <shuw@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-17 21:32:45 -07:00
Wentao Ye 8a8fc94639 [Log] Debugging Log with more Information (#20770) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-18 00:19:46 +00:00
Woosuk Kwon 4de7146351 [V0 deprecation] Remove V0 HPU backend (#21131) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-17 16:37:36 -07:00
Jee Jee Li a3a6c695f4 [Misc] Qwen MoE model supports LoRA (#20932) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-17 18:32:52 +00:00
Cyrus Leung 90bd2ab6e3 [Model] Update pooling model interface (#21058) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-17 16:05:40 +00:00
ElizaWszola 9fb2d22032 [Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
 2025-07-17 09:56:44 -04:00
Harry Mellor fe8a2c544a [Docs] Improve docstring formatting for FusedMoEParallelConfig.make (#21117) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-17 04:13:00 -07:00
kYLeandCyrus Leung 4ef00b5cac [VLM] Add Nemotron-Nano-VL-8B-V1 support (#20349) 
Signed-off-by: Kyle Huang <kylhuang@nvidia.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-07-17 03:07:55 -07:00
Asher 5a7fb3ab9e [Model] Add ToolParser and MoE Config for Hunyuan A13B (#20820) 
Signed-off-by: Asher Zhang <asherszhang@tencent.com>
 2025-07-17 09:10:09 +00:00
Varun Sundar RabindranathandVarun Sundar Rabindranath 11dfdf21bf [Kernel] DeepGemm MoE : Integrate triton permute / unpermute kernels (#20903) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-17 08:10:37 +00:00
Jee Jee Li c5b8b5953a [Misc] Fix PhiMoE expert mapping (#21085) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-17 05:47:49 +00:00
Michael Goin 28a6d5423d [Bugfix] Fix Machete zero point issue for GPTQ models on SM90 (#21066) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-16 19:54:45 -07:00
Kevin_Xiong c9ba8104ed [Bugfix] weight loading use correct tp_group with patch_tensor_parallel_group (#21024) 
Signed-off-by: KevinXiong-C <kevin_xiong1997@outlook.com>
 2025-07-16 19:36:36 -07:00
Nir DavidandUri Livne 01513a334a Support FP8 Quantization and Inference Run on Intel Gaudi (HPU) using INC (Intel Neural Compressor) (#12010) 
Signed-off-by: Nir David <ndavid@habana.ai>
Signed-off-by: Uri Livne <ulivne@habana.ai>
Co-authored-by: Uri Livne <ulivne@habana.ai>
 2025-07-16 15:33:41 -04:00
Cyrus Leung ac2bf41e53 [Model] Remove model sampler (#21059) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-16 19:03:37 +00:00
Avshalom Manevich a0f8a79646 [fix] fix qwen image_embeds input (#21049) 
Signed-off-by: h-avsha <avshalom.manevich@hcompany.ai>
 2025-07-16 15:17:20 +00:00
Cyrus Leung 1c3198b6c4 [Model] Consolidate pooler implementations (#20927) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-16 13:39:13 +00:00
zhiweizandqizixi c11013db8b [Meta] Llama4 EAGLE Support (#20591) 
Signed-off-by: qizixi <qizixi@meta.com>
Co-authored-by: qizixi <qizixi@meta.com>
 2025-07-15 21:14:15 -07:00
Peter Pan 1eb2b9c102 [CI] update typos config for CI pre-commit and fix some spells (#20919) 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
 2025-07-15 21:12:40 -07:00
Wentao Ye 76ddeff293 [Doc] Remove duplicate docstring (#21012) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-15 20:09:13 -07:00
Michael Goin f46098335b [Bugfix] Fix Mistral3 support on SM100/SM120 (#20998) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-15 20:08:41 -07:00
Ming Yang fcb9f879c1 [Bugfix] Correct per_act_token in CompressedTensorsW8A8Fp8MoECutlassM… (#20937) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-07-15 19:53:42 -07:00
Brayden Zhong 75a99b98bf [Chore] Remove outdated transformers check (#20989) 
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-07-15 19:42:40 -07:00
Thomas Parnell 6cbc4d4bea [Model] Add ModelConfig class for GraniteMoeHybrid to override default max_seq_len_to_capture (#20923) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-07-15 19:19:10 -07:00
Tuan, Hoang-TrongandTuan M. Hoang-Trong f29fd8a7f8 [BugFix] fix 3 issues: (1) using metadata for causal-conv1d, (2) indexing overflow in v1 vLLM, and (3) init_states in v0 (#20838) 
Signed-off-by: Tuan M. Hoang-Trong <tmhoangt@us.ibm.com>
Co-authored-by: Tuan M. Hoang-Trong <tmhoangt@us.ibm.com>
 2025-07-15 16:08:26 -04:00
Patrick von PlatenandCyrus Leung e7e3e6d263 Voxtral (#20970) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-07-15 07:35:30 -07:00
Thomas Parnell 3534c39a20 [V1] [Hybrid] Refactor mamba state shape calculation; enable V1 via cli (#20840) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-07-15 04:04:35 -07:00
Jennifer He 85bd6599e4 [Model] Add AutoWeightsLoader support for BERT, RoBERTa (#20534) 
Signed-off-by: Jennifer He <islandhe@gmail.com>
Signed-off-by: <islandhe@gmail.com>
Signed-off-by: Jen H <islandhe@gmail.com>
 2025-07-15 13:34:24 +08:00
Ruheena Suhani Shaik 016b8d1b7f Enabled BnB NF4 inference on Gaudi (#20172) 
Signed-off-by: Ruheena Suhani Shaik <rsshaik@habana.ai>
 2025-07-14 20:26:08 -07:00
XiongfeiWei d4170fad39 Use w8a8 quantized matmul Pallas kernel (#19170) 
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
 2025-07-15 03:06:33 +00:00
Michael Goin bcdfb2a330 [Bugfix] Fix incorrect dispatch for CutlassBlockScaledGroupedGemm and DeepGEMM (#20933) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-15 01:42:17 +00:00
Thomas Parnell 86f3ac21ce Fix overflow indexing in causal_conv1d kernel (#20938) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-07-14 21:43:07 +00:00
Varun Sundar RabindranathandVarun Sundar Rabindranath c0569dbc82 [Misc] ModularKernel : Perform WeightAndReduce inside TritonExperts & DeepGemmExperts (#20725) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-14 19:47:16 +00:00
ant-yy 38efa28278 [Model] Add Ling implementation (#20680) 
Signed-off-by: vito.yy <vito.yy@antgroup.com>
 2025-07-14 22:10:32 +08:00
Jee Jee Li a99b9f7dee [Quantization] add BNB for MixtralForCausalLM (#20893) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-14 07:34:34 +00:00
TJian 80d38b8ac8 [V1] [ROCm] [AITER] Upgrade AITER to commit 916bf3c and bugfix APIs (#20880) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-07-13 15:19:32 +00:00
Minkyu Kim bd4c1e6fdb Support for LlamaForSequenceClassification (#20807) 
Signed-off-by: thechaos16 <thechaos16@gmail.com>
 2025-07-13 00:09:34 -07:00
Nicolò LucchesiandDarkLight1337 020f58abcd [Core] Support multiple tasks per model (#20771) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-12 19:40:11 -07:00
Wentao Ye c1acd6d7d4 [Refactor] Change the way of import triton (#20774) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-12 19:39:55 -07:00
ElizaWszola 3b3b778d4a [Bugfix] Fix a couple PPLX+CUTLASS MoE bugs (#20825) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
 2025-07-12 19:39:14 -07:00
Wentao Ye 42d440c22b [Perf] Use Triton instead of Torch for DeepGEMM Per Token Group Quant (#20841) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-12 19:38:45 -07:00
Michael Goin 6e2c176e1f [Bugfix] Restrict Machete to only run on Hopper (#20830) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-12 17:34:40 +00:00
Alex Brooks c2a2f19aba [Bugfix] Fix Tensor Parallelism Padding Consistency in Granite Models (#20843) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2025-07-12 06:11:30 -07:00
Congcong Chen 2c11a738b3 [Model] New model support for microsoft/Phi-4-mini-flash-reasoning (#20702) 
Signed-off-by: Congcong Chen <congcongchen@microsoft.com>
 2025-07-12 06:02:10 -07:00
Zhiyu 4afe687a82 Enable ModelOpt Llama4 fp8 checkpoint deployment (#20419) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-07-11 23:07:16 -07:00
Wentao Ye 0d4891cd03 [Bug] Fix DeepGemm for EP low latency case (#20833) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-11 23:05:12 -07:00
Nicolò Lucchesi 3c7d942da8 [Frontend] Abstract prompt and SpeechToTextConfig for transcriptions models (#20637) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-07-11 21:33:26 -07:00
Varun Sundar Rabindranath 890323dc1b [Bugfix] : Fix typo - logger.warn_once -> logger.warning_once (#20852) 2025-07-11 20:56:24 -07:00