iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
1981 Commits
Author SHA1 Message Date
Wentao Ye 774d0c014b [Perf] Cuda Kernel for Per Token Group Quant (#21083) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-22 07:27:15 -07:00
Duncan Mossjiahancmgoin
 		2c8db17cfd [feat]: add SM100 support for cutlass FP8 groupGEMM (#20447) 
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-22 07:27:12 -07:00
Mickaël Seznecandmgoin 4fb56914c5 [perf] Add fused MLA QKV + strided layernorm (#21116) 
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-22 07:07:44 -07:00
Shu Wang 9e23ad9655 Update fp4 quantize API (#21327) 
Signed-off-by: Shu Wang <shuw@nvidia.com>
 2025-07-21 23:40:21 -07:00
Ming Yang e7b2042681 Revert "[Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) (#21334) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-07-21 21:49:01 -07:00
Himanshu Jaju 0ec82edda5 [perf] Speed up align sum kernels (#21079) 
Signed-off-by: Himanshu Jaju <hj@mistral.ai>
 2025-07-21 11:19:23 -07:00
Zhiyu 6b46c4b653 Add Nvidia ModelOpt config adaptation (#19815) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-07-21 10:02:58 -04:00
Cyrus Leung 042af0c8d3 [Model][1/N] Support multiple poolers at model level (#21227) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-21 02:22:21 -07:00
Thomas Parnell 881e3cbe3b [V1] [Hybrid] Enable piecewise CUDA Graph for mamba layers (#21194) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-07-19 19:27:21 +00:00
Kaixi Houandmgoin 6d0734c562 [NVIDIA] Add SM100 Flashinfer MoE blockscale fp8 backend for low latency (#20645) 
Signed-off-by: kaixih <kaixih@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-19 02:33:01 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath dcc6cfb991 [Kernel][Performance] Tweak MoE Batched silu_mul_fp8_quant_deep_gemm kernel (#21193) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-18 23:09:51 -07:00
Woosuk Kwon dd572c0ab3 [V0 Deprecation] Remove V0 Spec Decode workers (#21152) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-18 21:47:50 -07:00
Rui Qiao 217937221b Elastic Expert Parallel Initial Support (#20775) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-07-18 17:46:09 -07:00
Richard Zou b2eb2b5ad7 [Kernel] Apply torch.Tag.needs_fixed_stride_order only for torch==2.6.0 (#19346) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-07-18 14:10:21 -04:00
Cyrus Leung 45badd05d0 [Core] Set pooling params based on task and model (#21128) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-18 05:41:17 -07:00
ElizaWszola 4adc66f64d [Bugfix] Allocate less memory in non-batched CUTLASS MoE (#21121) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
 2025-07-18 18:55:52 +08:00
Shu Wangandmgoin c7d8724e78 [Core] FlashInfer CUTLASS fused MoE backend (NVFP4) (#20037) 
Signed-off-by: shuw <shuw@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-17 21:32:45 -07:00
Wentao Ye 8a8fc94639 [Log] Debugging Log with more Information (#20770) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-18 00:19:46 +00:00
Woosuk Kwon 4de7146351 [V0 deprecation] Remove V0 HPU backend (#21131) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-17 16:37:36 -07:00
Cyrus Leung 90bd2ab6e3 [Model] Update pooling model interface (#21058) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-17 16:05:40 +00:00
ElizaWszola 9fb2d22032 [Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
 2025-07-17 09:56:44 -04:00
Harry Mellor fe8a2c544a [Docs] Improve docstring formatting for FusedMoEParallelConfig.make (#21117) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-17 04:13:00 -07:00
Asher 5a7fb3ab9e [Model] Add ToolParser and MoE Config for Hunyuan A13B (#20820) 
Signed-off-by: Asher Zhang <asherszhang@tencent.com>
 2025-07-17 09:10:09 +00:00
Varun Sundar RabindranathandVarun Sundar Rabindranath 11dfdf21bf [Kernel] DeepGemm MoE : Integrate triton permute / unpermute kernels (#20903) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-17 08:10:37 +00:00
Michael Goin 28a6d5423d [Bugfix] Fix Machete zero point issue for GPTQ models on SM90 (#21066) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-16 19:54:45 -07:00
Kevin_Xiong c9ba8104ed [Bugfix] weight loading use correct tp_group with patch_tensor_parallel_group (#21024) 
Signed-off-by: KevinXiong-C <kevin_xiong1997@outlook.com>
 2025-07-16 19:36:36 -07:00
Nir DavidandUri Livne 01513a334a Support FP8 Quantization and Inference Run on Intel Gaudi (HPU) using INC (Intel Neural Compressor) (#12010) 
Signed-off-by: Nir David <ndavid@habana.ai>
Signed-off-by: Uri Livne <ulivne@habana.ai>
Co-authored-by: Uri Livne <ulivne@habana.ai>
 2025-07-16 15:33:41 -04:00
Cyrus Leung 1c3198b6c4 [Model] Consolidate pooler implementations (#20927) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-16 13:39:13 +00:00
Peter Pan 1eb2b9c102 [CI] update typos config for CI pre-commit and fix some spells (#20919) 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
 2025-07-15 21:12:40 -07:00
Wentao Ye 76ddeff293 [Doc] Remove duplicate docstring (#21012) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-15 20:09:13 -07:00
Ming Yang fcb9f879c1 [Bugfix] Correct per_act_token in CompressedTensorsW8A8Fp8MoECutlassM… (#20937) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-07-15 19:53:42 -07:00
Tuan, Hoang-TrongandTuan M. Hoang-Trong f29fd8a7f8 [BugFix] fix 3 issues: (1) using metadata for causal-conv1d, (2) indexing overflow in v1 vLLM, and (3) init_states in v0 (#20838) 
Signed-off-by: Tuan M. Hoang-Trong <tmhoangt@us.ibm.com>
Co-authored-by: Tuan M. Hoang-Trong <tmhoangt@us.ibm.com>
 2025-07-15 16:08:26 -04:00
Thomas Parnell 3534c39a20 [V1] [Hybrid] Refactor mamba state shape calculation; enable V1 via cli (#20840) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-07-15 04:04:35 -07:00
Ruheena Suhani Shaik 016b8d1b7f Enabled BnB NF4 inference on Gaudi (#20172) 
Signed-off-by: Ruheena Suhani Shaik <rsshaik@habana.ai>
 2025-07-14 20:26:08 -07:00
XiongfeiWei d4170fad39 Use w8a8 quantized matmul Pallas kernel (#19170) 
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
 2025-07-15 03:06:33 +00:00
Michael Goin bcdfb2a330 [Bugfix] Fix incorrect dispatch for CutlassBlockScaledGroupedGemm and DeepGEMM (#20933) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-15 01:42:17 +00:00
Thomas Parnell 86f3ac21ce Fix overflow indexing in causal_conv1d kernel (#20938) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-07-14 21:43:07 +00:00
Varun Sundar RabindranathandVarun Sundar Rabindranath c0569dbc82 [Misc] ModularKernel : Perform WeightAndReduce inside TritonExperts & DeepGemmExperts (#20725) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-14 19:47:16 +00:00
TJian 80d38b8ac8 [V1] [ROCm] [AITER] Upgrade AITER to commit 916bf3c and bugfix APIs (#20880) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-07-13 15:19:32 +00:00
Wentao Ye c1acd6d7d4 [Refactor] Change the way of import triton (#20774) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-12 19:39:55 -07:00
ElizaWszola 3b3b778d4a [Bugfix] Fix a couple PPLX+CUTLASS MoE bugs (#20825) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
 2025-07-12 19:39:14 -07:00
Wentao Ye 42d440c22b [Perf] Use Triton instead of Torch for DeepGEMM Per Token Group Quant (#20841) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-12 19:38:45 -07:00
Michael Goin 6e2c176e1f [Bugfix] Restrict Machete to only run on Hopper (#20830) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-12 17:34:40 +00:00
Congcong Chen 2c11a738b3 [Model] New model support for microsoft/Phi-4-mini-flash-reasoning (#20702) 
Signed-off-by: Congcong Chen <congcongchen@microsoft.com>
 2025-07-12 06:02:10 -07:00
Zhiyu 4afe687a82 Enable ModelOpt Llama4 fp8 checkpoint deployment (#20419) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-07-11 23:07:16 -07:00
Wentao Ye 0d4891cd03 [Bug] Fix DeepGemm for EP low latency case (#20833) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-11 23:05:12 -07:00
Varun Sundar Rabindranath 890323dc1b [Bugfix] : Fix typo - logger.warn_once -> logger.warning_once (#20852) 2025-07-11 20:56:24 -07:00
Li, Jiang b1235c3e10 [Bugfix] Lazy import fused_experts in BitsAndBytesMoEMethod to avoid break not-cuda-alike devices (#20822) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-07-11 20:52:05 -07:00
Jee Jee Li 44d02f54db [Misc] Restrict deep_gemm's log output (#20827) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-11 20:50:42 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath 53fa457391 [Misc] Add unit tests for MoE ModularKernel combinations + Profiling utility (#20449) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-11 07:51:46 -07:00