 Tyler Michael Smith
|
72c62eae5f
|
[V1] EP/TP MoE + DP Attention (#13931)
|
2025-03-04 21:27:26 -08:00 |
|
|
rainkertanddangshunya
|
0df25101d6
|
[Bugfix] Fix gptq_marlin for deepseek-v3 (#13750)
Signed-off-by: dangshunya <dangshunya@baichuan-inc.com>
Co-authored-by: dangshunya <dangshunya@baichuan-inc.com>
|
2025-03-05 12:25:53 +08:00 |
|
|
Divakar Verma
|
bb5b640359
|
[core] moe fp8 block quant tuning support (#14068)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
|
2025-03-04 01:30:23 +00:00 |
|
|
Michael Goin
|
19d98e0c7d
|
[Kernel] Optimize moe intermediate_cache usage (#13625)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2025-03-03 16:29:53 -05:00 |
|
|
Mengqing Cao
|
b87c21fc89
|
[Misc][Platform] Move use allgather to platform (#14010)
Signed-off-by: Mengqing Cao <cmq0113@163.com>
|
2025-03-03 15:40:04 +08:00 |
|
|
YajieWang
|
6a92ff93e1
|
[Misc][Kernel]: Add GPTQAllSpark Quantization (#12931)
|
2025-02-28 22:30:59 -08:00 |
|
|
Luka Govedič
|
bd56c983d6
|
[torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename RedundantReshapesPass to NoopEliminationPass (#10902)
Signed-off-by: luka <luka@neuralmagic.com>
|
2025-02-28 16:20:11 -07:00 |
|
|
Jee Jee Li
|
e0734387fb
|
[Bugfix] Fix MoeWNA16Method activation (#14024)
|
2025-02-28 15:22:42 +00:00 |
|
|
Jee Jee Li
|
67fc426845
|
[Misc] Print FusedMoE detail info (#13974)
|
2025-02-27 18:53:13 -05:00 |
|
|
qli88
|
8294773e48
|
[core] Perf improvement for DSv3 on AMD GPUs (#13718)
Signed-off-by: qli88 <qiang.li2@amd.com>
|
2025-02-27 22:14:30 +00:00 |
|
|
Szymon Ożóg
|
7f0be2aa24
|
[Model] Deepseek GGUF support (#13167)
|
2025-02-27 02:08:35 -08:00 |
|
|
Woosuk Kwon
|
b382a7f28f
|
[BugFix] Make FP8 Linear compatible with torch.compile (#13918)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-02-26 13:48:55 -08:00 |
|
|
Michael Goin
|
07c4353057
|
[Model] Support Grok1 (#13795)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2025-02-26 01:07:12 +00:00 |
|
|
|
f75aa72732
|
[Neuron] Add custom_ops for neuron backend (#13246)
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: George Novack <gnovack@amazon.com>
Co-authored-by: Aoyu Zhang <aoyuzhan@amazon.com>
|
2025-02-25 11:47:49 -08:00 |
|
|
Michael Goin
|
4d251ad00e
|
Fix CompressedTensorsWNA16MoE with grouped scales (#13769)
|
2025-02-25 00:17:14 -08:00 |
|
|
Tyler Michael Smith
|
1e15aaef56
|
[Bugfix][Quantization] Fix FP8 + EP (#13784)
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2025-02-25 10:54:17 +08:00 |
|
|
cjackal
|
51010a1807
|
[Misc] set single whitespace between log sentences (#13771)
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
|
2025-02-25 10:26:12 +08:00 |
|
|
Harry Mellor
|
cdc1fa12eb
|
Remove unused kwargs from model definitions (#13555)
|
2025-02-24 17:13:52 -08:00 |
|
|
Michael Goin
|
db986c19ea
|
Fix precommit fail in fused_moe intermediate_cache2 chunking (#13772)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2025-02-24 09:25:47 -08:00 |
|
|
Zhonghua Deng
|
ccc00515fd
|
[BugFix] Illegal memory access for MoE On H20 (#13693)
|
2025-02-24 07:37:32 -08:00 |
|
|
Jongseok Park
|
781096e385
|
Expert Parallelism (EP) Support for DeepSeek V2 (#12583)
|
2025-02-24 07:33:20 -08:00 |
|
|
Mengqing Cao
|
23eca9cf68
|
[model][refactor] remove cuda hard code in models and layers (#13658)
|
2025-02-24 06:10:14 -08:00 |
|
|
Gregory Shtrasberg
|
c904fdddf6
|
[ROCm] Apply FP8 weights padding to values not divisible by 512 bytes on ROCm (#13231)
|
2025-02-22 05:54:38 -08:00 |
|
|
Cyrus Leung
|
7f6bae561c
|
[CI/Build] Fix pre-commit errors (#13696)
|
2025-02-22 00:31:26 -08:00 |
|
|
Yu Chin Fabian Lim
|
fca20841c2
|
Correction to TP logic for Mamba Mixer 2 when Num Groups not divisible by TP Size (#13660)
|
2025-02-22 00:19:10 -08:00 |
|
|
|
288cc6c234
|
[Attention] MLA with chunked prefill (#12639)
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2025-02-21 15:30:12 -08:00 |
|
|
chenxiaobing
|
ed6e9075d3
|
[Bugfix] Fix deepseekv3 grouped topk error (#13474)
Signed-off-by: Chen-XiaoBing <chenxb002@whu.edu.cn>
|
2025-02-20 06:47:01 -08:00 |
|
|
Divakar Verma
|
0d243f2a54
|
[ROCm][MoE] mi300 mixtral8x7B perf for specific BS (#13577)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
|
2025-02-20 04:01:02 +00:00 |
|
|
Divakar Verma
|
8aada19dfc
|
[ROCm][MoE configs] mi325 mixtral & mi300 qwen_moe (#13503)
|
2025-02-18 22:23:24 -08:00 |
|
|
Nick Hill
|
30172b4947
|
[V1] Optimize handling of sampling metadata and req_ids list (#13244)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-02-18 12:15:33 -08:00 |
|
|
Harry Mellor
|
efbe854448
|
[Misc] Remove dangling references to SamplingType.BEAM (#13402)
|
2025-02-17 19:52:35 -08:00 |
|
|
Tyler Michael Smith
|
b3942e157e
|
[Bugfix][CI][V1] Work around V1 + CUDA Graph + torch._scaled_mm fallback issue (#13425)
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2025-02-18 00:32:48 +00:00 |
|
|
Tyler Michael SmithandYu Chin Fabian Lim
|
1f69c4a892
|
[Model] Support Mamba2 (Codestral Mamba) (#9292)
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
|
2025-02-17 20:17:50 +08:00 |
|
|
Kyle Sayers
|
12913d17ba
|
[Quant] Add SupportsQuant to phi3 and clip (#13104)
|
2025-02-15 19:28:33 -08:00 |
|
|
rasmith
|
ed0de3e4b8
|
[AMD] [Model] DeepSeek tunings (#13199)
|
2025-02-15 03:58:09 -08:00 |
|
|
|
3bcb8c75da
|
[Core] Reduce TTFT with concurrent partial prefills (#10235)
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
|
2025-02-14 15:36:07 -08:00 |
|
|
Michael Goin
|
5e5c8e091e
|
[Quant][Perf] Use moe_wna16 kernel by default for MoEs with many experts (#13236)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2025-02-14 12:53:42 -08:00 |
|
|
Tyler Michael Smith
|
c1e37bf71b
|
[Kernel][Bugfix] Refactor and Fix CUTLASS 2:4 Sparse Kernels (#13198)
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2025-02-14 00:01:14 +00:00 |
|
|
Michael Goin
|
2344192a55
|
Optimize moe_align_block_size for deepseek_v3 (#12850)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2025-02-13 18:43:37 -05:00 |
|
|
Daniel Han
|
cb944d5818
|
Allow Unsloth Dynamic 4bit BnB quants to work (#12974)
|
2025-02-12 23:13:08 -08:00 |
|
|
Michael Goin
|
09972e716c
|
[Bugfix] Allow fallback to AWQ from AWQMarlin at per-layer granularity (#13119)
|
2025-02-12 09:19:53 -08:00 |
|
|
Qubitium-ModelCloud
|
36a08630e8
|
[CORE] [QUANT] Support for GPTQModel's dynamic quantization per module override/control (#7086)
|
2025-02-12 09:19:43 -08:00 |
|
|
Szymon Ożóg
|
2b25b7d2e1
|
Fix initializing GGUF weights for ColumnParallelLinear when using tensor parallel > 1 (#13023)
|
2025-02-11 08:38:48 -08:00 |
|
|
Sanju C Sudhakaran
|
2880e21e3d
|
[Hardware][Intel-Gaudi] Enable long-contexts + LoRA support for Intel Gaudi (#12812)
Signed-off-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
|
2025-02-08 17:15:30 +08:00 |
|
|
TJian
|
eaa92d4437
|
[ROCm] [Feature] [Doc] [Dockerfile] [BugFix] Support Per-Token-Activation Per-Channel-Weight FP8 Quantization Inferencing (#12501)
|
2025-02-07 08:13:43 -08:00 |
|
|
Amit Garg
|
538fab93cd
|
PR #12718 (#12718)
|
2025-02-07 06:22:37 -08:00 |
|
|
Lu Fang
|
741429a4cd
|
[MISC] Check space in the file names in the pre commit checks (#12804)
Signed-off-by: Lu Fang <lufang@fb.com>
|
2025-02-06 15:36:21 -08:00 |
|
|
Yu Chin Fabian LimandTyler Michael Smith
|
aff404571b
|
Add Bamba Model (#10909)
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2025-02-06 15:22:42 -08:00 |
|
|
Varun Sundar RabindranathandVarun Sundar Rabindranath
|
467a96a541
|
[V1] LoRA Support (#10957)
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
|
2025-02-06 09:32:51 -08:00 |
|
|
Dipika Sikka
|
7ca9934fe7
|
[Misc] Update w2 scale loading for GPTQMarlinMoE (#12757)
|
2025-02-06 01:02:14 -08:00 |
|