 SeongJun LeeandIsotr0py
|
699c892dd5
|
[Perf] Skip the M-RoPE warmup when every holder of the RoPE fuses it (#60246)
Signed-off-by: lesj0610 <lesj0610@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-10-10 06:47:39 +00:00 |
|
|
|
02aa10539f
|
[Bugfix][ModelLoader] Make enable_weights_track check quantized layers (#57108)
Signed-off-by: siliangchen-amd <SiLiang.Chen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
|
2026-10-10 10:44:06 +08:00 |
|
|
|
02e55d2af3
|
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-10-09 15:48:37 -04:00 |
|
|
Ning XieandSimon Mo
|
a0f28d73a8
|
[sharded state loader] support pp in sharded state loader (#52110)
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
|
2026-10-09 11:08:31 -07:00 |
|
|
Juan Pérez de Algabaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
3d52578e13
|
[Security] Do not retain Phi-4 MM audio attention matrices (#60614)
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-09 22:23:35 +08:00 |
|
|
Harry MellorandClaude Opus 5.5
|
2c99ee9333
|
[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-09 09:29:03 +00:00 |
|
|
|
9f3d70e081
|
[Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) (#55686)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-08 12:04:46 -04:00 |
|
|
Woosuk Kwon
|
194bd72be7
|
[Compile] Remove torch.compile-based sequence parallelism and AsyncTP (#60517)
|
2026-10-08 08:28:23 -07:00 |
|
|
Harry MellorandClaude Opus 5.5
|
21d9758459
|
Remove Python 3.10 support now that it is EOL (#60402)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-08 02:30:52 -07:00 |
|
|
Jiahan Chang (Cyrus)
|
816173ca9d
|
[Kernel] Remove WAR for LL CuTeDSL BF16 GEMM (#60434)
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
|
2026-10-08 05:38:07 +00:00 |
|
|
MattandClaude Opus 5.5
|
9dba339d07
|
[Bugfix][ROCm] Keep MLA derived weights in place on weight reload (#60389)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-07 20:38:06 -05:00 |
|
|
|
f38c6778ce
|
[quantization] Add per-token NVFP4 quantization MoE support for ReLU2 (#56740)
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-10-07 22:01:09 +00:00 |
|
|
|
e572e02d21
|
[UX] Skip checkpoint shards an MTP head does not load (#60152)
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 15:16:19 -04:00 |
|
|
|
fba462e008
|
[Bugfix]Fix FlashInfer warmup crash: autotune(tuning_buckets=...) must round M up, not down (Invalid MXFP8 split-K tactic with spec-decode) (#58165)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 14:01:21 -04:00 |
|
|
|
c83935b300
|
[Bugfix][Model Loader] Support reload_weights with runai_streamer load format (#58124)
Signed-off-by: Viktor Koukouliev <vkoukouliev@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-10-07 14:04:35 +00:00 |
|
|
|
2a54f6b625
|
[ROCm][Perf][GLM-5.3-Flash] Enable AITER fused shared experts on gfx950 (#59221)
Signed-off-by: Jin Tao <jin.tao@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
|
2026-10-07 20:53:40 +08:00 |
|
|
|
3403e0f176
|
[Bugfix] Persist FlashInfer autotune cache per rank to fix the rank-0-only cache-hit deadlock (#57635)
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-10-06 17:51:01 +00:00 |
|
|
Tianyu GuoandOpenAI Codex
|
8c417a2dae
|
[Bugfix][Model] Fix Qwen3-Omni interleaved M-RoPE boundary positions (#59842)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-10-06 03:35:53 +00:00 |
|
|
C
|
44198f577f
|
[Bugfix][Model] Fix M-RoPE offset double-count in Qwen3-Omni (#58890)
Signed-off-by: C1ves <3121985758@qq.com>
|
2026-10-03 09:00:51 +08:00 |
|
|
|
4c4003bb5f
|
[Bugfix] Bind routed-experts capture to the MoE layer, not the kernel (#59455)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-02 15:50:38 +00:00 |
|
|
Qiu ChunshuoandLucas Wilkinson
|
8486f1ebfe
|
[Perf][PCP] Shard decode requests across PCP ranks (#52162)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-10-01 23:50:13 -04:00 |
|
|
Wentao Ye
|
4d6874d137
|
[GLM 5.3 Perf] Skip qlnorm calculation for MHA, 4.4~7.7% E2E TTFT Improvement (#58845)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-10-01 15:13:57 +08:00 |
|
|
Andrii SkliarandAndrii Skliar
|
deb148e730
|
[Mamba] Add FlashInfer ReplaySSM support for MTP (#52928)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
|
2026-09-30 21:41:17 +00:00 |
|
|
VenkyandClaude Opus 5.5
|
42f0c17ea7
|
[Bugfix][PP][Spec Decode] MiniMax-M3 EAGLE3 aux-state relay at PP > 1 and per-stage FlashInfer autotune (#57197)
Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-01 01:14:24 +04:00 |
|
|
Harry MellorandClaude Opus 5.5
|
fc2c801ad9
|
Disable mypy arg-type and assignment checks in tests (#59428)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-30 15:18:50 +00:00 |
|
|
almayneandLi, Jiang <jiang1.li@intel.com>
|
93dd80f636
|
[CPU] Conv1d optimised kernel for aarch64 (#54093)
Signed-off-by: Anna Mayne <anna.mayne@arm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-09-30 20:23:42 +08:00 |
|
|
Jakub ZakrzewskiandCodex
|
5a4351ab09
|
[MM] Keep device input normalization fused when encoder compilation is enabled (#59195)
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-09-30 11:08:24 +00:00 |
|
|
Alexandra Sidorova
|
b81984879a
|
[K2 Horizon] fold partial-RoPE permutation into q/k (and norm) weights (#55335)
Signed-off-by: Alexandra Sidorova <asidorov@amd.com>
|
2026-09-30 16:03:37 +08:00 |
|
|
|
42d9f8ccdb
|
support rl feature : weight checker (#51350)
Signed-off-by: shiyuan680 <yangcheng104@huawei.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: shiyuan680 <yangcheng104@huawei.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: DeepSeek Harness <noreply@deepseek.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-09-30 03:13:43 +00:00 |
|
|
|
5dd693122c
|
[MyPy][2/N] Fix mypy errors in small tests/ dirs (#51043)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-29 21:27:45 +01:00 |
|
|
|
f5ed3d9c03
|
[Fast Start] Charge daemon-held weights against gpu_memory_utilization (#57298)
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Xun Sun <UNIDY2002@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
|
2026-09-29 17:58:35 +00:00 |
|
|
|
af5b4857e1
|
[Fast Start] Support PP (#55477)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
|
2026-09-29 09:37:43 +00:00 |
|
|
Xun SunandIsotr0py
|
e7900156e1
|
[Fast Start] Add /health endpoint for the weight cache daemon (#58552)
Signed-off-by: Xun Sun <UNIDY2002@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-27 17:48:54 +00:00 |
|
|
Isotr0py
|
24928a4d62
|
[UX][Frontend] Introduce vllm preload cli for fast restart (#56680)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-25 06:56:39 +00:00 |
|
|
|
8a269919f4
|
[MM] Add Triton kernel for mm_input_normal. (#56798)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-25 04:31:20 +00:00 |
|
|
wangxiyuan
|
44257c2e35
|
[Feature] Triton kernel dispatcher (#43048)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
|
2026-09-25 04:07:43 +00:00 |
|
|
Samuel Kimandmgoin
|
f55419c43c
|
[Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxfp8 shape limits (#54223)
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-09-25 02:54:11 +00:00 |
|
|
|
6936e77ba4
|
[Perf][Engram] Serialize offloaded lookups and pack host tables into huge pages (#56926)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-25 10:52:02 +08:00 |
|
|
Thien Tran
|
ade1056ee5
|
[Kimi-K3][Perf] Dispatch GEMM for vision patch embedder (#58527)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-09-24 18:30:11 -07:00 |
|
|
Misha GoinandCodex
|
71891bdb94
|
[Perf] Parallelize registered CUDA Triton kernel warmup at startup (#58582)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-24 19:52:00 -04:00 |
|
|
|
90a9515006
|
[Cleanup] Remove online quantization support in fp8.py in favor of online shorthands (#53585)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-24 12:48:14 -07:00 |
|
|
Xun Sun
|
e773247f8a
|
[Fast Start] Wait for weight cache daemon readiness (#58370)
|
2026-09-24 19:18:04 +00:00 |
|
|
Harry MellorandClaude Opus 5.5
|
d881812500
|
[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-09-24 14:14:10 +00:00 |
|
|
Thien TranandOpenAI Codex
|
00b7847c80
|
[Perf] Use Conv3dLayer for MiniMax M3 patch embedding (#58512)
Signed-off-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-24 00:41:36 -07:00 |
|
|
|
1291bdb85e
|
[EPD][Model Loader] Skip language-model checkpoint shards for --mm-encoder-only (#58086)
Signed-off-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
|
2026-09-24 00:41:15 -07:00 |
|
|
|
8b06a343ec
|
[Bugfix][CPU][MoE] Fix out-of-bounds write and segfault when router weights are fp32 (#56168)
Signed-off-by: Farzad Abdolhosseini <farzad@elastix.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-09-24 13:15:17 +08:00 |
|
|
Juntian LiuandClaude Opus 5.5
|
9ef37771be
|
[Bugfix][DSA] Bound DeepSelect sentinel columns in the sparse top-k remap (#58215)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-09-24 00:54:27 +00:00 |
|
|
00CC
|
eb2e91d3ba
|
[ROCm][Perf] Use wvSplitK for single-output GEMMs (#53283)
Signed-off-by: tangzzycc <3081129260@qq.com>
|
2026-09-23 06:38:17 +00:00 |
|
|
Tianyu GuoandCyrus Leung
|
3ee2906f47
|
[EPD] Support metadata-only audio inputs (#57887)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-09-23 09:03:50 +08:00 |
|
|
siyuandIsotr0py
|
c024cfdb02
|
[Fast Start] Support data parallelism in the weight cache daemon (#57386)
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-22 18:19:40 +00:00 |
|