|
03e8300d2c
|
[Bugfix] Enable inductor batch-invariant mode under VLLM_BATCH_INVARIANT (#61118)
Signed-off-by: Yifan Feng <evanfengff@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-17 14:53:06 -04:00 |
|
|
|
837e87e1c5
|
[Feature][LoRA][MoE] Expert LoRA for FlashInfer TRTLLM MXFP4 experts (#61979)
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-17 20:17:56 +08:00 |
|
|
Vadim GimpelsonandClaude
|
ec8b20fbf0
|
[Bugfix] Fix Gemma 4 encoder CUDA graph capture OOM at startup (#61365)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-17 11:04:49 +00:00 |
|
|
|
2067297b2f
|
[ROCm][Perf] Use AITER tuned configs to select a4w4 gemm kernel instead of env var (#61278)
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Signed-off-by: afriedri <afriedri@amd.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-10-17 04:00:28 -05:00 |
|
|
Wei Zhao
|
90d6d871d6
|
[Attention] Add CuTe DSL backend selection for MLA context prefill (#62069)
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
|
2026-10-17 01:21:55 -04:00 |
|
|
Jee Jee Li
|
39983a5e7a
|
[Kernel] Add FP32 router GEMM support for Step-5 Preview and Muse Spark Open (#62126)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-10-16 22:17:55 -07:00 |
|
|
|
7b6310c109
|
[Perf][Sampler] Speed up the small-batch split top-p pipeline (#61996)
Signed-off-by: Bowen Wang <bowen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-16 21:22:11 -07:00 |
|
|
|
71c20f73f2
|
[FlashInfer] Migrate autotune warmup to flashinfer.autotune_v2 managed cache (#55671)
Signed-off-by: Yang Xu <yanxu@nvidia.com>
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Signed-off-by: YangXu <yanxu@nvidia.com>
Co-authored-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-17 02:08:28 +00:00 |
|
|
Vadim GimpelsonandClaude
|
783380af39
|
[Bugfix] Fix NaN output of the Mamba2 chunk scan kernel on sm90 (#61718)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-16 17:24:51 -07:00 |
|
|
|
b544587408
|
[Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning (#53558)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-16 23:16:47 +00:00 |
|
|
|
28e92eeff2
|
[ROCm][Perf][Kimi-K3] Use AITER Gluon chunk_kda for KDA prefill (#59552)
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-16 22:58:56 +00:00 |
|
|
|
596e530675
|
[Kernel] Add router GEMM kernels for (5120, 256) and BF16 output to ll_bf16_gemm (#62042)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Lain <fusiyuan2000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-16 15:39:46 -07:00 |
|
|
|
68ebea789a
|
[Perf][MoE] Skip padded work in SM100 FP4 DeepGEMM experts (#59044)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-10-16 16:20:52 -04:00 |
|
|
Wentao Ye
|
8e25bbca19
|
[Refactor] Remove dead code multiple places (#61843)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-10-16 15:01:07 +00:00 |
|
|
Ayushman Singh
|
de7eeaa87b
|
[Perf] Share Triton JIT warmup compilation pool across registered kernels (#61422)
|
2026-10-16 08:59:42 -04:00 |
|
|
zofia
|
07f3857fb9
|
[XPU][Linear][MXFP4] enable torch as mxfp4 linear backend on xpu (#51965)
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Signed-off-by: zofia <110436990+zufangzhu@users.noreply.github.com>
|
2026-10-16 17:07:05 +08:00 |
|
|
Andreas KaratzasandClaude
|
b466fbacd2
|
[ROCm][Perf] Any-shape low-latency MoE router GEMM on MI355 (#61895)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-15 23:22:31 -05:00 |
|
|
Andreas KaratzasandClaude
|
29345f42a6
|
[ROCm][Qwen4Exp] Support HC and MoE sequence parallelism on ROCm (#61471)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-15 20:11:17 -05:00 |
|
|
Kevin H. LuuandClaude Opus 5.5
|
a956a8126e
|
[Bugfix][MoE] Keep the gate's hash_indices_table out of RoutedExperts parameters (#61930)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 18:00:17 -07:00 |
|
|
|
a8be1500ba
|
[Bugfix][MoE] Restore TP-sharded routed-expert capture for DP=1 (#61388)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: Ao Shen <281812371+aoshen02@users.noreply.github.com>
Signed-off-by: Ao Shen <aoshen@inferact.ai>
Co-authored-by: Ao Shen <281812371+aoshen02@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Ao Shen <aoshen@inferact.ai>
|
2026-10-16 00:21:24 +00:00 |
|
|
Jack HuandDouglas Lehr
|
ef38e8d006
|
[Model][Hardware][AMD][Kernel] Part 2/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_ATTN (#42755)
Signed-off-by: Jack Hu <Jack.Hu@amd.com>
Signed-off-by: Jack Hu <jingxujaster@hotmail.com>
Signed-off-by: Jack Hu <jack.hu@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
|
2026-10-15 17:10:14 -07:00 |
|
|
Stefano CastagnettaandKevin H. Luu
|
784d1fff0c
|
[MRV2][Spec Decode] Quantized draft lm_head for drafters that share the target head (#59973)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 22:06:40 +00:00 |
|
|
|
e711723352
|
[ROCm] gfx1250 optimizations and model enablement (#58800)
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: Juan Villamizar <jpvillam@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
|
2026-10-15 14:51:57 -07:00 |
|
|
|
f8460ef236
|
[Kernel][DSv4.1] Bump FlashMLA to the 2026.09.30 release and its interleaved V4.1 KV cache (#61371)
Signed-off-by: juntianl <juntianl@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-15 13:58:25 -07:00 |
|
|
Rohan PotdarandClaude Opus 5.5
|
105605b8e9
|
[ROCm] Support DeepSeek-V4 hash routing in AITER Triton W4A16 MoE (#60282)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 14:10:12 -05:00 |
|
|
Thien Tran
|
82b9ccaa79
|
[Perf][MoE] Skip the TRTLLM fp8-block activation-scale transpose copy (#61773)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-10-15 12:09:07 -07:00 |
|
|
|
4d58fa254a
|
[MoE] Unify DeepGEMM MegaMoE experts behind one feature-matrix layer; add NVFP4 (#61538)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 19:02:01 +00:00 |
|
|
Mikko TukiainenandCursor
|
47fd76e4ce
|
[ROCm][Perf] Route softmax MoE top-k, including fused shared-expert sigmoid, through AITER topk_gating (#56226)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-10-15 17:53:18 +00:00 |
|
|
|
4f49b55c1f
|
[MoE] Shape-based padding for FlashInfer TRT-LLM MXFP8 experts (gated and non-gated) (#61293)
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-10-15 13:44:11 -04:00 |
|
|
Matthias Gehre
|
011fa68ca9
|
[ROCm][Perf] Pad fp16/bf16 GEMM weight row stride for L2 cache (#55090)
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
|
2026-10-15 14:26:32 +00:00 |
|
|
littlecircle0730andShanshan Shen
|
71536bd5f8
|
[MM][CG] Support ViT full CUDA graph for DeepSeek-VL2 (#46005)
Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com>
Signed-off-by: littlecircle0730 <43994952+littlecircle0730@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
|
2026-10-15 21:38:52 +08:00 |
|
|
|
0951fc183c
|
[ROCm][DSv4.1] Disambiguate ROCm attention class names (#58689)
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 21:38:39 +08:00 |
|
|
Xunzhuo
|
766c837bd5
|
[Model] Support every Decision 3.0 size (d3-flash, d3-mini, d3-nano, d3-lite, d3-edge) (#61778)
Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com>
|
2026-10-15 13:29:24 +00:00 |
|
|
TimofeyandCodex
|
716c5225ae
|
[Model] Add AliceAI support (#57963)
Signed-off-by: timka <timka.byzov@yandex.ru>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-15 10:15:03 +00:00 |
|
|
|
b965ee564d
|
[Chore] Use upstream Transformers DeepseekV4Config (#61392)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 01:08:08 -07:00 |
|
|
Vivaan SharmaandKevin H. Luu
|
96b5b0ee4b
|
[Model] Enable torch.compile and CUDA graphs for EmbeddingGemma2 (#61572)
Signed-off-by: Vivaan Sharma <99265792+Vivaansharma2003@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 12:32:39 +08:00 |
|
|
Haiyu ZuoandKevin H. Luu
|
ff5a91a456
|
[Feature][Model Loader] Apply model-owned MTP shard selection to RunAI ModelStreamer (#60759)
Signed-off-by: 浩韵 <zuohaiyu.zhy@alibaba-inc.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 12:23:11 +08:00 |
|
|
Chaitanya Sri Krishna LollaandCursor
|
833665cded
|
[ROCm][DI] Enable MoRI FP4 dispatch for mxfp4 w4a4 moe (#59596)
Signed-off-by: lcskrishna <lollachaitanya@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-10-14 23:20:14 -05:00 |
|
|
Shanshan Shen
|
f46021a7fa
|
[ROCm][Perf] Enable DPA+ETP prefill compute/comm dual-batch-overlap for DeepSeek-V4 (#57773)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
Signed-off-by: shen-shanshan <467638484@qq.com>
|
2026-10-15 11:16:48 +08:00 |
|
|
KaiandKunshang Ji
|
de2b869ee9
|
[Bugfix][Kernel] Register int8 scaled_mm as a custom op (#57331)
Signed-off-by: Kai <limkc.reply@gmail.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-10-15 09:27:49 +08:00 |
|
|
Woosuk KwonandClaude Opus 5.5
|
8530aed2b7
|
[Spec Decode] Remove Medusa and MLP speculator support (#61701)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 17:29:23 -07:00 |
|
|
|
984c242ede
|
[Quantization] Support compressed-tensors MXFP4 MoE with FP8 activations via Mxfp4MoEMethod (#59815)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-14 13:33:35 -07:00 |
|
|
|
fe98d90a9d
|
[Dependency] Update Humming and remove NVFP4 scale workaround (#61556)
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-14 13:32:36 -07:00 |
|
|
Juntian Liu
|
7c3f7e5500
|
[Perf][DSv4.1] Length-aware candidate block selection in the decode indexer (#61039)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
|
2026-10-14 20:15:58 +00:00 |
|
|
simon-veitner-redhatandClaude
|
ae94e9c9e8
|
[Bugfix] Don't crash DeepGEMM MegaMoE warmup when get_bf16_mega_gate_config is missing (#61657)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-14 18:58:35 +00:00 |
|
|
|
d1d8e6c096
|
[Perf][Nemotron-Parse] Do image rescale/normalize on the device (#60500)
Signed-off-by: DiegoCao <DiegoCao@users.noreply.github.com>
Co-authored-by: DiegoCao <DiegoCao@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 18:33:22 +00:00 |
|
|
TaimysandOpenAI Codex
|
2c74fd1873
|
[Model Runner V2] Enable batch-sharded sampling for GPT-OSS (#61257)
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-10-14 10:11:11 -07:00 |
|
|
Xunzhuo
|
89cab55086
|
[Model] Support Decision 3.0 decision models (vllm-sr/d3) (#61565)
Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com>
|
2026-10-14 17:10:55 +00:00 |
|
|
zexplorerhjandCodex
|
3895a8babe
|
[Bugfix] Fix TP loading of small block-FP8 MLA gates in BailingMoeV3 (#61642)
Signed-off-by: zexplorerhj <zhjoneson@163.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-14 10:09:23 -07:00 |
|
|
|
13dbf96093
|
[Model Loader] Apply EP weight filtering in InstantTensor (#61167)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-14 09:10:10 -07:00 |
|