|
9446ea1680
|
[Bugfix][Spec Decode] Only create draft_id_to_target_id when draft vocab differs (#53458)
Signed-off-by: drslark <slarksblood@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: GLM-5.3 <glm@z.ai>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-15 14:59:02 +00:00 |
|
|
Abhinav VermaandClaude Opus 5
|
1e47ec00d2
|
[Bugfix][Gemma 4] Don't read fft_length when profiling unified audio (#56721)
Signed-off-by: Abhinav Verma <verma.abhinav275@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-09-15 14:57:32 +00:00 |
|
|
|
4bac767695
|
[Perf][Nemotron] Skip redundant latent-MoE all-reduce at TP>1 (~13% decode win) (#52301)
Signed-off-by: Junhao Shen <junshen@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-15 07:33:51 -07:00 |
|
|
aoshen02
|
eb61707851
|
[Bugfix] Fix stale HPC QK-norm weights after weight refit (#56706)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
|
2026-09-14 21:34:41 -07:00 |
|
|
|
9d14fd3095
|
[Kernel][MoE] DeepEP v2: async finalize to overlap shared experts with combine (#52781)
Signed-off-by: Mario <404mario@users.noreply.github.com>
Co-authored-by: Mario <404mario@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-15 00:04:22 -04:00 |
|
|
|
567f745c1b
|
[Bugfix] Export weight-cache IPC tensors separately for each client (#56472)
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-15 02:43:42 +00:00 |
|
|
JohnQinAMDandClaude Opus 5
|
ef5f7cd119
|
[ROCm][DSv4.1][Perf] Dequantize the MXFP8 weight once when dot_scaled cannot be used (#56560)
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 09:07:26 +08:00 |
|
|
Fangzhou AiandCursor
|
00972dfd72
|
[ROCm][DSV4.1][Perf] Fold the mHC post step into the delayed pre projection (#56513)
Signed-off-by: Fangzhou Ai <fangzhou@semianalysis.ai>
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-09-14 16:44:56 -05:00 |
|
|
Netanel Haber
|
d4ee7fe7a9
|
[Quantization] Select linear backends per quantization (#51204)
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
|
2026-09-14 16:24:18 -04:00 |
|
|
Nicolò LucchesiandJorge Gallego Feliciano
|
995e8581f4
|
[Model] Voxtral Realtime: add support for CUDAGraphMode.FULL_DECODE_ONLY (#51167)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Jorge Gallego Feliciano <jorge.gallego@mistral.ai>
|
2026-09-14 19:16:39 +00:00 |
|
|
Yongye Zhu
|
5372e72a98
|
[Perf][DSv4.1] Fold the mHC post block into the delayed pre projection (#56633)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-09-14 18:57:37 +00:00 |
|
|
|
03dc26e639
|
[Perf][Kernel][Quantization] Fuse ReLU2 with static FP8 activation quantization (#53793)
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: snordmann <snordmann@nvidia.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-14 10:26:15 -07:00 |
|
|
|
b3124a8237
|
[Model] Add support for Nanbeige4.2 (transformers backend) (#56071)
Signed-off-by: zqlcode <lizongqiang@kanzhun.com>
Signed-off-by: zql <37731799+zqlcode@users.noreply.github.com>
Co-authored-by: zqlcode <lizongqiang@kanzhun.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-09-14 15:06:01 +00:00 |
|
|
|
c676e4930b
|
[Spec Decode] Fix Qwen3 DSpark d2t requirement for padded-vocab drafts (#55133)
Signed-off-by: Orestis Zambounis <23146389+orestis-z@users.noreply.github.com>
Signed-off-by: Orestis Zambounis <orestis.zambounis@gmail.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-09-14 10:07:07 -04:00 |
|
|
linitra24
|
9d4d9aa5bc
|
[LoRA][Refactor] Unify multimodal LoRA token count hooks (#55071)
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
|
2026-09-14 13:19:15 +00:00 |
|
|
arkohutandHarry Mellor
|
6623fe5b4e
|
[Bugfix] MiniCPM-V 4.6: fix ViT self-attn qkv weight loading (#49417)
Signed-off-by: arkohut <39525455+arkohut@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-09-14 13:18:55 +00:00 |
|
|
Ronald
|
e0c04c7b4d
|
[Bugfix][Models] Fix OpenPangu sleep mode with static sinks (#53696)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
|
2026-09-14 21:05:47 +08:00 |
|
|
Matthias Gehre
|
dc2e8f1157
|
[ROCm][Perf] W4A16: keep skinny GEMM zero-points packed 4-bit (#54965)
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
|
2026-09-14 17:40:25 +08:00 |
|
|
|
238cb2b191
|
[Perf][GLM-5.3-Flash] Dense/masked-MHA sparse prefill for the NoPE (256, 0, 256) layout + skip the NoPE K concat (#55738)
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-14 09:36:39 +00:00 |
|
|
Li, Jiang
|
79f0be21ff
|
[Bugfix][CPU] Fix DeepSeek-R1 (FP8 MLA + MoE) correctness on CPU backend (#56773)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-09-14 17:30:28 +08:00 |
|
|
|
78e84261ab
|
[Performance] Add Triton kernel for Gemma3n sparse GELU (#48498)
Signed-off-by: BWAAEEEK <jooho414@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-09-14 07:42:15 +00:00 |
|
|
Isotr0py
|
dbf49dad11
|
[Fast Start] Use GPU uuid as socket folder identifier (#56669)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-14 06:28:13 +00:00 |
|
|
Bugen Zhao
|
cf1584f373
|
[Refactor] Normalize DeepSeek V4.1 model package naming (#56741)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-13 23:17:50 -07:00 |
|
|
siyuandIsotr0py
|
52dd0d7562
|
[Fast Start] Fast loader support nnode>1 (#55468)
Signed-off-by: siyu <mg21330037@smail.nju.edu.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
|
2026-09-14 03:05:54 +00:00 |
|
|
Woosuk Kwon
|
3b533197a9
|
[Perf] Parallelize mHC pre-norm JIT warmup (#56683)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-09-13 18:58:20 -07:00 |
|
|
Woosuk KwonandOpenAI Codex
|
71888f507a
|
[Perf] Avoid redundant conversions in FP8 dummy initialization (#56688)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-13 12:06:13 -07:00 |
|
|
Kevin H. Luu
|
dec0b5d63f
|
[Bugfix] Skip Triton autotune inspection without Triton (#56676)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
|
2026-09-13 17:05:29 +00:00 |
|
|
|
b7e0cdac5d
|
[Bugfix] Detect unloaded NVFP4 weight scales with a NaN sentinel (#52501)
Signed-off-by: pavelzak <pavel.zakharov@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-13 14:58:09 +00:00 |
|
|
Woosuk Kwon
|
e7a3963339
|
[Bugfix] Avoid repeated dummy initialization and random CPU Engram fills (#56682)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-09-13 02:23:20 -07:00 |
|
|
YleanandCodex
|
a987777755
|
[Bugfix][Gemma4] Keep image kwargs out of video preprocessing (#56652)
Signed-off-by: Ylean <61880605+Woolgathererer@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-13 09:07:12 +00:00 |
|
|
|
fa008bdccf
|
[Perf][Kernel] Integrate DeepSelect TopK for the DSA sparse indexer (#56464)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-09-13 00:44:37 -07:00 |
|
|
Isotr0py
|
533dbe424b
|
[Misc] Clean up fast loader daemon quant method verification (#55656)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-12 20:16:39 -07:00 |
|
|
Roberto L. CastroandCodex
|
e52be1a62d
|
[9/N][warmup][DSv4] Migrate MHC TileLang kernels (#50178)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-09-12 17:25:23 -07:00 |
|
|
|
aed894c190
|
[6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-12 17:24:24 -07:00 |
|
|
|
ebe1dec2da
|
[PCP][DCP] Enable PCP+DCP on sparse-MLA models (#56157)
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai>
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai>
Signed-off-by: PatrykSaffer <patryk.saffer@mistral.ai>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai>
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patryk Saffer <PatrykSaffer@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-12 23:24:44 +00:00 |
|
|
|
7ee8a6dd01
|
[watermarking] Dual-key gumbel-max watermarking for speculative decoding support (#56122)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
|
2026-09-12 18:42:01 +00:00 |
|
|
Ayushman Singhandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
7bcca16729
|
[Bugfix] Fix DeepGEMM FP8 warmup coverage (#56452)
Signed-off-by: Ayushman Singh <40520701+ayush1399@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-12 14:20:38 -04:00 |
|
|
Itay Alroy
|
658c8131c7
|
[Elastic EP] Reuse CUDA graphs across reconfiguration (#54985)
|
2026-09-12 10:26:39 -04:00 |
|
|
JartX
|
986e2f870a
|
[Refactor][ROCm] Migrate the RDNA3 W4A16 MoE to the oracle/experts pa… (#55522)
Signed-off-by: JartX <sagformas@epdcenter.es>
|
2026-09-12 21:20:26 +08:00 |
|
|
Taneem Ibrahim
|
a0914ab7d0
|
[Nano-Nemotron] Fix Nano-Nemotron precomputed multimodal embeddings (#56398)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-09-12 09:07:10 -04:00 |
|
|
|
d43bb2f37f
|
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-09-12 00:36:47 -04:00 |
|
|
Roberto L. CastroandCodex
|
120ec4ebd2
|
[5/N][warmup][DSv4] Migrate NVIDIA CuTeDSL attention kernels (#53566)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-09-12 00:05:59 -04:00 |
|
|
Fangzhou AiandFangzhou Ai
|
46d2b23ac5
|
[ROCm][DSV4.1][Perf] Use AITER mHC for the delayed pre block (#56503)
Signed-off-by: Fangzhou Ai <fangzhou@semianalysis.ai>
Co-authored-by: Fangzhou Ai <fangzhou@semianalysis.ai>
|
2026-09-12 00:35:48 +00:00 |
|
|
Andy Lo
|
dc07f1638f
|
[Bugfix][MLA] Read sparse model settings from text config (#56160)
Signed-off-by: Andy Lo <andy@mistral.ai>
|
2026-09-11 18:02:13 +00:00 |
|
|
|
6fe67cbbf3
|
[Spec][V2] Support MTP speculative decoding under pipeline parallelism (#46994)
Signed-off-by: Chris Eastwood <chris.eastwood@pwn4g3.dev>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-11 17:55:21 +00:00 |
|
|
Shanshan Shen
|
127143e27f
|
Revert "[Rocm][Kimi-k3] Fix pipeline_parallel support for the kimik3 DCP mode (#53664)" (#56429)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
|
2026-09-11 09:17:26 -07:00 |
|
|
Wentao Ye
|
5fe77aecfc
|
[Deprecation] Deprecate items scheduled for 0.29 (#55353)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-09-11 08:17:02 -07:00 |
|
|
Harry MellorandClaude Opus 5
|
c191787a68
|
[Bugfix] Align vLLM YaRN with Transformers and stop re-scaling max_model_len (#56446)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-11 13:01:38 +00:00 |
|
|
labAxiaoming
|
a5b714eee4
|
[Bugfix] Fix Qwen3-VL and Cosmos3-Edge text architectures for CPU and pipeline parallelism (#53699)
Signed-off-by: xiaoming <1259730330@qq.com>
|
2026-09-11 05:58:15 -07:00 |
|
|
Yuxuan Zhang
|
89dbb26445
|
[Bugfix] Fix GLM-OCR MTP position masking during CUDA graph capture (#56447)
Signed-off-by: zRzRzRzRzRzRzR <Yuxuan.Zhang2@liverpool.ac.uk>
|
2026-09-11 12:31:10 +00:00 |
|