 
|
c9bfa574ce
|
[Online quantization] Extend re-quantization API to MOE (showcase NVFP4 modelopt/CT/quark to any online MOE method) (#60276)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com>
|
2026-10-14 15:49:37 +08:00 |
|
 ChaunceyandCyrus Leung
|
e1726d9142
|
[Frontend] Add cache salt and tracing to openai decisions (#61550)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-14 00:48:44 -07:00 |
|
zofia
|
4dc5add6ea
|
[XPU] [Linear] [MXFP8] enable torch as mxfp8 linear backend (#51933)
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Signed-off-by: zofia <110436990+zufangzhu@users.noreply.github.com>
|
2026-10-14 00:48:38 -07:00 |
|
Ronald
|
ec8a372566
|
[ROCm][Bugfix] Pin AITER back to v0.1.24.post1 for DSv4.1 mono decode numerics (#61559)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
|
2026-10-14 00:19:38 -07:00 |
|
 
|
30e79c3a76
|
[Core] Update PyTorch to 2.14.0, torchvision to 0.29.0, triton to 3.8.0 (release channel) (#52183)
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com>
Signed-off-by: Andrey Talman <atalman@fb.com>
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 00:17:59 -07:00 |
|
 
|
ec317a4618
|
[CI] Keep vLLM log propagation on through the first LLM init in caplog_vllm (#61535)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
|
2026-10-14 15:10:35 +08:00 |
|
 Jiangyun ZhuandClaude Opus 5.5
|
badfcdbf91
|
[Perf][GDN] Internal prefill checkpoints for the FlashInfer GDN prefill backend (#60050)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 14:49:46 +08:00 |
|
 ![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
a2fb482732
|
[KVConnector][P2P] Register peers off the scheduler thread (#57139)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-10-14 08:21:57 +03:00 |
|
  
|
77154ca611
|
[Perf][Model][Qwen4Exp] QSA: decode four FP8 e4m3 bytes per register below sm_89 (#60992)
Signed-off-by: kimseunghyun <hellonypcteam@gmail.com>
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-10-14 13:06:16 +08:00 |
|
Akshan Krithick
|
02d69c663a
|
[Bugfix][Benchmark] Preserve SSE output across transport chunk boundaries (#51172)
Signed-off-by: Akshan Krithick <akshankrithick305@gmail.com>
|
2026-10-13 21:58:58 -07:00 |
|
 
|
a78d050eff
|
[Bugfix] Bump tokenspeed-mla to 2.0.2 to fix nanobind conflict with Triton 3.9 (#61125)
Signed-off-by: Warren Deng <warrdeng@meta.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-13 21:28:07 -07:00 |
|
  
|
3ccc0a90b6
|
[KV Connector] Support sleep mode with MooncakeConnector over RDMA (#59625)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Dakai An <77474977+andakai@users.noreply.github.com>
|
2026-10-14 04:18:47 +00:00 |
|
 Lanze LiuandNing Xie
|
f4b8d43a68
|
[Bugfix] Keep force-killing the process tree when a child exits mid-walk (#59984)
Signed-off-by: Lanze Liu <lanzetech@gmail.com>
Co-authored-by: Ning Xie <andy.xning@gmail.com>
|
2026-10-13 20:33:18 -07:00 |
|
Shanshan Shen
|
2521446ac8
|
[ROCm][Perf] Enqueue the default chain first in CSA/HCA at small batch sizes for DeepSeek-V4 (#61378)
Signed-off-by: shen-shanshan <467638484@qq.com>
|
2026-10-14 10:57:26 +08:00 |
|
 
|
2308b6f00b
|
[Perf][MoE] Cheaper triton_kernels 3.8 routing build in the OAI Triton MoE (#58897)
Signed-off-by: Zheng Gong <zgong@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
|
2026-10-14 10:48:05 +08:00 |
|
xaguilar-amd
|
a1f51a0226
|
[ROCm][MLA] Route gfx942 DCP causal multi-token verify to Triton split-KV decode (#58555)
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com>
|
2026-10-14 10:17:15 +08:00 |
|
 John CalderonandArtem Perevedentsev
|
449c05a0ca
|
[CI][Tests] Upgrade torch version for Rubin prerelease (#61468)
Signed-off-by: John Calderon <jcalderon@nvidia.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
|
2026-10-14 10:10:00 +08:00 |
|
 Kevin H. LuuandCodex
|
d5e62627a3
|
[CI] Shard chat APIs, model evals and model tests automatically (#61475)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-13 18:16:27 -07:00 |
|
 Stefano CastagnettaandKartikey Vyas
|
4e0681ed56
|
[Quantization] Support packed NVFP4 Qwen4Exp PLE embeddings (#56273)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Kartikey Vyas <kartikey@artificialanalysis.ai>
|
2026-10-14 08:24:55 +08:00 |
|
 Almog TavorandItay Alroy
|
87e7dce3a7
|
[MRV2] Keep warmup out of the EPLB load stats (#56716)
Signed-off-by: almogtavor <almogtavor@gmail.com>
Co-authored-by: Itay Alroy <75032521+itayalroy@users.noreply.github.com>
|
2026-10-14 00:00:48 +00:00 |
|
 Ryan RockandMicah Williamson
|
f43d7e1a0f
|
[CI][AMD] Gate Aiter MHA FA on dtype (#60679)
Signed-off-by: Ryan Rock <ryan.rock@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
|
2026-10-13 18:55:19 -05:00 |
|
 josh-schell-amdandMicah Williamson
|
7a9d3db896
|
[ROCm] Bump ROCm base images to AITER v1.25.0 (#61230)
Signed-off-by: josh schell <josh.schell@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
|
2026-10-13 18:55:09 -05:00 |
|
 
|
c923e7769b
|
[Feat] Add text-only OpenAI Decisions API MVP (#60465)
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-14 07:51:08 +08:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
ad8d39cbab
|
[Bugfix] Fix DSpark graph-padding handling in the DeepSeek-V4.1 MoE fused top-k (#59166)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-14 07:49:52 +08:00 |
|
 Giancarlo DelfinandWoosuk Kwon
|
791f92135a
|
[Attention] Set ALWAYS cudagraph support for FA4 (#60908)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-10-13 16:22:52 -07:00 |
|
 Kevin H. LuuandCodex
|
6a03954b45
|
[CI] Shard quantization, pooling, multimodal APIs, core kernels and PCP eval automatically (#61477)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-13 16:15:30 -07:00 |
|
 Kevin H. LuuandCodex
|
0da66a6e46
|
[CI] Shard batch invariance, Kimi prefix cache, EP eval and generate APIs automatically (#61474)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-13 16:14:18 -07:00 |
|
 
|
8082b62f32
|
[Mooncake] Allow Mamba cache mode none for capacity-only workers (#60712)
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-13 23:13:58 +00:00 |
|
 Tim002andThien Tran
|
96fc2da315
|
[Perf][Qwen4Exp] Add SM121 TP=2 merged QSA projection plans (#61437)
Signed-off-by: TimRen01 <HolaTim@hotmail.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-10-13 22:58:44 +00:00 |
|
 Nick HillandClaude Opus 5.5
|
78b32b3fdc
|
Revert "[Build] DeepGEMM migration to torch Stable ABI (#59482)" (#61510)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-13 15:33:50 -07:00 |
|
 Wenbin ChenandMisha Goin
|
6463d04601
|
Fix write-after-write hazard in xpu KV RoPE triton kernel (#46685)
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-13 22:17:14 +00:00 |
|
  
|
698e79f3bd
|
[Model] Kolibri 1 via the Transformers modeling backend (#60827)
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-10-13 22:17:52 +01:00 |
|
Anirudh Ravichandran
|
e90158d912
|
[Bugfix][MoE] Clamp BLOCK_SIZE_K to the FP8 quantization block in the batched Triton MoE (#60733)
|
2026-10-13 17:02:17 -04:00 |
|
 Matthew BonanniandClaude Opus 5.5
|
292da21f9d
|
[HiSparse] Read GPU-resident KV directly for mixed and BF16 prefills (#60417)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-13 16:40:20 -04:00 |
|
Wentao Ye
|
c19c263ac8
|
[GLM 5.3 Perf] Handle two elements at the same time for per token group quant, 2.8% E2E ITL improvement (#60927)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-10-13 13:06:41 -07:00 |
|
 Andreas KaratzasandClaude
|
67516b6c42
|
[Bugfix][ROCm][Kimi-K3] Stage fused KDA decode weights after loading (#61207)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-13 14:55:57 -05:00 |
|
djramic
|
93d0bd9d50
|
[ROCm][CI] Drop the -inf padding check from the AITER MQA logits test (#61420)
Signed-off-by: Djordje Ramic <djoramic@amd.com>
|
2026-10-13 15:49:30 -04:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
a31054f5ef
|
[NIXL] Bump CUDA 12 NIXL to 1.5.0 (#60428)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-13 12:47:41 -07:00 |
|
 
|
68fc395554
|
[Bugfix] Keep GateLinear batch invariant under VLLM_BATCH_INVARIANT (#59985)
Signed-off-by: yifanFengg <evanfengff@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-10-13 15:43:54 -04:00 |
|
 yifanFenggandClaude Fable 5.1
|
b33cb81237
|
[Bugfix] Keep compressed-tensors wNa16 linear layers batch invariant (#60524)
Signed-off-by: yifanFengg <evanfengff@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-10-13 15:43:40 -04:00 |
|
Wentao Ye
|
2f18d13cf0
|
[GLM5.3 Perf] Enable fp8 kv cache for GLM5.3 Flash by default, 9.8% E2E Throughput Improvement (#61228)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-10-13 15:29:17 -04:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)  
|
483e0c24c7
|
[KV Offload] Split replicated-layout pre-faulting across workers (#59545)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-13 19:18:56 +00:00 |
|
 
|
c068db0e1d
|
[Bugfix][Compile] Replace Transformers AOT caching with graph caching (#58918)
Signed-off-by: fangyao fan <2281693949@qq.com>
Signed-off-by: Taimys <ffy2281693949@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Taimys <ffy2281693949@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-10-13 19:17:29 +00:00 |
|
 Anish ShanbhagandOpenAI Codex
|
3f550129e8
|
[Perf][Attention][MiniMax-M3] Select CUTLASS MSA by EAGLE3 decode shape (#55838)
Signed-off-by: Anish Shanbhag <ashanbhag@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-10-13 15:05:28 -04:00 |
|
 GuanLuoandsfeng33
|
6d1daa9fe5
|
[Bugfix][Frontend] Expose completion response schemas in OpenAPI (#60926)
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
|
2026-10-13 18:54:06 +00:00 |
|
 
|
c5bda658d2
|
[Model][MoE] Load MegaMoE experts directly into the DeepGEMM layout (#60096)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Aaron Hao <hao-aaron@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Aaron Hao <hao-aaron@users.noreply.github.com>
|
2026-10-13 11:52:08 -07:00 |
|
 Matthew BonanniandClaude Opus 5.5
|
624a5ddafd
|
[Attention] Fetch MLA prefill workspace at use instead of init (#60654)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-13 17:25:03 +00:00 |
|
 
|
e83222396f
|
[ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches (#57199)
Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
|
2026-10-13 17:21:30 +00:00 |
|
 Roberto L. CastroandClaude Opus 5.5
|
7fad7db330
|
[Perf][Feat] Add generic cuteDSL LL FP32 router (GEMM) (#44343)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-13 12:38:46 -04:00 |
|
 
|
6849ca696a
|
[Quantization] Add ModelOpt IQ2_XS, IQ2_XXS, and Q8_0 support via b12x (#59604)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Luke Alonso <lalonso@gmail.com>
|
2026-10-13 09:06:33 -07:00 |
|