Commit Graph
23022 Commits
Author SHA1 Message Date
Ronald ec8a372566 [ROCm][Bugfix] Pin AITER back to v0.1.24.post1 for DSv4.1 mono decode numerics (#61559)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
2026-10-14 00:19:38 -07:00
30e79c3a76 [Core] Update PyTorch to 2.14.0, torchvision to 0.29.0, triton to 3.8.0 (release channel) (#52183)
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com>
Signed-off-by: Andrey Talman <atalman@fb.com>
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:17:59 -07:00
ec317a4618 [CI] Keep vLLM log propagation on through the first LLM init in caplog_vllm (#61535)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
2026-10-14 15:10:35 +08:00
Jiangyun ZhuandClaude Opus 5.5 badfcdbf91 [Perf][GDN] Internal prefill checkpoints for the FlashInfer GDN prefill backend (#60050)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:49:46 +08:00
liranschourItay Etelismergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Or Ozeri
a2fb482732 [KVConnector][P2P] Register peers off the scheduler thread (#57139)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
2026-10-14 08:21:57 +03:00
77154ca611 [Perf][Model][Qwen4Exp] QSA: decode four FP8 e4m3 bytes per register below sm_89 (#60992)
Signed-off-by: kimseunghyun <hellonypcteam@gmail.com>
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-10-14 13:06:16 +08:00
Akshan Krithick 02d69c663a [Bugfix][Benchmark] Preserve SSE output across transport chunk boundaries (#51172)
Signed-off-by: Akshan Krithick <akshankrithick305@gmail.com>
2026-10-13 21:58:58 -07:00
a78d050eff [Bugfix] Bump tokenspeed-mla to 2.0.2 to fix nanobind conflict with Triton 3.9 (#61125)
Signed-off-by: Warren Deng <warrdeng@meta.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-13 21:28:07 -07:00
3ccc0a90b6 [KV Connector] Support sleep mode with MooncakeConnector over RDMA (#59625)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Dakai An <77474977+andakai@users.noreply.github.com>
2026-10-14 04:18:47 +00:00
Lanze LiuandNing Xie f4b8d43a68 [Bugfix] Keep force-killing the process tree when a child exits mid-walk (#59984)
Signed-off-by: Lanze Liu <lanzetech@gmail.com>
Co-authored-by: Ning Xie <andy.xning@gmail.com>
2026-10-13 20:33:18 -07:00
Shanshan Shen 2521446ac8 [ROCm][Perf] Enqueue the default chain first in CSA/HCA at small batch sizes for DeepSeek-V4 (#61378)
Signed-off-by: shen-shanshan <467638484@qq.com>
2026-10-14 10:57:26 +08:00
2308b6f00b [Perf][MoE] Cheaper triton_kernels 3.8 routing build in the OAI Triton MoE (#58897)
Signed-off-by: Zheng Gong <zgong@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
2026-10-14 10:48:05 +08:00
xaguilar-amd a1f51a0226 [ROCm][MLA] Route gfx942 DCP causal multi-token verify to Triton split-KV decode (#58555)
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com>
2026-10-14 10:17:15 +08:00
John CalderonandArtem Perevedentsev 449c05a0ca [CI][Tests] Upgrade torch version for Rubin prerelease (#61468)
Signed-off-by: John Calderon <jcalderon@nvidia.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-10-14 10:10:00 +08:00
Kevin H. LuuandCodex d5e62627a3 [CI] Shard chat APIs, model evals and model tests automatically (#61475)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-13 18:16:27 -07:00
Stefano CastagnettaandKartikey Vyas 4e0681ed56 [Quantization] Support packed NVFP4 Qwen4Exp PLE embeddings (#56273)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Kartikey Vyas <kartikey@artificialanalysis.ai>
2026-10-14 08:24:55 +08:00
Almog TavorandItay Alroy 87e7dce3a7 [MRV2] Keep warmup out of the EPLB load stats (#56716)
Signed-off-by: almogtavor <almogtavor@gmail.com>
Co-authored-by: Itay Alroy <75032521+itayalroy@users.noreply.github.com>
2026-10-14 00:00:48 +00:00
Ryan RockandMicah Williamson f43d7e1a0f [CI][AMD] Gate Aiter MHA FA on dtype (#60679)
Signed-off-by: Ryan Rock <ryan.rock@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
2026-10-13 18:55:19 -05:00
josh-schell-amdandMicah Williamson 7a9d3db896 [ROCm] Bump ROCm base images to AITER v1.25.0 (#61230)
Signed-off-by: josh schell <josh.schell@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
2026-10-13 18:55:09 -05:00
c923e7769b [Feat] Add text-only OpenAI Decisions API MVP (#60465)
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
2026-10-14 07:51:08 +08:00
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> ad8d39cbab [Bugfix] Fix DSpark graph-padding handling in the DeepSeek-V4.1 MoE fused top-k (#59166)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 07:49:52 +08:00
Giancarlo DelfinandWoosuk Kwon 791f92135a [Attention] Set ALWAYS cudagraph support for FA4 (#60908)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
2026-10-13 16:22:52 -07:00
Kevin H. LuuandCodex 6a03954b45 [CI] Shard quantization, pooling, multimodal APIs, core kernels and PCP eval automatically (#61477)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-13 16:15:30 -07:00
Kevin H. LuuandCodex 0da66a6e46 [CI] Shard batch invariance, Kimi prefix cache, EP eval and generate APIs automatically (#61474)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-13 16:14:18 -07:00
8082b62f32 [Mooncake] Allow Mamba cache mode none for capacity-only workers (#60712)
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:13:58 +00:00
Tim002andThien Tran 96fc2da315 [Perf][Qwen4Exp] Add SM121 TP=2 merged QSA projection plans (#61437)
Signed-off-by: TimRen01 <HolaTim@hotmail.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-10-13 22:58:44 +00:00
Nick HillandClaude Opus 5.5 78b32b3fdc Revert "[Build] DeepGEMM migration to torch Stable ABI (#59482)" (#61510)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 15:33:50 -07:00
Wenbin ChenandMisha Goin 6463d04601 Fix write-after-write hazard in xpu KV RoPE triton kernel (#46685)
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
2026-10-13 22:17:14 +00:00
698e79f3bd [Model] Kolibri 1 via the Transformers modeling backend (#60827)
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-10-13 22:17:52 +01:00
Anirudh Ravichandran e90158d912 [Bugfix][MoE] Clamp BLOCK_SIZE_K to the FP8 quantization block in the batched Triton MoE (#60733) 2026-10-13 17:02:17 -04:00
Matthew BonanniandClaude Opus 5.5 292da21f9d [HiSparse] Read GPU-resident KV directly for mixed and BF16 prefills (#60417)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 16:40:20 -04:00
Wentao Ye c19c263ac8 [GLM 5.3 Perf] Handle two elements at the same time for per token group quant, 2.8% E2E ITL improvement (#60927)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-10-13 13:06:41 -07:00
Andreas KaratzasandClaude 67516b6c42 [Bugfix][ROCm][Kimi-K3] Stage fused KDA decode weights after loading (#61207)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-10-13 14:55:57 -05:00
djramic 93d0bd9d50 [ROCm][CI] Drop the -inf padding check from the AITER MQA logits test (#61420)
Signed-off-by: Djordje Ramic <djoramic@amd.com>
2026-10-13 15:49:30 -04:00
Itay EtelisRobert Shawmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a31054f5ef [NIXL] Bump CUDA 12 NIXL to 1.5.0 (#60428)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 12:47:41 -07:00
68fc395554 [Bugfix] Keep GateLinear batch invariant under VLLM_BATCH_INVARIANT (#59985)
Signed-off-by: yifanFengg <evanfengff@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
2026-10-13 15:43:54 -04:00
yifanFenggandClaude Fable 5.1 b33cb81237 [Bugfix] Keep compressed-tensors wNa16 linear layers batch invariant (#60524)
Signed-off-by: yifanFengg <evanfengff@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-13 15:43:40 -04:00
Wentao Ye 2f18d13cf0 [GLM5.3 Perf] Enable fp8 kv cache for GLM5.3 Flash by default, 9.8% E2E Throughput Improvement (#61228)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-10-13 15:29:17 -04:00
Itay EtelisRobert ShawCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
483e0c24c7 [KV Offload] Split replicated-layout pre-faulting across workers (#59545)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 19:18:56 +00:00
c068db0e1d [Bugfix][Compile] Replace Transformers AOT caching with graph caching (#58918)
Signed-off-by: fangyao fan <2281693949@qq.com>
Signed-off-by: Taimys <ffy2281693949@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Taimys <ffy2281693949@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-10-13 19:17:29 +00:00
Anish ShanbhagandOpenAI Codex 3f550129e8 [Perf][Attention][MiniMax-M3] Select CUTLASS MSA by EAGLE3 decode shape (#55838)
Signed-off-by: Anish Shanbhag <ashanbhag@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-13 15:05:28 -04:00
GuanLuoandsfeng33 6d1daa9fe5 [Bugfix][Frontend] Expose completion response schemas in OpenAPI (#60926)
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
2026-10-13 18:54:06 +00:00
c5bda658d2 [Model][MoE] Load MegaMoE experts directly into the DeepGEMM layout (#60096)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Aaron Hao <hao-aaron@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Aaron Hao <hao-aaron@users.noreply.github.com>
2026-10-13 11:52:08 -07:00
Matthew BonanniandClaude Opus 5.5 624a5ddafd [Attention] Fetch MLA prefill workspace at use instead of init (#60654)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 17:25:03 +00:00
e83222396f [ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches (#57199)
Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
2026-10-13 17:21:30 +00:00
Roberto L. CastroandClaude Opus 5.5 7fad7db330 [Perf][Feat] Add generic cuteDSL LL FP32 router (GEMM) (#44343)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 12:38:46 -04:00
6849ca696a [Quantization] Add ModelOpt IQ2_XS, IQ2_XXS, and Q8_0 support via b12x (#59604)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Luke Alonso <lalonso@gmail.com>
2026-10-13 09:06:33 -07:00
Woosuk KwonandClaude Opus 5.5 ac71b17517 [Model Loader] Keep norm layers neutral under dummy weight init (#61323)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 08:54:24 -07:00
Taksh Kothari 12b005eefa [Bugfix] Don't split a multimodal item that a prefill chunk starts exactly on (#52412)
Signed-off-by: Taksh <takshkothari09@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-10-13 15:32:38 +00:00
Elvir Crnčević b9b401b2ef [Kernel] Fuse DeepEP-v2 decode globalize+align into one launch (#53911)
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
2026-10-13 11:10:38 -04:00