This website requires JavaScript.
30e79c3a76
[Core] Update PyTorch to 2.14.0, torchvision to 0.29.0, triton to 3.8.0 (release channel) (#52183 )
2026-10-14 03:17:59 -04:00
ec317a4618
[CI] Keep vLLM log propagation on through the first LLM init in caplog_vllm (#61535 )
2026-10-14 00:10:35 -07:00
badfcdbf91
[Perf][GDN] Internal prefill checkpoints for the FlashInfer GDN prefill backend (#60050 )
Jiangyun Zhu and Claude Opus 5.5
2026-10-14 14:49:46 +08:00
a2fb482732
[KVConnector][P2P] Register peers off the scheduler thread (#57139 )
2026-10-14 08:21:57 +03:00
77154ca611
[Perf][Model][Qwen4Exp] QSA: decode four FP8 e4m3 bytes per register below sm_89 (#60992 )
2026-10-14 13:06:16 +08:00
02d69c663a
[Bugfix][Benchmark] Preserve SSE output across transport chunk boundaries (#51172 )
Akshan Krithick
2026-10-13 21:58:58 -07:00
a78d050eff
[Bugfix] Bump tokenspeed-mla to 2.0.2 to fix nanobind conflict with Triton 3.9 (#61125 )
2026-10-13 21:28:07 -07:00
3ccc0a90b6
[KV Connector] Support sleep mode with MooncakeConnector over RDMA (#59625 )
2026-10-14 12:18:47 +08:00
f4b8d43a68
[Bugfix] Keep force-killing the process tree when a child exits mid-walk (#59984 )
Lanze Liu and Ning Xie
2026-10-13 20:33:18 -07:00
2521446ac8
[ROCm][Perf] Enqueue the default chain first in CSA/HCA at small batch sizes for DeepSeek-V4 (#61378 )
Shanshan Shen
2026-10-14 10:57:26 +08:00
2308b6f00b
[Perf][MoE] Cheaper triton_kernels 3.8 routing build in the OAI Triton MoE (#58897 )
2026-10-14 10:48:05 +08:00
a1f51a0226
[ROCm][MLA] Route gfx942 DCP causal multi-token verify to Triton split-KV decode (#58555 )
xaguilar-amd
2026-10-14 04:17:15 +02:00
449c05a0ca
[CI][Tests] Upgrade torch version for Rubin prerelease (#61468 )
John Calderon and Artem Perevedentsev
2026-10-13 22:10:00 -04:00
d5e62627a3
[CI] Shard chat APIs, model evals and model tests automatically (#61475 )
Kevin H. Luu and Codex
2026-10-13 18:16:27 -07:00
4e0681ed56
[Quantization] Support packed NVFP4 Qwen4Exp PLE embeddings (#56273 )
Stefano Castagnetta and Kartikey Vyas
2026-10-13 17:24:55 -07:00
87e7dce3a7
[MRV2] Keep warmup out of the EPLB load stats (#56716 )
Almog Tavor and Itay Alroy
2026-10-14 03:00:48 +03:00
f43d7e1a0f
[CI][AMD] Gate Aiter MHA FA on dtype (#60679 )
Ryan Rock and Micah Williamson
2026-10-13 18:55:19 -05:00
7a9d3db896
[ROCm] Bump ROCm base images to AITER v1.25.0 (#61230 )
josh-schell-amd and Micah Williamson
2026-10-13 18:55:09 -05:00
c923e7769b
[Feat] Add text-only OpenAI Decisions API MVP (#60465 )
2026-10-13 19:51:08 -04:00
ad8d39cbab
[Bugfix] Fix DSpark graph-padding handling in the DeepSeek-V4.1 MoE fused top-k (#59166 )
Chauncey and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 07:49:52 +08:00
791f92135a
[Attention] Set ALWAYS cudagraph support for FA4 (#60908 )
Giancarlo Delfin and Woosuk Kwon
2026-10-13 16:22:52 -07:00
6a03954b45
[CI] Shard quantization, pooling, multimodal APIs, core kernels and PCP eval automatically (#61477 )
Kevin H. Luu and Codex
2026-10-13 16:15:30 -07:00
0da66a6e46
[CI] Shard batch invariance, Kimi prefix cache, EP eval and generate APIs automatically (#61474 )
Kevin H. Luu and Codex
2026-10-13 16:14:18 -07:00
8082b62f32
[Mooncake] Allow Mamba cache mode none for capacity-only workers (#60712 )
2026-10-13 19:13:58 -04:00
96fc2da315
[Perf][Qwen4Exp] Add SM121 TP=2 merged QSA projection plans (#61437 )
Tim002 and Thien Tran
2026-10-14 06:58:44 +08:00
78b32b3fdc
Revert "[Build] DeepGEMM migration to torch Stable ABI (#59482 )" (#61510 )
Nick Hill and Claude Opus 5.5
2026-10-13 15:33:50 -07:00
6463d04601
Fix write-after-write hazard in xpu KV RoPE triton kernel (#46685 )
Wenbin Chen and Misha Goin
2026-10-14 06:17:14 +08:00
698e79f3bd
[Model] Kolibri 1 via the Transformers modeling backend (#60827 )
2026-10-13 23:17:52 +02:00
e90158d912
[Bugfix][MoE] Clamp BLOCK_SIZE_K to the FP8 quantization block in the batched Triton MoE (#60733 )
Anirudh Ravichandran
2026-10-13 14:02:17 -07:00
292da21f9d
[HiSparse] Read GPU-resident KV directly for mixed and BF16 prefills (#60417 )
Matthew Bonanni and Claude Opus 5.5
2026-10-13 16:40:20 -04:00
c19c263ac8
[GLM 5.3 Perf] Handle two elements at the same time for per token group quant, 2.8% E2E ITL improvement (#60927 )
Wentao Ye
2026-10-13 16:06:41 -04:00
67516b6c42
[Bugfix][ROCm][Kimi-K3] Stage fused KDA decode weights after loading (#61207 )
Andreas Karatzas and Claude
2026-10-13 14:55:57 -05:00
93d0bd9d50
[ROCm][CI] Drop the -inf padding check from the AITER MQA logits test (#61420 )
djramic
2026-10-13 21:49:30 +02:00
a31054f5ef
[NIXL] Bump CUDA 12 NIXL to 1.5.0 (#60428 )
2026-10-13 22:47:41 +03:00
68fc395554
[Bugfix] Keep GateLinear batch invariant under VLLM_BATCH_INVARIANT (#59985 )
2026-10-13 12:43:54 -07:00
b33cb81237
[Bugfix] Keep compressed-tensors wNa16 linear layers batch invariant (#60524 )
yifanFengg and Claude Fable 5.1
2026-10-13 12:43:40 -07:00
2f18d13cf0
[GLM5.3 Perf] Enable fp8 kv cache for GLM5.3 Flash by default, 9.8% E2E Throughput Improvement (#61228 )
Wentao Ye
2026-10-13 15:29:17 -04:00
483e0c24c7
[KV Offload] Split replicated-layout pre-faulting across workers (#59545 )
2026-10-13 22:18:56 +03:00
c068db0e1d
[Bugfix][Compile] Replace Transformers AOT caching with graph caching (#58918 )
2026-10-14 03:17:29 +08:00
3f550129e8
[Perf][Attention][MiniMax-M3] Select CUTLASS MSA by EAGLE3 decode shape (#55838 )
Anish Shanbhag and OpenAI Codex
2026-10-13 12:05:28 -07:00
6d1daa9fe5
[Bugfix][Frontend] Expose completion response schemas in OpenAPI (#60926 )
GuanLuo and sfeng33
2026-10-13 11:54:06 -07:00
c5bda658d2
[Model][MoE] Load MegaMoE experts directly into the DeepGEMM layout (#60096 )
2026-10-14 02:52:08 +08:00
624a5ddafd
[Attention] Fetch MLA prefill workspace at use instead of init (#60654 )
Matthew Bonanni and Claude Opus 5.5
2026-10-13 13:25:03 -04:00
e83222396f
[ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches (#57199 )
2026-10-13 19:21:30 +02:00
7fad7db330
[Perf][Feat] Add generic cuteDSL LL FP32 router (GEMM) (#44343 )
Roberto L. Castro and Claude Opus 5.5
2026-10-13 18:38:46 +02:00
6849ca696a
[Quantization] Add ModelOpt IQ2_XS, IQ2_XXS, and Q8_0 support via b12x (#59604 )
2026-10-13 18:06:33 +02:00
ac71b17517
[Model Loader] Keep norm layers neutral under dummy weight init (#61323 )
Woosuk Kwon and Claude Opus 5.5
2026-10-13 08:54:24 -07:00
12b005eefa
[Bugfix] Don't split a multimodal item that a prefill chunk starts exactly on (#52412 )
Taksh Kothari
2026-10-13 21:02:38 +05:30
b9b401b2ef
[Kernel] Fuse DeepEP-v2 decode globalize+align into one launch (#53911 )
Elvir Crnčević
2026-10-13 17:10:38 +02:00
669f85e686
[Misc] Break chat_utils into more maintainable modules. (#60972 )
wang.yuqi
2026-10-13 23:08:07 +08:00
f395a6b1ad
[ROCM][Bugfix] Deferring to TRITON_ATTN backend for Prefix lm models instead of ROCM_ATTN and fixed minor bug in Gemma3 Multimodal code (#46754 )
Rishabh Agarwal
2026-10-13 11:03:48 -04:00
f5fefbffe5
[Build] DeepGEMM migration to torch Stable ABI (#59482 )
Chris Leonard and Shengqi Chen
2026-10-13 11:01:44 -04:00
61d9c777bf
[CI] Run the Clef pooler test on GPU (#61409 )
Jiangyun Zhu and Claude Opus 5.5
2026-10-13 22:42:53 +08:00
de8291e24f
[Core] Add lifecycle hooks for worker extensions (#59463 )
2026-10-13 21:51:05 +08:00
0fae39a014
[Perf] per_token_group_quant_8bit: multiply by reciprocal instead of dividing per element (#61343 )
Thien Tran
2026-10-13 21:37:10 +08:00
31e90ac370
[Attention] Enable FA4 sparse MQA decode on Blackwell (#55866 )
2026-10-13 15:31:37 +02:00
0838cd6593
[Bugfix] Load and apply Nemotron Super final vision LayerNorm (#61401 )
Nathan Sala and Shaun Kotek - Nvidia
2026-10-13 15:42:29 +03:00
7342dc0317
[MoRIIO] Require pending remote prefill for READ cache hits (#60638 )
Hexiang Wang
2026-10-13 20:19:53 +08:00
253e02b95e
[Perf] Warm up DeepGEMM MegaMoE kernels before serving (#60468 )
2026-10-13 14:17:48 +02:00
d9a543c500
[Bugfix][Frontend] Forward --enable-log-outputs to the Cohere Chat v2 (#61089 )
Aditya Ganesh Kumar
2026-10-13 17:31:59 +05:30
818f70c295
[Bugfix] Mask special added tokens in xgrammar JSON, regex and EBNF grammars (#60749 )
2026-10-13 15:38:23 +04:00
cd7517fb17
[Bugfix] Fix ReplaySSM ring shape assertion when TP does not divide Mamba2 n_groups (#60704 )
Vadim Gimpelson and Claude
2026-10-13 15:31:53 +04:00
e6e84f350c
[Feature][Pooling] Add offline fixed-length late chunking (#60164 )
Linze Shi
2026-10-13 04:16:21 -07:00
426f3bd36b
[Bugfix][Model][Rust Frontend] Qwen3_ASR: Handle ragged Qwen audio batches from Rust frontend (#60979 )
noone-zhhh and Isotr0py
2026-10-13 18:49:39 +08:00
54db759eea
[Bugfix][Structured Output] Reject structural_tag and unhandled specs in backend validation (#58862 )
samayp01 and Artem Perevedentsev
2026-10-13 06:27:34 -04:00
7173c64e2e
[Security] Bound decoded and downloaded media per request (#61226 )
Juan Pérez de Algaba and OpenAI Codex
2026-10-13 12:17:14 +02:00
877f6df889
[Bugfix][DSv4] Don't bake the C128 compressor skip into CUDA graphs (#59709 )
JackRao123 and Claude Opus 5.5
2026-10-13 03:15:11 -07:00
c7a305ec10
[Bugfix][Mamba] Store conv_kernel - 1 inputs in Mamba2 prefill checkpoints (#60797 )
2026-10-13 03:09:17 -07:00
206e45d470
[CI] Shard V1 Attention automatically instead of by fixed pytest shards (#61200 )
Kevin H. Luu and Claude Opus 5.5
2026-10-13 02:24:06 -07:00
656d0d4506
[Perf] Tune Triton GDN decode tiles on SM100-family GPUs (#61355 )
Thien Tran and OpenAI Codex
2026-10-13 17:01:12 +08:00
183625d4f9
[Bugfix] Fall back to auto for unquantized MoE layers when --moe-backend is marlin (#60219 )
Vadim Gimpelson and Claude
2026-10-13 12:48:13 +04:00
6f11c220b5
[Transformers backend] Fuse MoE routers that score after the top-k (#57410 )
Harry Mellor and Claude Opus 5
2026-10-13 09:41:23 +01:00
24aa1191f4
[ROCm][Bugfix] Run the AITER AR+RMSNorm fallback test on the platform… (#61350 )
stefankoncarevic
2026-10-13 10:35:57 +02:00
ff305934f1
[Tests][Mamba] Wait for accepted counts in fake MTP proposer (#61342 )
Jiangyun Zhu
2026-10-13 16:15:47 +08:00
3ad096faf8
[Bugfix][EC Connector] Don't create hash directories on cache-miss lookups in ECExampleConnector (#60826 )
zhouyou
2026-10-13 15:59:48 +08:00
2cc2e68d1c
[Core][Mamba] Remove leftovers of mamba_cache_mode="all" (#60043 )
2026-10-13 15:58:59 +08:00
bc3ddb198a
[Model][Qwen4Exp] FP8 QSA main KV cache on Ampere (sm_80, sm_86) (#60602 )
2026-10-13 15:30:19 +08:00
4e4eab73ed
[CI] Shard Model Executor and L4 Pipeline + Context Parallelism automatically (#61202 )
Kevin H. Luu and Claude Opus 5.5
2026-10-12 23:57:32 -07:00
e2ad0ce5f5
[Bugfix][LoRA] Slice expert LoRA by the unpadded per-rank size under TP (#61235 )
2026-10-13 14:25:16 +08:00
a6009ea85b
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer (#61149 )
Yongye Zhu and Claude Opus 5.5
2026-10-12 23:24:43 -07:00
86b1c2ef15
[Attention][DCP] Support a2a DCP comm backend for FlashMLA sparse (#61285 )
Nicolò Lucchesi and Claude Opus 5.5
2026-10-13 07:51:22 +02:00
380638afbb
[Model] Add Clef-Flash support (#61308 )
2026-10-13 13:46:37 +08:00
0d2689e0df
[ROCm][Perf] Skip redundant sparse index remap on non-indexer layers (#51309 )
2026-10-13 08:25:56 +03:00
2ce8012fa0
[ROCm][Perf] Skip cleaning sparse prefill MQA logits (#51314 )
2026-10-13 08:21:52 +03:00
f7ba007b5e
[ROCm][MoE] Split AiterExperts by weight-quantization key (#60322 )
Divakar Verma and Claude Sonnet 5
2026-10-13 01:13:33 -04:00
b337523c11
[XPU] [Test] Extend spawn-safe sitecustomize monkeypatch to XPU (#57911 )
2026-10-13 13:12:10 +08:00
5555a2d8f7
[Bugfix][Model] Fix EXAONE-4.5 image inputs: ViT received un-normalised pixels (#61104 )
Mook and cjackal
2026-10-13 14:02:12 +09:00
69d00bc264
[Bugfix] Accept whole-number floats for integer tool arguments in the response parser (#61297 )
Yoni Gozlan and Bugen Zhao
2026-10-12 21:55:20 -07:00
28d679e762
[Bugfix] Preserve independent tokenizer code_revision in snapshot resolution (#61272 )
dskwe and Cyrus Leung
2026-10-13 12:23:45 +08:00
657a2e9417
[Rust Frontend] Accept whole-number floats for integer tool arguments (#61299 )
Yoni Gozlan and Bugen Zhao
2026-10-12 21:22:48 -07:00
c5422f4c46
[Test][Renderer] Cover the reasoning -> reasoning_content mirror in chat message parsing (#56129 )
RAHUL KUMAR and Claude Sonnet 5.5
2026-10-13 09:28:32 +05:30
df6c7ad031
[Core][Refactor] Move KV cache planning logic to kv_cache_planning.py (#61183 )
2026-10-13 11:05:39 +08:00
c077dc663d
[ROCm][Perf][DSv4.1] Enable decoder SWA bounded replay (#59039 )
frida-andersson
2026-10-12 21:19:14 -05:00
69d2b6f82d
[CI][XPU] Standardize test job labels for intel jobs (#51755 )
xiangdong and Kunshang Ji
2026-10-13 10:18:25 +08:00
cc2a6eea1a
[ROCm][Perf][DSv4.1] Mono decode on every backbone layer (#61082 )
Fangzhou Ai and Claude Opus 5.5
2026-10-12 22:16:50 -04:00
f47792af7f
[Model Loader] Make dummy weight init dtype-aware (#61134 )
Woosuk Kwon and Claude Opus 5.5
2026-10-12 18:33:26 -07:00
c6566bb9f6
[ROCm][MLA] Keep DCP prefill context in FP8 through AllGather (#59070 )
2026-10-13 09:20:38 +08:00
98f7211162
[Attention][Spec Decode] Multi-query Triton split-K for spec-decode verify, with empty-segment/padding hardening (#56148 )
2026-10-12 17:37:56 -07:00
189bbfb9f4
Add MoNe experts pruning support (#50415 )
2026-10-13 00:18:43 +01:00
1484266938
[Perf] Improve _topk_topp_kernel() for masked logits (#61213 )
2026-10-13 01:01:11 +02:00
