This website requires JavaScript.
5bf32605a9
[Rust Frontend] Deduplicate request preprocessing for /tokenize (#50448 )
Sage and Bugen Zhao
2026-08-05 11:27:35 +03:00
fd859d2e06
[KV Connector][Mooncake] Add store group semantics (#44956 )
2026-08-05 16:25:38 +08:00
f5cd862dbd
[ROCm][Bugfix] Kimi-K3 Fix KDA NaN on mixed batches and racy autotune config (#50649 )
kliuae
2026-08-05 16:07:21 +08:00
b92352ca2c
[Perf][KV Offload] Avoid quadratic ARC batch eviction (#50992 )
MINJUN GIL
2026-08-05 16:58:36 +09:00
96e333e81a
[CI] Run control-plane workflows on vLLM runners (#51127 )
Kevin H. Luu and OpenAI Codex
2026-08-05 00:14:27 -07:00
999dd8b490
[XPU] Alias is_current_stream_capturing to XPU in cuda wrapper (#50526 )
Sundaresan G and Kunshang Ji
2026-08-05 12:40:17 +05:30
4719a9b8f5
[Bugfix][Spec Decode] Fix EAGLE3 DeepSeek draft crash on non-YaRN rope configs (#51092 )
qizixi and Claude Opus 5
2026-08-04 23:10:52 -07:00
50c51682a1
[Bugfix][EC Connector] Don't stop an encoder-instance request before its images are encoded (#50275 )
Tianyu Guo
2026-08-05 13:53:58 +08:00
7794b1e08b
[Bugfix] Skip Qwen3 deepstack buffers without vision (#49397 )
Wei-Cheng (Wayne) Chiu and OpenAI Codex
2026-08-05 12:46:23 +08:00
eb3dce9757
[Bugfix][Model] Gemma3n/Gemma4: pad variable-length audio batches (#50958 )
TrainToGPB and Claude
2026-08-05 13:46:00 +09:00
c25093a305
[Kernel] Batch invariant NVFP4 MoE using cutlass (#40372 )
2026-08-05 06:31:32 +02:00
41a7e7da0c
[KV Offload] Support partial secondary-tier load results (#50321 )
2026-08-04 20:48:58 -07:00
6153dbe036
[R3] Unify routed expert shape configuration (#50940 )
aoshen02 and yihengz
2026-08-05 11:20:41 +08:00
0187f4c88e
[CPU] Enable tcmalloc for s390x (#50841 )
Rehan Khan and Li, Jiang <jiang1.li@intel.com>
2026-08-05 08:25:39 +05:30
2cb3ff88cc
[BUGFIX][Quant]Fix test_kv_scale_reload failed (#50405 )
Yejing Lai and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 10:50:54 +08:00
c416f15710
[Model] Fix Kimi-K3 MLA with disabled context parallelism (#50404 )
Dr Tasos Varoudis and Lucas Wilkinson
2026-08-05 03:33:14 +01:00
bb543efdfd
[ROCm][CI] Add aiter per-token FP8 quant roundtrip and RMSNorm determinism tests (#50905 )
Divakar Verma
2026-08-04 22:32:08 -04:00
9cd9f8c20b
Prune redundant tests points in correctness_e2e/[test_sequence_parallel,test_async_tp] (#51068 )
Michael Goin
2026-08-04 22:18:50 -04:00
33c50587d2
[ROCm] Restore Inkling MTP backend parity (#50806 )
Andreas Karatzas
2026-08-04 21:02:57 -05:00
d16e7f9da4
[CI] Prune PyTorch Compilation Unit Tests (#51069 )
Michael Goin
2026-08-04 21:53:50 -04:00
77b519912b
[CI][Bugfix] Fix test_shutdown_on_engine_failure startup deadlock (#51050 )
Nick Hill
2026-08-04 18:29:35 -07:00
d0ce3dadb6
[ROCm]: Bump torch 2.12, triton 3.7, torchaudio, torchvision (#50607 )
Rohan Potdar and Claude Opus 4.8
2026-08-04 20:14:11 -05:00
43c4bdcae9
[CI] Add run-all comment commands (#51087 )
Kevin H. Luu and OpenAI Codex
2026-08-04 17:34:27 -07:00
ffee32460d
[BugFix][Mooncake] Use global data_parallel_index for the DP engine index (#48061 )
Yifan Qiao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-04 17:10:51 -07:00
12292d94b2
[CI] Detect and fail evals on when NaNs appear in logits (#50323 )
Tyler Michael Smith and OpenAI Codex
2026-08-04 19:02:58 -04:00
3756bf1e2b
[Kernel][SM100] Add a CuTeDSL fused query kernel (#49792 )
xiaozhoupy and Claude
2026-08-05 06:23:06 +08:00
c687c1abb8
[ci] Update CI notify workflow with PR write permissions (#51079 )
Kevin H. Luu
2026-08-04 14:15:27 -07:00
a5149b2fee
[CI] Stabilize GLM-5.2 PCP evaluation (#51015 )
Kevin H. Luu and OpenAI Codex
2026-08-04 14:13:59 -07:00
05b7876f50
[MoE][Humming] Support SiTU activation for Kimi-K3 (#50510 )
2026-08-05 03:39:18 +08:00
edbc4969a7
[Kernel][Inkling] Fuse shared-expert partial addition into the Lamport collective (#50697 )
Canlin Guo
2026-08-05 02:41:35 +08:00
8ae8337ffa
[Spec Decode] Enable fused non-causal TokenSpeed MLA for DSpark (#50911 )
NVShreyas and OpenAI Codex
2026-08-04 11:05:40 -07:00
d31de3c421
[Kimi K3 Perf] option to shard the shared expert for non mega case, 16.98 GiB memory/GPU saved (#50912 )
Wentao Ye
2026-08-04 14:00:44 -04:00
7cab4368f2
[MM][CG] Support ViT full CUDA graph for Kimi-K2.5 (#50929 )
Linkun and lkchen
2026-08-04 10:15:56 -07:00
7635a9002b
[Core] Explicitly manage torch CPU threads in workers (#49919 )
Nick Hill and Claude
2026-08-04 10:06:38 -07:00
52c0e3cb08
[Kernel] Add support for Flashinfer Mamba SSU algorithm selection (#50157 )
amitz-nv and OpenAI Codex
2026-08-04 19:40:23 +03:00
7ac2ec7582
[Kimi-K3][AMD] Fuse AttnRes state updates and norms (#50593 )
2026-08-05 00:32:13 +08:00
7153fd70f6
[Bugfix][MoE] Filter packed expert weights during EP loading (#49558 )
aoshen02 and aoshen02
2026-08-05 00:32:03 +08:00
166f4e2dc3
fix: fuse weightless RMSNorms at their declared width (#50867 )
2026-08-04 21:33:09 +05:30
4f819f801b
[Model Runner v2] E/P/D disaggregation support (#38390 )
Wentao Ye
2026-08-04 11:52:23 -04:00
7743486190
[Frontend] DeepSeek V4 0731 reasoning effort prompts & mappings (#50580 )
Bugen Zhao
2026-08-04 23:51:32 +08:00
f9c74b4b9c
[Mamba] enable prefix cache by default (#50991 )
Jiangyun Zhu and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-04 23:20:51 +08:00
7b50d2c0bc
[Misc] Avoid importing nixl_ep on every vllm serve config (#50879 )
Nicolò Lucchesi
2026-08-04 16:56:54 +02:00
122b3d46b6
[Bugfix][CPU] Fix macOS build: std::sqrt is not constexpr under libc++ (#50915 )
Harjoth Khara
2026-08-04 07:56:33 -07:00
199644d410
[Bugfix][Attention] Guard sparse MLA masked MHA workspace (#50906 )
2026-08-04 21:59:33 +08:00
7c40d61eaf
[Bugfix] Flatten >2D multimodal embeddings, not just 3D (#50250 )
Michał Ganczarenko
2026-08-04 15:34:58 +02:00
5f2ee2fa8c
[Bugfix][Core] Log KV cache capacity after block-size resolution (#50462 )
2026-08-04 08:53:32 -04:00
5789897aa4
[Spec Decode] Add top-k DSpark Markov projection (#49969 )
Andrii Skliar and Andrii Skliar
2026-08-04 14:52:53 +02:00
1eb3694521
[Docs] Fix two docs build warnings (#51014 )
Harry Mellor and Claude Opus 5
2026-08-04 13:29:04 +01:00
59b2fdfc4e
Support MLA properly in the Transformers modeling backend (#48250 )
Harry Mellor
2026-08-04 13:24:11 +01:00
24c939c47d
[XPU] fix collecting oneccl version info (#47104 )
Yan Ma
2026-08-04 19:53:28 +08:00
6a9fdf0dc7
[Bugfix] Resolve seq-cls num_labels from the top-level config for multimodal checkpoints (#50950 )
Lazurite
2026-08-04 19:39:39 +08:00
a1657a0235
[Bugfix][Build] Fix DeepGEMM CUDA 12.9 FP8 header visibility (#51003 )
Kevin H. Luu
2026-08-04 03:06:04 -07:00
e98a8774cb
[Rust Frontend][gRPC] Add multimodal image inference (#50368 )
Connor Carpenter and Bugen Zhao
2026-08-04 02:34:31 -07:00
0b1c151cbc
[Rust Frontend] Align tool rendering for Kimi K3 (#50540 )
Bugen Zhao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-04 16:03:46 +08:00
72cd5424da
[Rust][Benchmark] Preserve UTF-8 across benchmark stream chunks (#50868 )
Reid
2026-08-04 14:15:02 +08:00
cb8104839c
[XPU][CI]Adjust Samplers test ENV for Intel GPU (#50199 )
xiangdong and Kunshang Ji
2026-08-04 11:05:45 +08:00
adbf08d977
[Bugfix][Reasoning] kimi_k3: O(delta) reasoning-end check on the decode path (#50886 )
Bowen Wang and Claude Opus 5
2026-08-04 10:37:03 +08:00
385d4c084e
[ROCm][AITER] Hotfix for memory access fault errors in AITER triton MOE routing (#50859 )
fxmarty-amd and Andreas Karatzas
2026-08-04 04:07:46 +02:00
74295e3bd4
[CI][Bugfix] Fix flaky test_store_orders_after_compute_write (#50926 )
Nick Hill and Claude
2026-08-03 17:54:49 -07:00
413e70d52c
Perf/h20 moe config e256 n512 (#48825 )
zzt
2026-08-04 08:47:58 +08:00
8adc840c45
[ROCm][Test] Use BF16 for Jina v5 nano MTEB test (#50917 )
Andreas Karatzas
2026-08-03 19:33:05 -05:00
11f88260a2
[Kimi-K3] Migrate FlashKDA to PyTorch stable ABI (#50818 )
Thien Tran and Shengqi Chen
2026-08-04 08:18:08 +08:00
41ba11b841
[Bugfix][Kimi-K3] Enforce packed rows and op availability in AttnRes dispatch (#50567 )
namgyu-youn and Claude Opus 5
2026-08-04 09:13:33 +09:00
f42761204f
llmd+vllm+mori-ep(inter node wide-ep)+mori-io(write) for 2p2d with dp=ep=16 tp=1 (#45043 )
2026-08-03 17:13:15 -07:00
c810937573
[Kernel] Extend CuTe DSL skinny GEMM to GLM-5.2 (#49791 )
2026-08-04 07:49:41 +08:00
6a9109d865
[Bugfix] Fix Qwen3-Omni crash on video with no audio track when use_audio_in_video=True (#48420 )
Ryan Hamby and Isotr0py
2026-08-03 19:22:00 -04:00
c2881ce603
[Bugfix][Hybrid] Fix cross-block race on num_accepted in MRv2 align prefix cache (#50432 )
Francesco Fusco
2026-08-04 00:54:24 +02:00
0b37d8389f
fix: NVFP4 quantization out_dtype should match model dtype, not torch default (#48861 )
2026-08-03 16:47:37 -06:00
f57123aa2d
feat(frontend): session id plumbing into requests (#48048 )
Karen Chung
2026-08-03 15:18:58 -07:00
e578de311c
[ModelRunnerV2] Fix scalar Mamba state update with int32 mappings (#50327 )
Varun Vinayak Shenoy and Nick Hill
2026-08-03 15:05:12 -07:00
f43e1d26e3
[ROCm] Enable AITER and FP8 inference on GFX120x (#43615 )
skysnow2001 and Douglas Lehr
2026-08-03 17:45:37 -04:00
4a3447d200
[Bugfix][Model Runner V2] Restore multimodal draft capability detection (#50417 )
Raphaël Rialland
2026-08-03 23:18:14 +02:00
42ab184ea7
[MRV2] Enable routed-experts capture (#50721 )
aoshen02 and Nick Hill
2026-08-04 05:00:13 +08:00
e279f71583
[ROCm][Test] Fix AITER MXFP4 oracle contract (#50728 )
Andreas Karatzas
2026-08-03 15:53:18 -05:00
c4e9f09de7
[Bugfix][Frontend] Reject empty gRPC stop strings (#50746 )
zcxGGmu and zq
2026-08-04 04:50:15 +08:00
755513d9a2
[Hybrid] Stage the postprocess inputs with a single loop over the request list (#48120 )
Francesco Fusco and Claude
2026-08-03 22:24:22 +02:00
1c0d207915
[Bugfix] Default Gemma3 Model intermediate_tensors to None (#50777 )
Taneem Ibrahim and Wentao Ye
2026-08-03 15:07:59 -04:00
7f2e78ba4d
[Bugfix] Emit a valid media type from encode_{audio,image,video}_url (#49056 )
Vineeth Sai Varikuntla
2026-08-03 12:02:23 -07:00
8f50685c48
[ROCm][Kimi-K3] aiter moe environment variable cleanup (#50582 )
Hongxia Yang
2026-08-03 14:54:04 -04:00
952694e384
[Bugfix] Validate NIXL speculative config compatibility (#49230 )
2026-08-03 13:51:04 -05:00
65cf1276a2
[Kernel] Skip fully masked key blocks in windowed Triton prefill (#50776 )
Almog Tavor
2026-08-03 21:48:47 +03:00
76d995df2c
[CI][ROCm] Export Helion benchmark script in test artifacts (#50726 )
Andreas Karatzas
2026-08-03 13:09:44 -05:00
f0de1a604c
[Bugfix] Shard UniformTypeKVCacheSpecs block table width under DCP (#50823 )
2026-08-03 20:34:29 +03:00
5df9999fcf
[Kimi-K3] Add option to shard the shared expert instead of replicating (#50656 )
2026-08-03 11:41:24 -04:00
dd11df04f3
[Misc] Remove deprecated calculate_kv_scales runtime KV scale calculation (#49389 )
wangxiyuan
2026-08-03 23:15:13 +08:00
0cf49a5d15
[Refactor] Remove multiple dead codes (#50285 )
Wentao Ye
2026-08-03 10:59:08 -04:00
68ca6fd02c
[Bugfix] Remove bad startup assertion (#50869 )
Benjamin Chislett
2026-08-03 10:21:50 -04:00
8ba87e0181
[CI] Mooncake PD integration tests (#46844 )
Nicolò Lucchesi
2026-08-03 15:55:51 +02:00
005fa01756
docs: document reasoning_content output removal as a breaking client change (#50624 )
Fede Kamelhar and Harry Mellor
2026-08-03 09:45:23 -04:00
9ae11a6b89
[Model] Add K-EXAONE-2.0-750B-A37B (#50524 )
Kyungmin Lee and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-03 22:33:02 +09:00
b977407d8b
Support quantized DSpark Markov heads (#50424 )
2026-08-03 14:35:39 +02:00
ad0bf3963e
[CI] KimiLinear PD in nightlies (#50266 )
Nicolò Lucchesi and Claude
2026-08-03 14:20:41 +02:00
ec40f6a8a6
[CPU] Fix torch.compile crash from torch.accelerator.synchronize on CPU-only hosts (#49960 )
2026-08-03 17:18:48 +05:30
32c42c4f2f
[UX] remove torch compile warning when using breakable cudagraph (#50750 )
Jiangyun Zhu
2026-08-03 19:25:49 +08:00
9acb7b3699
[CI] And PPL test for multimodal generation models (#50839 )
wang.yuqi
2026-08-03 19:25:08 +08:00
c8602c7906
[CPU] Refine CPU kernel dispatch (#50801 )
Li, Jiang
2026-08-03 17:42:53 +08:00
b9d1e2437e
K3: Move LatentMoERunner (#50678 )
Jee Jee Li
2026-08-03 17:36:17 +08:00
d9dac2b3d4
fix: remove stray duplicate from serving benchmark config (#46870 )
Christopher Miyai and Harry Mellor
2026-08-03 04:34:12 -04:00
89ac407e3d
[Perf] Speed up multimodal placeholder and token-match scanning (#50716 )
Hareesh Gali and Claude Fable 5
2026-08-03 11:27:56 +03:00
d83eb0b36b
fix(security): classify DeepStream as GPU backend and enforce pixel limits (#50755 )
Juan Pérez de Algaba
2026-08-03 10:23:09 +02:00
