Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 5633405964 Added extra_repr() to pooler classes to improve debuggability (#44805) Taneem Ibrahim 2026-06-07 23:19:31 -04:00
• 6124a98a9b [Bugfix] Fix FunASR-Nano crash during initialization (#44215) SunskyXHandRoger Wang 2026-06-08 12:00:02 +09:00
• 2ed0a9627b [Kernel][Test] Make kernel tests for mamba dual-HW (CUDA + XPU) (#42736) Agata Dobrzyniewicz 2026-06-08 02:22:47 +02:00
• 4dcd10eb0d [1/N][KV-Cache Layout Refactor] Refactor DSV4 KV cache config construction (#44454) 4 people 2026-06-07 08:53:37 -06:00
• 228bcc436b [ROCm][Kernel] Enable permute_cols for ROCm (#44674) Charlie Fu 2026-06-07 04:50:03 -05:00
• 3d3ba460a2 Modify torch dependency in xpu.txt (#43087) Bram VanroyandKunshang Ji 2026-06-07 10:33:50 +02:00
• 66ecfd0568 [Dependency] Remove stale cuDNN frontend upper bound (#42599) Mohammad Miadh Angkad 2026-06-07 16:09:25 +08:00
• f0f6805d8a [CI] Stabilize the multi-audio OpenAI server path (#44051) Andreas Karatzas 2026-06-07 02:54:32 -05:00
• 15652a6b70 [Doc] Fix multimodal torch.compile troubleshooting to not use removed VLLM_TORCH_COMPILE_LEVEL (#44378) Daoyuan LiandCyrus Leung 2026-06-07 00:34:07 -07:00
• 51ef688831 [Bugfix][Mooncake] Fix per-group block_size/block_hash and group_idx in MooncakeStoreConnector KV events (#44103) Yifan Qiao 2026-06-07 00:12:43 -07:00
• 6ac69203e8 [videoloader] implement glm46v video loader (#44417) Jared Wen 2026-06-07 14:27:20 +08:00
• 1505b3d8a1 [Cohere] Enable Cohere Mini Code model and update Command A-plus test registry (#44707) Terrence Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-07 01:44:16 -04:00
• 32f34d3935 [feature] add index share feature for DSA MTP (#44420) Jared Wenandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-07 13:04:14 +08:00
• 9c7f7741d4 [Bugfix] Fix benchmark_moe.py after inplace mechanism removal (#44041) Qiuyang Yue 2026-06-06 21:32:00 -07:00
• 6181e80fe0 [XPU] add xpu branch in compressed_tensors_moe_w4a4_mxfp4 (#44540) zofiaandKunshang Ji 2026-06-07 12:27:34 +08:00
• 3bb46975bd [XPU][Feature] transparent sleep mode support for XPU platform (#37149) Yan Ma 2026-06-07 10:45:31 +08:00
• 810966453a [XPU] Support cpu kv offloading and tiering offloading on XPU platform (#36423) Chaojun Zhang 2026-06-07 09:59:28 +08:00
• 2a983c79ac [DSV4] Decouple DS V4 Sparse MLA Metadata from DS V3.2 (#44699) Woosuk Kwon 2026-06-06 17:37:56 -07:00
• bc5745a00f [ROCm][MLA] Replace torch.cat in sparse-MLA forward_mqa with fused concat_mla_q (#42838) 5 people 2026-06-07 02:20:50 +03:00
• 3b3d5287fa [BugFix] Resolve multiple async kv load deadlock (#44560) Nick Hill 2026-06-06 16:05:47 -07:00
• 062b05ff3a [ROCm][Perf] Fused MoE W4A16 HIP kernel for AMD RDNA3 (gfx1100) (#44075) JartXandTJian 2026-06-06 22:30:39 +02:00
• fa27d4e9cf [PERF] [Qwen3.5] Split mixed prefill+decode batches: route decodes to the recurrent kernel (#44700) Vadim Gimpelsonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-06 18:13:50 +04:00
• 67d3792d99 [Bugfix] Fix Qwen3.5-FP8 nightly fail. Guard fused_add_rms_norm input/weight dtype mismatch in RMSNorm + quant fusion (#44694) Vadim Gimpelson 2026-06-06 16:46:14 +04:00
• 00d1fb7747 [Bugfix][ROCm] ApplyRotaryEmb: fall back to native when flash_attn rotary grid would exceed the HIP per-dim limit (#43684) 3 people 2026-06-06 17:29:16 +08:00
• c9b4b184b4 [Bugfix][Voxtral] Add fetch_audio to MistralCommonFeatureExtractor (transformers>=5.10 compat) (#44559) 3 people 2026-06-06 00:58:09 -07:00
• f87df1df9e [Bugfix][MoE] Snapshot max_cudagraph_capture_size into FusedMoEConfig (#44613) aoshen02andClaude Opus 4.8 2026-06-06 14:14:41 +08:00
• eafbb06331 [Misc] Replaced asserts with proper exceptions to improve UX for pooling (#44593) Taneem Ibrahim 2026-06-06 00:57:26 -05:00
• ec0a31d4aa [Bugfix][Kernel] Fix mHC fused-RMSNorm big-fuse miscompile for hidden_size != 4096 (#44692) Yongye ZhuandClaude 2026-06-05 22:44:21 -04:00
• c8beda4cc3 [Rust Frontend] Add Phi-4 mini JSON tool parser (#44213) Devin Lai 2026-06-06 10:40:00 +08:00
• 2f27c9a150 Preserve layout-changing clones (#44574) Mike GandMichael Gschwind 2026-06-05 17:45:24 -07:00
• 4765f0f189 [Bugfix] Fix sequence_parallel_chunk_impl custom op aliasing its input (#44130) Vadim GimpelsonandCursor 2026-06-06 03:56:36 +04:00
• a50e675b0d [Cohere] fix RoutingMethodType (#44021) Terrence Zhao 2026-06-05 19:25:53 -04:00
• f6a708ab2b [Doc] Add Llama-3.2-3B-Instruct to batch-invariance tested models (#44435) Daoyuan Li 2026-06-05 16:04:32 -07:00
• 4200f62147 [ROCm][GPT-OSS] Fuse RoPE + static Q FP8 quant on fused RoPE+KV path (#42832) akii96andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-06 00:22:19 +03:00
• c73b0d0db9 [Core][Engine] allow DP ray placement groups to be set on specific nodes (#44669) Walter Beller-Morales 2026-06-05 16:07:47 -04:00
• e28e369f78 Male Mergify comment less spammy (#44666) Harry Mellor 2026-06-05 18:56:52 +01:00
• 703fb17b13 [Bugfix] GPT-OSS instruction rendering (#44330) yzong-rh 2026-06-05 13:52:32 -04:00
• b593396c7a Upgrade tpu-inference to v0.21.0 (#44621) Sting Lin 2026-06-06 00:12:49 +08:00
• 91e17d4315 Fix sarvam forward compatibility with transformers v5 (#38804) Flame 2026-06-05 21:21:44 +05:30
• aa6fb8a329 [Bugfix] [ROCm] [Critical] fallback to regular abi for ROCm (#44648) TJian 2026-06-05 23:51:17 +08:00
• 6a894574bf Add objectstore as a secondary tier to multi-tier kv cache offloading (#41968) Effi Ofer 2026-06-05 18:05:41 +03:00
• 7f003a1285 Support MiniCPMV batched preprocessing (#44609) Yan Ma 2026-06-05 23:05:31 +08:00
• ef0df7dbd6 [CI] Bump mypy version 1.19.1 -> 1.20.2 (#44647) Harry Mellor 2026-06-05 15:56:27 +01:00
• a80af24356 Speed up docs build (#44635) Harry Mellor 2026-06-05 15:51:44 +01:00
• c66b19800b [CI] Bump mistral-common (#44649) Harry Mellor 2026-06-05 15:18:50 +01:00
• 6a11d72df7 [Reasoning][Structured Outputs] Add Command A plus tags for structural tags (#44588) rishitdholakia13andChauncey 2026-06-05 09:51:20 -04:00
• 02d2da0748 [DSV4] Move more ops out of eager breakpoint (#44561) Woosuk Kwon 2026-06-05 06:42:41 -07:00
• bbb6c274c8 [Bugfix] Fix gemma4 crash on CPU: guard mem_get_info call (#44615) adhithyamulticoreware 2026-06-05 18:17:56 +05:30
• 62215e72c6 Remove KV cache scale boilerplate from model weight loading methods (#43167) Harry Mellorandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-05 13:19:04 +01:00
• 7fe7800fa4 [BUG] Fix FP64 Gumbel precision coverage (#43150) 3 people 2026-06-05 04:04:14 -07:00
• 8a83e6f2d7 [Rust Frontend] Batch auto-abort requests by engine (#44591) HueCodesandBugen Zhao 2026-06-05 02:59:09 -07:00
• efc347f1b2 docs: fix tokenizer optimization typo (#44066) Chunyang Wen 2026-06-05 17:12:49 +08:00
• d98b8f371c [NixlConnector] Initiate deprecation cycle for kv_both role (#43874) Nicolò Lucchesi 2026-06-05 11:08:17 +02:00
• e64237ae82 [Rust Frontend] Support include_reasoning=false (#44391) Chao-Ju Chen 2026-06-05 16:47:50 +08:00
• d61d8566ec [Bugfix] Update mistral tokenizer test for continue_final_message fix (#44622) XuZhouandXu Zhou 2026-06-05 16:13:26 +08:00
• d2f70da116 fix: pad dummy run query_start_loc (#44603) Uranus 2026-06-05 15:43:04 +08:00
• 6542d48964 [Bugfix] Fix test_invocations flaky failure with newer openai SDK (#44618) XuZhouandXu Zhou 2026-06-05 15:36:20 +08:00
• ca73293fa6 [Bugfix][Rust Frontend] Fix UTF-8 char-boundary panic in incremental detokenizer (#44620) Ting SUN 2026-06-05 15:36:17 +08:00
• ef3af56a97 Fix LLM.wait_for_completion output type docstring (#44617) Vic Wen 2026-06-05 15:16:38 +08:00
• b4a6f26c90 [ROCm][perf] Use workspace manager for sparse indexer allocations (#41002) 3 people 2026-06-05 09:46:29 +03:00
• 165b7864d0 [ROCM] [FEAT] Integrate Aiter hipBLASLt GEMM online tuning (#40426) 3 people 2026-06-05 14:45:36 +08:00
• c505cd93ef [CI/Build] Disable CPU-Compatibility Tests (#44605) Li, Jiang 2026-06-05 13:14:26 +08:00
• 96229fa99e [KVConnector][1/N] PP-aware handshake aggregation and intermediate-PP output plumbing (#43720) qizixi 2026-06-04 22:04:19 -07:00
• da1daf40bf [Bugfix] Exclude vision embedder from quantization in Gemma4 Unified (#44571) Luciano MartinsandLuciano Martins 2026-06-05 00:47:38 -03:00
• 4efd6ffde0 [DSV4] Refactor DeepseekV4Attention (#44569) Woosuk Kwon 2026-06-04 20:23:07 -07:00
• 56aff0dd15 [10/n] Migrate cuda_view and silu_and_mul_per_block_quant kernels to torch stale ABI. (#44334) Chris Leonard 2026-06-04 23:14:43 -04:00
• 063ce98fb7 [XPU][MoE] support block_fp8_moe on xpu (#42139) zofia 2026-06-05 08:36:58 +08:00
• 62d6f06e3d [Rust Frontend] Skip loading multimodal processor if --language-model-only is specified (#44500) Bugen Zhao 2026-06-05 08:02:54 +08:00
• b7c5baf63d fix: keep DeepSeek V4 RoPE cache on inv_freq device (#43926) Schwinn Saereesitthipitak 2026-06-04 15:30:29 -07:00
• a55fccfc7c [mamba] unify KDA conv states into one cache to match 2-state SSM layout (#44539) Jiangyun Zhu 2026-06-05 02:38:05 +08:00
• 41a4829f22 [Logs Refactor] Optimize shutdown logs, easier to follow and consistent (#43707) Wentao Ye 2026-06-04 14:36:32 -04:00
• 38fd2405f3 use split_group for pytorch process group creation (#41980) Tushar JainandTushar Jain 2026-06-04 14:36:07 -04:00
• a947f7a420 [Kernel][Test] Extend lightning_attn and awq_triton kernel tests to XPU (#43307) Agata Dobrzyniewicz 2026-06-04 20:25:59 +02:00
• 439203d32c [Bugfix] Fix test_cutlass_moe.py (#44380) bnellnm 2026-06-04 14:18:52 -04:00
• 8d9536a775 [Misc] Add unit tests for pooler head classes (#44471) Taneem Ibrahim 2026-06-04 13:59:25 -04:00
• 3da29aa4a5 [DOC] Add INT8 W4A8 docs and Arm's supported quantization schemes (#34894) Fadi Arafeh 2026-06-04 17:27:17 +01:00
• 06f94633e7 [ROCm][CI] Add test for Aiter unified attn kernel (#44436) Divakar VermaandAndreas Karatzas 2026-06-04 12:15:05 -04:00
• 99ef652907 [Bugfix] Reject non-positive values for ParallelConfig int knobs (#44057) JianweiZhengandWentao Ye 2026-06-04 23:46:50 +08:00
• 4cc78c9d5d [Core] Freeze garbage collector in workers after model initialization (#44363) Tyler Michael Smith 2026-06-04 11:39:04 -04:00
• 3dbb4e0ace [Bugfix] MiniCPM-V-4.6 video inference crash: placeholder count mismatches visual embedding count (#44509) tc-mb 2026-06-04 23:22:30 +08:00
• b21443e23c Add model support for granite speech plus (#43519) Zvi KonsandClaude Opus 4.7 2026-06-04 17:47:48 +03:00
• 06ee2d8433 [Quant] Support compressed-tensors WNA8O8Int linears and WNInt embeddings (#44340) Michael Goin 2026-06-04 10:40:33 -04:00
• b5235fca2e [DSv4] Adding TRTLLM gen attention kernel (#43827) Yongye Zhu 2026-06-04 10:35:09 -04:00
• 3e77036768 [ROCm][CI] Specifying time outs for the lm eval models (#44255) Andreas Karatzas 2026-06-04 09:35:00 -05:00
• 6f68ca3e91 [ROCm][CI] Stabilize memory-release in the Hybrid model generation tests (#44046) Andreas Karatzas 2026-06-04 09:34:24 -05:00
• 0c96dd64fb [ROCm] Bump fastsafetensors to v0.3.2 from PyPI, remove git source build (#43625) Turner Jabbour 2026-06-04 15:30:57 +01:00
• 68f5e565c9 [PD][Nixl] Mamba prefix caching mode support (#42554) Nicolò Lucchesi 2026-06-04 15:41:46 +02:00
• 9354fb1ba5 [Bugfix][Compile] Guard per_token_group_fp8_quant lookup on non-CUDA platforms (#44476) QiliangCui2023 2026-06-04 06:31:50 -07:00
• f35b557239 Add GH token to docs build pre run check (#44534) Harry Mellor 2026-06-04 13:43:49 +01:00
• e68988a248 Refactor CT NVFP4 linear to use a single class (#42443) Dipika Sikka 2026-06-04 08:25:08 -04:00
• 4b87b3e845 [Bugfix] fix EVS for qwen3-vl (#44205) Rui "Garry" GaoandCyrus Leung 2026-06-04 19:06:51 +08:00
• 90619351e3 [Attention] Mamba attention module refactor - LINEAR (#43556) wangxiyuanandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-04 18:45:29 +08:00
• d0975a4b50 [perf] Add gemma RMS AR fusion (#42646) 3 people 2026-06-04 16:33:59 +08:00
• 1bdc60ed53 Fix Kimi-K2.5 FlashInfer ViT metadata (#44493) Kevin_Xiong 2026-06-04 16:14:35 +08:00
• a6183563b6 [Prefix Caching] DeepSeekv4 - Support selective prefix-cache retention for sliding-window KV cache (#43447) Wei ZhaoandYifan Qiao 2026-06-04 03:48:31 -04:00
• 22c2e87555 [CI] Reverted gitignore changes (#44497) Andreas Karatzas 2026-06-04 02:37:44 -05:00
• d01d0b4646 [Frontend] Consolidate online serving utils. (#44479) wang.yuqi 2026-06-04 14:49:31 +08:00
• b4b4aaa70e [Inductor] Fast-path Inductor fallback for vllm::*/vllm_aiter::* custom ops (#42129) Oxana KorzhandCursor 2026-06-03 23:03:52 -06:00
• 5e2af28838 [CI] Resolve release V2 docker build after ROCm CI wheels change (#44463) Andreas Karatzas 2026-06-03 23:35:40 -05:00
• 4f423bd5bc [EPLB] Nixl communicator optimization. Zero-copy transfers (#41633) 5 people 2026-06-04 05:40:34 +02:00