Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• a9484dac7b [Perf] Intergrate Tile Kernels head_compute_mix_kernel for Deepseek-V4 (#41255) Isotr0pyandRoger Wang 2026-05-02 03:01:17 +08:00
• f3fef12350 [Attention] Abstract the MLA prefill backends and eliminate cuDNN (#32623) 4 people 2026-05-01 13:36:20 -04:00
• 51295793a2 [Model Runner V2] Add logprob_token_ids support (#40559) Wentao YeandNick Hill 2026-05-01 13:02:03 -04:00
• 3ccc1ff495 [Eval][CI] Add basic mrcr eval to tests/evals/ (#40164) Michael Goin 2026-05-01 12:00:38 -04:00
• 529c671e80 [ROCm][FEAT] AITER Fused Allreduce + RMSNorm (#37646) 5 people 2026-05-01 23:07:18 +08:00
• bc635fad23 [ROCm][Deepseek] dsv3.2 further optimization (#41217) PleaplusoneandMatthew Wong 2026-05-01 22:06:00 +08:00
• c3e64696cd [Perf] Warmup forward_native sampler kernel (#41375) Artem Perevedentsevandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-01 17:04:11 +03:00
• 4f7bde572a [Kernel] Pack output and LSE in DCP A2A (#41160) sungsoo ha 2026-05-01 06:01:17 -07:00
• 2fa1f8ec00 [kv_offload+HMA][13/N]: Enable HMA support (#41445) Or Ozeri 2026-05-01 14:30:03 +03:00
• 7075df79b3 [ROCm] Enable DBO (Dynamic Batch Optimization) on ROCm (#34726) raviguptaamd 2026-05-01 02:18:30 -07:00
• 0dbaf9daad Refractor longcat loading to use AutoWeightsLoader (#41448) Yuyi Ao 2026-05-01 05:07:23 -04:00
• a3ec4a35f5 [Bugfix][Metrics] Fix RayPrometheusMetric.labels() returning shared labeled child (#40840) 3 people 2026-05-01 00:43:39 -07:00
• 32964e7700 [ROCm][CI] Upgraded UCX and RIXL (#41210) Andreas Karatzas 2026-05-01 02:40:47 -05:00
• a07642667d [Bugfix] Pass reasoning parser kwargs to structured output (#41199) Bugen Zhaoandwang.yuqi 2026-05-01 14:38:02 +08:00
• c3868bbbe4 [compile] Add FlashInfer FP8 async TP fusion and preserve allreduce fusion ordering #27893 (#39505) baonudesifeizhai 2026-05-01 01:08:34 -04:00
• 947138b6c2 Add nvfp4 kv cache support (#40177) sychen52 2026-04-30 21:55:16 -07:00
• 941fb50835 [kv_offload+HMA][12/N]: Scheduler-side support for sliding window groups (#41228) Or Ozeri 2026-05-01 06:59:17 +03:00
• 6b6ac6c3c7 [Kernel][MoE] Support GELU on TRT-LLM NvFP4 fused MoE for Gemma4 (#41050) Juhi Mittalandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-30 20:37:43 -07:00
• b542bdf7fb [Bugfix] Disable FlashInfer CUTLASS MoE on SM110 (Jetson Thor AGX) (#40808) Stefano Castagnetta 2026-05-01 05:08:49 +02:00
• 415a879899 [KV Offload] Use Collection instead of Sequence/Iterable for OffloadingManager key parameters (#41361) Ronen Schaffer 2026-05-01 05:18:38 +03:00
• 7198940b39 [Model] Add Moondream3 model support(only query and caption skills) (#32325) Dong W 2026-04-30 19:06:48 -07:00
• 14043dfecd feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API (#40720) Luis 🚀andAndrew Sansom 2026-04-30 22:05:55 -04:00
• 1adaa5056b [ROCm][CI] Add ROCm score absolute tolerance floor (#41341) Andreas Karatzas 2026-04-30 20:59:35 -05:00
• 4d5c89295b (bugfix): block_size check for flex attn (#41363) SoyaazzandMatthew Bonanni 2026-05-01 09:59:26 +08:00
• dd5506a157 [Core] Simplify handling of scheduler_reserve_full_isl option (#41064) Nick Hill 2026-04-30 18:10:00 -07:00
• a3c83ff2fd Faster per-token fp8 group quant packed kernel for blackwell (#41326) Yongye ZhuandRoger Wang 2026-04-30 21:09:55 -04:00
• 9c61864bf8 [DeepSeek] Use torch.mm for bf16xbf16->fp32 gemm (#41300) Woosuk Kwon 2026-04-30 16:28:57 -07:00
• 71725f6730 [Bugfix] Fix RoutedExpertsCapturer for Gemma 4 MoE (top_k_experts) (#41401) Tran Le 2026-04-30 16:19:59 -07:00
• b4806c8ee1 [DSV4] Add BF16 and MXFP8 A2A support for flashinfer a2a one sided (#40960) 3 people 2026-04-30 18:33:12 -04:00
• 526927be94 [Model Runner v2] Fix v2 compile counter num_gpu_runner_capture_triggers and num_cudagraph_captured (#41285) Wentao Ye 2026-04-30 18:20:11 -04:00
• 75a4c166f2 Fix typo in log message for indexer cache (#41419) Michael Goin 2026-04-30 18:02:14 -04:00
• 2917d6363a [NVFP4][Hopper/AMD Instinct] Add Triton kernels for NVFP4 dequantization and QDQ emulation (#40033) fxmarty-amdandClaude 2026-04-30 23:35:48 +02:00
• efb4cdf2b8 [CI/Build] Skip Prithvi/Terratorch model-registry tests when terratorch is missing (#41389) Stefano Castagnetta 2026-04-30 21:47:55 +02:00
• 92a7c121b6 [CI] Add MTP coverage: Qwen3.5 correctness + no-sync spec decode (#40472) Stefano Castagnettaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-30 21:24:09 +02:00
• 307b17ce33 [DSV4] Avoid redundant dtype conversion. (#41374) Jee Jee Li 2026-05-01 00:57:27 +08:00
• 3ca6ca210f xpu docker: pin oneAPI to 2025.3 and avoid unintended 2026 upgrade (#41380) wenjun liuandKunshang Ji 2026-05-01 00:02:23 +08:00
• 10558f5f46 [CI/Build] Skip terratorch + torchgeo while PyPI has lightning quarantined (#41377) Stefano Castagnetta 2026-04-30 16:59:07 +02:00
• 121dbe7a22 [ROCm] ROCm DeepEP API updated to latest (#39721) 3 people 2026-04-30 09:46:59 -05:00
• f03d82efdd [UX][Bugfix] Fix OOM by setting PyTorch max_split_size_mb during model loading (#41268) Matthew Bonanni 2026-04-30 10:46:54 -04:00
• a7fb008510 [EPLB] Optimize memory overhead in Nixl communicator (#40013) 3 people 2026-04-30 16:46:49 +02:00
• ff449b6426 Stop mergify labelling from skipping pre-commit (#41362) Harry Mellor 2026-04-30 13:48:38 +01:00
• 3527229517 [Doc] Fix RTD build: pytorch.org/docs/stable/objects.inv returns 404 (#41353) Stefano CastagnettaandHarry Mellor 2026-04-30 14:06:44 +02:00
• b55b26520c [MoE] Make MoERunnerInterface a PluggableLayer for OOT support (#35178) 3 people 2026-04-30 18:31:08 +08:00
• 3179e53135 [P/D] Prefill compute optimizations with bi-directional KV cache transfers between P and D nodes (#32553) snadampal 2026-04-30 03:14:20 -07:00
• efdc95674d [KVConnector] MultiConnector SupportsHMA (#39571) Nicolò Lucchesi 2026-04-30 11:10:50 +02:00
• 54146a9bf9 [Bugfix] correct h matrix layout in chunk_kda output kernel (#40956) Chenxi Qianandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-30 16:22:41 +08:00
• ca97f7b9bb Fix Gemma4 MoE expert weight remapping (#41206) 3 people 2026-04-30 16:12:42 +09:00
• a04e0cf3b8 Fix Cohere ASR after HF upgrade (#40582) Ekagra Ranjan 2026-04-30 02:39:04 -04:00
• cb1b02d0e8 [Frontend] Add VLLM_SKIP_MODEL_NAME_VALIDATION environment variable (#34676) 6 people 2026-04-29 23:19:09 -07:00
• a749a33d8d [Bugfix] Fix persistent_topk cooperative deadlock at TopK=1024 (#41189) Yongye ZhuandClaude Opus 4.7 2026-04-30 00:03:45 -04:00
• c42981d034 [Refactor][kv_offload] KV Offloading maintainability improvements (#40538) Martin HickeyandOr Ozeri 2026-04-30 03:55:31 +01:00
• 0ff1bf9bb1 [Bugfix] Fix failure to allocate KV blocks error (#41282) Wei Zhao 2026-04-29 21:44:07 -04:00
• 0ab67c0222 [CI] Add key field to all test_areas pipeline steps (#41201) Kevin H. LuuandClaude 2026-04-29 16:59:16 -07:00
• 3795d7acf4 [ROCm][Bugfix][GPTOSS]: fix input_ids and expert_map args for quark w4a8 gptoss (#41165) Rohan Potdar 2026-04-29 18:39:01 -05:00
• 18599bfdf2 [Ci][BugFix] Fix slow DP tests due to bad teardown logic (#41166) Nick Hill 2026-04-29 16:31:00 -07:00
• 296741d025 [DSv4] Use cvt PTX for FP32->FP4 conversion (#41015) Thien Tran 2026-04-30 06:16:40 +07:00
• a966aaed30 [Bugfix][MLA] Size arange_buffer to max_num_batched_tokens to prevent CUDA IMA (#39277) 3 people 2026-04-30 07:14:50 +08:00
• 6841f5dc77 [ROCm] Add env flags to disable dynamic MXFP4 quant and enable AITER tuned GEMMs for Attention Projection Layers (#39987) Hemanth Acharya 2026-04-30 04:37:46 +05:30
• c2fb013312 [Bugfix][Compile] Fix gc.collect/empty_cache patch arity in CUDAGraphWrapper (#41235) roikoren755 2026-04-30 00:59:18 +03:00
• ccfb620c62 Create tests/distributed/test_mnnvl_alltoall.py (#35241) 3 people 2026-04-29 18:56:56 -03:00
• 0335316a9b [BUG] Two phase pause to prevent deadlock (#39366) 3 people 2026-04-29 14:51:03 -07:00
• 944e138bcf [ROCm][Bugfix]: W4A4 MOE using emulation instead of AITER on MXFP4-supported hardware (#41175) Rohan Potdar 2026-04-29 16:39:03 -05:00
• b58669cb42 [Perf][Spec Decode] Avoid per-step numpy allocation in prepare_next_t… (#41043) Luochao Wangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-29 14:20:13 -07:00
• 1628239eb2 [Multimodal][Render] Skip mm processor initialization and warmup for text-only mode (#41246) Isotr0py 2026-04-30 05:16:19 +08:00
• 93da1fe97a [CI] Add temperature to bfcl eval, default greedy (#41059) yzong-rh 2026-04-29 17:01:57 -04:00
• 169988a3c0 [ROCm] Use quant_dtype in per_token_quant instead of hardcoded FP8 (#39121) Andrew Barnes 2026-04-29 16:46:01 -04:00
• faab189554 [Feature]: IndexCache support for DSA models (#37735) Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-30 03:15:35 +08:00
• 6f20f81cbf Replace shape_invariants with simpler apprach in dynamic_arg_dims utilizing shape_id property. (#36194) Laith Sakka 2026-04-29 11:32:15 -07:00
• d1a75e303d Fix timeout when using LoRA adapters with Nemotron Super (#40916) danisereb 2026-04-29 20:39:49 +03:00
• 4a42aba380 [CI/Build] Enable FP8 on NVIDIA Thor (#39712) Cyrus Leung 2026-04-30 00:48:52 +08:00
• a80d6f150c better logging for large uncachable items (#41145) Avshalom Manevich 2026-04-29 18:48:47 +02:00
• 91a2d39014 [Models] Cohere MoE (#40817) Terrence Zhao 2026-04-29 11:54:54 -04:00
• a05848e255 [Bugfix] Report compile time for in-memory cache hit path (#41023) Frederik Gossen 2026-04-29 11:32:03 -04:00
• 51fda1ba44 [Model Runner v2] Fix block table IMA issue (#40648) Wentao YeandNick Hill 2026-04-29 11:30:33 -04:00
• 39a7f4f4e2 [Perf] Optimize AllPool.forward by slicing first, 51% faster in the method level benchmark (#41163) Wentao Ye 2026-04-29 11:11:04 -04:00
• b92ef9ec5a [Perf] Enable FlashInfer top-k/top-p sampler by default (#40376) Artem Perevedentsev 2026-04-29 18:10:34 +03:00
• 5560cac7e2 [Bugfix][CPU] Backport PT cpp codegen indirect_assert scalar-mask fix (#40973) Lalithnarayan Candmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-29 19:51:55 +05:30
• 5b39b268f5 hf_name argument for vllm bench throughput CLI (#41012) pmaybankandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-29 13:57:58 +01:00
• 22524f7a92 [Feat] CPU fp8 attn for AMX/AVX-512 (#39445) 3 people 2026-04-29 05:43:21 -07:00
• 9d8ad5b408 [Bugfix] Fix repeated DSv4 RoPE cache initialization (#41148) Jee Jee Liandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-04-29 20:29:55 +08:00
• 11b69129e2 [Frontend] Add defer_loading and tool_reference support for Anthropic and OpenAI APIs (#40190) 3 people 2026-04-29 19:35:50 +08:00
• 33f36d4260 [DSV4] Support max reasoning effort (#40982) Bugen Zhao 2026-04-29 19:03:47 +08:00
• 37e288214b [KV Offload] Tighten keys type from Iterable to Sequence in OffloadingManager (#41200) Ronen Schaffer 2026-04-29 13:50:42 +03:00
• 5371d6fb40 Fix PP in Gemma4 (#40786) Rohit Kumar SinghandCyrus Leung 2026-04-29 15:47:51 +05:30
• 6d7d4da99e [Bugfix] BailingMoeV2.5: rotate full qk_rope_head_dim in MLA RoPE (#41185) Jiangyun Zhu 2026-04-29 18:08:55 +08:00
• 3f1a4bb639 build: embed image provenance metadata in vLLM containers (#40653) AlecandOpenAI Codex 2026-04-29 03:07:41 -07:00
• 762022cafb [Bugfix] DSV32/V4 add missing type conversion for non-streaming tool calls (#41198) Chauncey 2026-04-29 17:55:07 +08:00
• 3885d340a4 [Frontend]Responses API supports Tool/Function calling with streaming with named tool/function (#41110) Chauncey 2026-04-29 17:11:27 +08:00
• ef70057ca7 [CI][CPU] Split CPU-Distributed Tests into per-scenario labels (#41203) haosdent 2026-04-29 16:28:45 +08:00
• e48cb85185 [CI/Build] Auto-detect manylinux ABI tag for nightly wheels (#41149) Shengqi ChenandClaude 2026-04-29 15:37:14 +08:00
• 92879e12ba [CI] fix test_rotary_embedding_opcheck format error (#41202) Chauncey 2026-04-29 15:32:37 +08:00
• 68dd7db810 [Reasoning] Support for speculative decoding with thinking budget (#34668) 3 people 2026-04-29 02:14:52 -04:00
• 8a8c9b564e [KV Offload] Per-job store completion for CPU offloading connector (#39186) 4 people 2026-04-29 08:52:55 +03:00
• a269744e9f [Bugfix] Fix rope (#41113) Jee Jee Li 2026-04-29 13:42:35 +08:00
• 8b49cf3a37 [Bugfix] Fix max_num_batched_token not captured in cuda graph (#40734) Wei ZhaoandWei Zhao 2026-04-29 00:33:06 -04:00
• 2ae73c758c [Bugfix] fix inductor error for dpsk v4 (#41135) Jiangyun Zhu 2026-04-29 12:18:46 +08:00
• d95d03c719 [BugFix][CPU] fix error on CPU runner shutdown (#41034) Fadi Arafeh 2026-04-29 05:08:35 +01:00
• 803b9d7881 [Bugfix] Fix Deepseek V4 import error due to AOT compile cache loading (#41090) Wei Zhao 2026-04-29 00:08:16 -04:00
• 1312f07531 [Feature] add cohere reasoning and tool parsers (#40422) Walter Beller-Morales 2026-04-29 00:07:53 -04:00
• fa1b9840f6 [BE][Torch 2.12] Remove workaround code for fixed cublas issue (#40845) Lucas KabelaandWentao Ye 2026-04-28 21:07:24 -07:00