Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• c1c8ca57ff [cold start time] add envs.VLLM_COMPILE_DEPYF to guard decompile (#20790) Boyuan Feng 2025-07-11 23:06:13 -07:00
• a3a5a47e48 [Bugfix] Fix torch.compile x LoRA for PyTorch 2.8 (#20823) Richard Zou 2025-07-12 02:06:04 -04:00
• fb25e95688 [Docs] Update basic.md (#20846) Lucia Fang 2025-07-12 14:05:32 +08:00
• 0d4891cd03 [Bug] Fix DeepGemm for EP low latency case (#20833) Wentao Ye 2025-07-12 02:05:12 -04:00
• f56d2996ca [Misc] Respect no_use_tqdm_on_load flag while capturing CUDA graph (#20834) lkchen 2025-07-11 23:04:45 -07:00
• 147afb448b [Bugfix] Replace unavailable video url in multimodal test (#20854) Isotr0py 2025-07-12 13:25:39 +08:00
• 3c7d942da8 [Frontend] Abstract prompt and SpeechToTextConfig for transcriptions models (#20637) Nicolò Lucchesi 2025-07-12 06:33:26 +02:00
• 890323dc1b [Bugfix] : Fix typo - logger.warn_once -> logger.warning_once (#20852) Varun Sundar Rabindranath 2025-07-12 07:56:24 +04:00
• 01cae37713 [CI/Build] Ensure compatability with Transformers v4.53 (#20541) Isotr0py 2025-07-12 11:53:07 +08:00
• 11c0198615 [Bugfix] Fix tensor parallel issue in Qwen3 reranker weight loading (#20682) yurhettandIsotr0py 2025-07-12 11:52:43 +08:00
• b1235c3e10 [Bugfix] Lazy import fused_experts in BitsAndBytesMoEMethod to avoid break not-cuda-alike devices (#20822) Li, Jiang 2025-07-12 11:52:05 +08:00
• 44d02f54db [Misc] Restrict deep_gemm's log output (#20827) Jee Jee Li 2025-07-12 11:50:42 +08:00
• a8593237c0 Add pynccl all-gatherv and reducescatterv (#20154) Trevor Morrisandmgoin 2025-07-11 18:59:23 -07:00
• fc0f41d10a Integration SM100 FlashInfer fused allreduce RMSNorm (#20691) Ilya Markovandilmarkov 2025-07-12 03:58:15 +02:00
• 7b828e30d5 [CI Bug] Fix Async Engine, Inputs, Utils, Worker Test: 'State' object has no attribute 'enable_server_load_tracking' (#20845) Wentao Ye 2025-07-11 21:57:24 -04:00
• 5f0af36af5 Update kimi-k2 tool calling docs, enable unit tests (#20821) 3 people 2025-07-12 04:16:14 +08:00
• 0d21b2664c [Bugfix] Fix OOM in language generation test (#20814) Isotr0pyandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-07-12 02:21:52 +08:00
• 9907fc4494 [Docs] Data Parallel deployment documentation (#20768) Nick Hill 2025-07-11 17:42:10 +01:00
• d47661f0cd [Kernel] Basic tuned configs for NVFP4 CUTLASS dense GEMM (#20646) Michael Goin 2025-07-12 01:05:33 +09:00
• 53fa457391 [Misc] Add unit tests for MoE ModularKernel combinations + Profiling utility (#20449) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-07-11 10:51:46 -04:00
• 6fb162447b [doc] fix ordered list issue (#20819) Reid 2025-07-11 21:49:46 +08:00
• 66177189c5 [Bugfix] Add missing field to TritonLanguagePlaceholder (#20812) Li, Jiang 2025-07-11 20:25:11 +08:00
• b4f0b5f9aa Temporarily suspend google/gemma-3-1b-it. (#20722) QiliangCui 2025-07-11 04:21:26 -07:00
• cbd14ed561 [Bugfix] Refactor /invocations to be task-agnostic (#20764) Cyrus Leung 2025-07-11 18:20:54 +08:00
• 7bd4c37ae7 [Core] Add Flashinfer TRTLLM Backend for Flashinfer decode path (SM100). (#19825) 3 people 2025-07-11 02:23:23 -07:00
• 8020e98c9f [Quantization][1/N] MoE support BNB-Inflight Quantization (#20061) Jee Jee Li 2025-07-11 16:01:13 +08:00
• 762be26a8e [Bugfix] Upgrade depyf to 0.19 and streamline custom pass logging (#20777) Luka Govedič 2025-07-11 03:15:22 -04:00
• 6a9e6b2abf [doc] fold long code block (#20795) Reid 2025-07-11 14:16:41 +08:00
• 5d09152ff1 [V1] Enable Mamba2 layers other than MambaMixer2 in the v1 engine (#20660) nopperl 2025-07-11 14:53:31 +09:00
• 31d5c1797f [Perf][fp8] Use CustomOp abstraction for fp8 quant for better perf (#19830) Luka Govedičandmgoin 2025-07-11 00:56:28 -04:00
• 35514b682a [XPU] XCCL support enabled in torch 2.8.0.dev nightly builds (#20705) Ratnam Parikh 2025-07-10 20:39:52 -07:00
• e2de455c34 [Feature] Integrate SM100 DeepGEMM support (#20087) Wentao Ye 2025-07-10 23:18:05 -04:00
• 5b032352cc [Attention] MLA - Flashinfer Ragged Prefill (#20034) Alexander Matveev 2025-07-10 23:17:47 -04:00
• 922f316441 [Model] Support HF format of minimax (#20211) Michael Goin 2025-07-11 11:55:21 +09:00
• 5923ab9524 [fix]: disable cutlass block scaled group gemm for EP (#20781) Duncan Moss 2025-07-10 19:39:18 -07:00
• 0cf893cae1 Add kimi-k2 tool parser (#20789) 3 people 2025-07-11 10:36:23 +08:00
• cf75cd2098 [CI Bugfix] Specify same TORCH_CUDA_ARCH_LIST for flashinfer aot and install (#20772) Michael Goin 2025-07-11 10:16:01 +09:00
• b854321ffe [Docs] Lazy import gguf (#20785) Simon Mo 2025-07-10 16:06:37 -07:00
• 5b6fe23d05 [Bugfix][Benchmark] Make sure the output length > 0 when testing prefill workload. (#20786) Kuntai Duandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-07-10 14:52:46 -07:00
• f0c98cae27 [Misc] MoE ModularKernel : Introduce TopKWeightAndReduce (#20648) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-07-10 17:40:38 -04:00
• 574ad60db9 [KVConnector] Always call connector clear_metadata() at end of step (#20756) Nick HillandDavid Ben-David 2025-07-10 22:37:27 +01:00
• fdadb6f43a [Bugfix] Fused MoE Modular Kernel chunking loop (#20392) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-07-10 16:31:10 -04:00
• 41060c6e08 [Core] Add Support for Default Modality Specific LoRAs [generate / chat completions] (#19126) Alex Brooks 2025-07-10 14:09:37 -06:00
• 3de2ed767f [Bugfix] Remove assertion of expert_map being None (#20714) Ming Yang 2025-07-10 12:55:22 -07:00
• 299252ea82 [CI] Fix pre commit issue (#20782) Wentao Ye 2025-07-10 15:48:13 -04:00
• d6902ce79f [V0][V1][Core] Add outlines integration for V1, and update V0 integration. (#15975) Nathan Hoos 2025-07-10 14:30:26 -05:00
• 5e53c89a74 [Bugfix] [CI] Fix Tensorizer LoRA test (#20760) Sanger Steel 2025-07-10 15:07:06 -04:00
• c66e38ea4c [Test] Remove docker build from test. (#20542) QiliangCui 2025-07-10 11:21:58 -07:00
• 251595368f Fix DeepSeek-R1-0528 chat template (#20717) sfbemerkandBenjamin Merkel 2025-07-10 19:47:36 +02:00
• 4bed167768 [Model][VLM] Support JinaVL Reranker (#20260) shineran96 2025-07-11 01:43:43 +08:00
• b140416abf [Model] Add reason parser for Hunyuan A13B Model. (#20625) Asher 2025-07-11 00:33:26 +08:00
• 5b8366b61a [ROCm][Regression] Remove tensor creation that harms performance on ROCm (#20741) Gregory Shtrasberg 2025-07-10 12:22:23 -04:00
• c7753a9809 [Hardware][CPU] Vllm int8 quantization enablement for ARM CPU (#14129) nishith-fujitsu 2025-07-10 21:29:04 +05:30
• 4b9a9435bb Update Dockerfile FlashInfer to v0.2.8rc1 (#20718) Michael Goin 2025-07-11 00:09:02 +09:00
• 3482fd7e4e [Doc] Add engine args back in to the docs (#20674) Harry Mellor 2025-07-10 16:02:40 +01:00
• 77f77a951e [Misc] Clean up mark to fork process in BNB tests (#20692) Isotr0py 2025-07-10 21:59:40 +08:00
• 1a4f35e2ea Normalize lm-eval command between baseline and correctness test (#18560) Michael Goin 2025-07-10 22:27:32 +09:00
• be1e128dfb [CI Bugfix] Skip failing Tensorizer+LoRA test (#20724) Michael Goin 2025-07-10 21:15:03 +09:00
• 65393ee064 [doc] fix ordered list (#20749) Reid 2025-07-10 18:13:52 +08:00
• dc221ad72d [Bugfix][Build][Non-CUDA] Only referencing CMAKE_CUDA_COMPILER_VERSION on CUDA where it is defined (#20738) Gregory Shtrasberg 2025-07-10 05:58:11 -04:00
• 7571a4a7e5 [CI/Build] Fix Basic Models Test (#20728) Jee Jee Li 2025-07-10 17:57:19 +08:00
• f67d986dd1 [Misc] loose new-model tagger conditions (#20747) Isotr0py 2025-07-10 17:54:47 +08:00
• cc876d0f29 [KVConnector] Aggregate finished requests on the scheduler (#19555) Or Ozeri 2025-07-10 11:22:18 +03:00
• fdfd409f8f [TPU][Core]Make load weight exceed hbm error more instructive for customers (#20644) Chenyaaang 2025-07-10 00:01:17 -07:00
• ffbcc9e757 [BugFix] Fix VllmConfig() construction on all platforms (#20695) Nick Hill 2025-07-10 08:00:20 +01:00
• 59389c927b [BugFix][CPU] Fix CPU worker dependency on cumem_allocator (#20696) Nick Hill 2025-07-10 07:24:20 +01:00
• 8f2720def9 [Frontend] Support Tool Calling with both tool_choice='required' and $defs. (#20629) Chauncey 2025-07-10 13:56:35 +08:00
• ad6c2e1a0b Correct PPMissingLayer handling in Deepseek-V2-Lite PP deployment (#20665) Seiji Eicher 2025-07-09 20:34:40 -07:00
• 49e8c7ea25 Use NVCC --compress-mode to reduce binary size by 30% (#20694) Michael Goin 2025-07-10 10:26:48 +09:00
• 805d62ca88 [Misc] DP : Add ExpertTokensMetadata (#20332) Varun Sundar RabindranathandVarun 2025-07-09 20:33:14 -04:00
• b7d9e9416f [CI/Build] Fix FlashInfer double build in Dockerfile (#20651) Michael Goin 2025-07-10 08:41:56 +09:00
• 7c12a765aa [Misc] Simplify the prefix caching logic on draft tokens (#20701) Woosuk Kwon 2025-07-09 14:48:35 -07:00
• cd587c93ef [BugFix]: Properly set engine_id when using multi connector (#19487) 3 people 2025-07-10 04:32:44 +08:00
• 332d4cb17b [Feature][Quantization] MXFP4 support for MOE models (#17888) fxmarty-amdandBowen Bao 2025-07-09 22:19:02 +02:00
• bf03ff3575 [Kernel] Add Conch backend for mixed-precision linear layer (#19818) Jacob Manning 2025-07-09 16:17:55 -04:00
• 47043eb678 [Kernel] Triton implementation of causal-conv1d for Mamba-based models (#18218) 4 people 2025-07-09 15:53:55 -04:00
• 31b96d1c64 Support Llama 4 for cutlass_moe_fp4 (#20453) Michael Goin 2025-07-10 04:53:38 +09:00
• e59ba9e142 [CI/Build] Enlarge tolerance for a CPU multi-modal test (#20684) Li, Jiang 2025-07-10 01:48:52 +08:00
• 403b481573 Remove heading form installation inc.md file (#20697) Harry Mellor 2025-07-09 18:42:51 +01:00
• 138709f8d1 [Doc] Update CPU doc (#20676) 3 people 2025-07-10 01:28:30 +08:00
• 0bbac1c1b4 [Bench] Add NVFP4 GEMM benchmark script (#20578) Michael Goin 2025-07-10 02:23:48 +09:00
• a3e4e85ece [XPU][CI] enhance xpu test support (#20652) Liangliang Maandzhenwei-intel 2025-07-10 00:53:09 +08:00
• eb58f5953d [TPU][Bugfix] fix test_pallas (#20666) Chengji Yao 2025-07-09 09:32:48 -07:00
• 4ac9c33f78 [Bugfix] Fix handling of Tensorizer arguments for LoadConfig (#20643) Sanger Steel 2025-07-09 11:36:37 -04:00
• efe73d0575 [doc] update doc format (#20673) Reid 2025-07-09 23:08:19 +08:00
• 853487bc1b [Docs] Improve docs for RLHF co-location example (#20599) Ricardo DecalandHarry Mellor 2025-07-09 08:06:43 -07:00
• 9ff2af6d2b [Benchmark] Parameterization of streaming loading of multimodal datasets (#20528) Li Wang 2025-07-09 21:35:16 +08:00
• 70ca5484f5 [Doc] Update notes (#20668) Cyrus Leung 2025-07-09 18:46:36 +08:00
• 5358cce5ff [V1] [Doc] Update V1 docs for Mamba models (#20499) Thomas ParnellandCyrus Leung 2025-07-09 10:02:41 +02:00
• 2155e95ef1 [Bugfix] Fix the issue where reasoning_content is None when Thinkng is enabled and tool_choice is set to 'required'. (#20662) Chauncey 2025-07-09 15:39:58 +08:00
• f95570a52d [Docs] fix minimax tool_calling docs error (#20667) qscqesze 2025-07-09 15:37:07 +08:00
• b6e7e3d58f [Intel GPU] support ray as distributed executor backend for XPU. (#20659) Kunshang Ji 2025-07-09 15:36:58 +08:00
• e760fcef22 [XPU] Use spawn with XPU multiprocessing (#20649) Dmitry Rogozhkin 2025-07-09 00:34:28 -07:00
• 6bbf1795b7 [Misc] Fix the size of batched_dummy_mm_inputs in profile_run (#20434) B-201 2025-07-09 11:15:44 +08:00
• 9e0ef888f0 Fix bullets in incremental_build.md (#20642) Michael Goin 2025-07-09 12:03:41 +09:00
• 97abeb1daa [feat] enable SM100 CUTLASS block scaled group gemm for smaller batch sizes (#20640) Duncan Moss 2025-07-08 20:03:35 -07:00
• 34dad19e7b [Bugfix] set default set cuda_graph_sizes to min(self.max_num_seqs * 2, 512) (#20628) zhrrr 2025-07-09 11:02:51 +08:00
• 6db31e7a27 [Hardware][PPC64LE] Enable V1 for ppc64le and ARM (#20554) 3 people 2025-07-09 08:30:41 +05:30
• 977180c912 [Docs] Improve documentation for multi-node service helper script (#20600) Ricardo Decal 2025-07-08 19:44:26 -07:00
• c40784c794 [BugFix][Intel GPU] Use refactored API for dist_backend in V1 worker (#20596) Ratnam Parikh 2025-07-08 19:44:23 -07:00