Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 370661856b [Frontend] Update OpenAI error response to upstream format (#22099) Moritz Sanft 2025-08-07 08:06:00 +02:00
• cbc8457b26 [Model] Switch to Fused RMS norm in Qwen2.5_VL model. (#22184) vllmellmandkf 2025-08-07 14:05:24 +08:00
• 4d4297e8fe [Bench] Split serve.py:main into async/async versions (#22405) lkchen 2025-08-06 23:05:07 -07:00
• 2a4c825523 [CI] Skip the pooling models that do not support transformers v4.55 (#22411) wang.yuqi 2025-08-07 14:05:03 +08:00
• 4be02a3776 [Bugfix] EPLB load statistics problem (#22167) WeiQing Chenandycyaw66 2025-08-07 12:07:54 +08:00
• f6278b6243 [gpt-oss] Convert user input to harmony format (#22402) Chen ZhangandWoosuk Kwon 2025-08-06 20:56:02 -07:00
• ad6c655dde preload heavy modules when mp method is forkserver (#22214) Lionel Villard 2025-08-06 23:33:24 -04:00
• 14bcf93a6a Optimize logger init performance by using module-level constants (#22373) ZiTian.Zhao 2025-08-07 11:32:19 +08:00
• ecbea55ca2 Update hf_xet pin to resolve hangs (#22356) Harry Mellor 2025-08-07 04:31:41 +01:00
• 609b533cb6 [Bugfix] Add proper comparison for package versions (#22314) Syed Muhammad Bin Asif 2025-08-07 11:31:03 +08:00
• 5e9455ae8f [Bugfix]: Fix the streaming output for function calls in the minimax (#22015) qscqesze 2025-08-07 11:30:27 +08:00
• a00d8b236f Use float32 for test_completion.py (#22385) Michael Goin 2025-08-06 23:07:47 -04:00
• 04cf435d95 [Bugfix] Fix wrong method name in Intern-S1 image processor (#22417) Cyrus Leung 2025-08-07 11:05:20 +08:00
• 7377131a2c [Qwen3] Enable dual-chunk-attention support for Qwen3 models. (#21924) Tao He 2025-08-07 10:58:08 +08:00
• 6b47ef24de [XPU]Fix flash_attn_varlen_func interface on xpu (#22350) Kunshang Ji 2025-08-07 10:28:11 +08:00
• 1dc8a70b6d [Attention] Support multiple attention metadata builders per kv_cache_spec + proper local attention no hybrid kv cache fix (#21588) Lucas Wilkinson 2025-08-06 21:40:52 -04:00
• f825c6bd22 Support encoder_only attention for FlexAttention (#22273) Maximilien de Bayser 2025-08-06 22:37:14 -03:00
• 41b67f4263 [model] Support MiniCPM-V 4.0 (#22166) tc-mbandimning3 2025-08-07 09:35:46 +08:00
• e8961e963a Update flashinfer-python==0.2.10 (#22389) Michael Goin 2025-08-06 21:10:24 -04:00
• 9a3835aaa9 Fix trtllm-gen attention env and add attention sink (#22378) 4 people 2025-08-06 18:07:41 -07:00
• 5c7cc33f4d [gpt-oss] fix model config with hf_config (#22401) Yongye Zhu 2025-08-06 18:04:04 -07:00
• 19c9365aa4 [gpt-oss] add demo tool server (#22393) Chen Zhang 2025-08-06 17:47:14 -07:00
• eec890c1c1 [Bug] Fix B200 DeepGEMM E8M0 Accuracy Issue (#22399) Wentao Ye 2025-08-06 20:03:53 -04:00
• 46a13949d5 [v1] - Mamba1 Attention Metadata (#21249) Asaf Joseph Gardinandasafg 2025-08-07 03:03:42 +03:00
• 31f09c615f [gpt-oss] flashinfer mxfp4 (#22339) Yongye Zhuandsimon-mo 2025-08-06 12:37:27 -07:00
• 31f5dc5b2a [gpt-oss] Enhance error msg on attention sink init (#22335) Yongye Zhuandsimon-mo 2025-08-06 11:41:42 -07:00
• ec7cb19224 [gpt-oss] Add loop for built-in tool call (#22374) 7 people 2025-08-06 10:32:21 -07:00
• 2435ea7ed5 [Bugfix] Make condition in triton kernel constexpr (#22370) Gregory Shtrasberg 2025-08-06 13:00:58 -04:00
• 4a6b72c2ab [BugFix] Fix triton compile error in kernel_unified_attention_2/3d caused by attention sinks (#22368) Lucas Wilkinson 2025-08-06 12:47:38 -04:00
• b4b9813b5e add the codes to check AMD Instinct GPU number (#22367) Zhang Jason 2025-08-06 23:58:38 +08:00
• 2cb6ef8996 [BugFix] Fix FA2 RuntimeError when sinks is provided (#22365) Lucas Wilkinson 2025-08-06 11:03:03 -04:00
• 9edd1db02b [Minor] Fix type (#22347) Woosuk Kwon 2025-08-06 02:22:03 -07:00
• f263a4b53f [gpt-oss] Support chat completion api (#22342) Woosuk Kwon 2025-08-06 01:57:39 -07:00
• 54991c548a [gpt-oss] add model to supported models doc (#22336) Roger Wang 2025-08-06 01:49:44 -07:00
• 178d03fbd6 [gpt-oss] Add Tool/ConversationContext classes and harmony_utils (#22340) 7 people 2025-08-06 01:08:49 -07:00
• fa00c5d75b [Misc] Clean up duplicated hf overrides (#22311) Isotr0py 2025-08-06 15:50:25 +08:00
• 134a8ee8fd [gpt-oss] Add openai-harmony as default dependency (#22332) 7 people 2025-08-06 00:10:14 -07:00
• 90ec006937 [gpt-oss] flashinfer attention sink init (#22330) 7 people 2025-08-05 23:48:19 -07:00
• a47e6ffe93 [GptOss] Add GptOss reasoning parser to support structure output (#22322) 7 people 2025-08-05 23:39:13 -07:00
• 98a3a81024 [ROCm] Add attention sink to use_rocm_custom_paged_attention (#22329) 7 people 2025-08-05 23:30:38 -07:00
• de98252f49 Add GPT-OSS model code and config [1/N] (#22327) Woosuk Kwon 2025-08-05 23:26:00 -07:00
• 796bae07c5 Update transformers to v4.55 (#21931) 5 people 2025-08-06 06:56:14 +01:00
• 6e20924350 Add attention sink in attention backends (#22320) 7 people 2025-08-05 22:37:21 -07:00
• dd16bdc798 Increase openai-python version (#22316) Woosuk Kwon 2025-08-05 21:43:21 -07:00
• e3c876dca3 Upgrade FA3 for attention sink (#22313) Woosuk Kwon 2025-08-05 21:36:21 -07:00
• 5d5d419ca6 [Bugfix][CI/Build][ROCm] Make sure to use the headers from the build folder on ROCm (#22264) Gregory Shtrasberg 2025-08-05 23:39:32 -04:00
• 302962e806 [Bugfix] Skip dead and non-GPU nodes for Ray DP engine allocation (#22275) Rui Qiao 2025-08-05 20:35:32 -07:00
• 7e6544c797 [Perf] Parallelize fill_bitmask to accelerate high-throughput guided decoding (#21862) Benjamin Chislett 2025-08-05 22:57:49 -04:00
• 8e6c7e873f [Bugfix] Fix MoE BNB version (#22260) Jee Jee Li 2025-08-06 10:56:22 +08:00
• 6a51530437 [Bugfix] Fix 3D input passed into cutlass_scaled_mm (#22278) Michael Goin 2025-08-05 22:35:20 -04:00
• 35509fc5be [Bugfix] Remove faulty test for oot attention backend (#22286) Michael Goin 2025-08-05 20:05:40 -04:00
• 4b29d2784b [CI][TPU] Fix docker clean up (#22271) Siyuan Liu 2025-08-05 16:54:56 -07:00
• 59a0b8554b [bugfix] fix blackwell deepep installation (#22255) youkaichao 2025-08-06 01:26:09 +08:00
• 469b3ffaaa [V1] port xformers backend to v1 (#21342) Giancarlo Delfin 2025-08-05 10:04:46 -07:00
• ae87ddd040 [Refactor] Remove Unused Environment Variable VLLM_NO_DEPRECATION_WARNING (#22199) Wentao Ye 2025-08-05 12:40:23 -04:00
• a7cb6101ca [CI/Build] Update flashinfer to 0.2.9 (#22233) Michael Goin 2025-08-05 12:39:38 -04:00
• c494f96fbc Use UV_LINK_MODE=copy in Dockerfile to avoid hardlink fail (#22128) Michael Goin 2025-08-05 09:57:10 -04:00
• 0c275ad5ad [V0 Deprecation][TPU] Remove V1 flag check from tests (#22248) Nicolò Lucchesi 2025-08-05 15:53:23 +02:00
• 74333ae2f6 [Misc] correct static type check for GroupCoordinator (#21946) Ning Xie 2025-08-05 18:17:46 +08:00
• 83156c7b89 [NVIDIA] Support Flashinfer TRT-LLM Prefill Attention Kernel (#22095) elvischenv 2025-08-05 17:45:34 +08:00
• 4771df7b2b [Feature] Non-contiguous Support for FP8 Quantization (#21961) Wentao Yeandmgoin 2025-08-05 05:36:43 -04:00
• 05fae02175 Migrate KimiVLImagePixelInputs to TensorSchema (#21769) Benji BeckandIsotr0py 2025-08-05 02:36:18 -07:00
• d1bf1b9711 [Docs][TPU] Highlight TPU Software version selection (#22242) Nicolò Lucchesi 2025-08-05 11:33:46 +02:00
• 586f286789 [Model] Pooling model activation supports per request control by PoolingParams (#20538) wang.yuqi 2025-08-05 15:37:00 +08:00
• 811ac13d03 [Core] Factor out common logic for MM budget calculation (#22228) Cyrus Leung 2025-08-05 14:54:55 +08:00
• e79a12fc3a [UX] Fail if an invalid attention backend is specified (#22217) Michael Goin 2025-08-05 02:54:52 -04:00
• cdfd6871a5 [Bugfix] Misaligned params in TreeAttentionImpl (#22226) Cyrus Leung 2025-08-05 13:40:09 +08:00
• 4b3e4474d7 Optimize configuration access with LRU cache in custom ops (#22204) ZiTian.Zhao 2025-08-05 12:43:24 +08:00
• bd3db7f469 [Misc] log more detailed message for ensure_model_parallel_initialized (#22144) Ning Xie 2025-08-05 10:36:55 +08:00
• 29b97c0995 [Doc] add backend to doc string of initialize_model_parallel (#22142) Ning Xie 2025-08-05 10:36:20 +08:00
• 7b455cf1c0 [Misc] Remove pass_config from CompilationConfig dump_json excluded (#21911) elvischenv 2025-08-05 10:17:18 +08:00
• 8a6e108e76 fix: kimi_k2 return empty tool call list (#22149) tlipoca9 2025-08-05 10:15:31 +08:00
• d7b28f3415 [Log] DeepGEMM Update Log for Unaligned Problem Size (#22208) Wentao Ye 2025-08-04 22:13:19 -04:00
• 6fa41e0c32 self.gate dtype update for GLM-4.5 (#22203) Yuxuan Zhang 2025-08-05 10:12:38 +08:00
• 031ca762d7 [ROCm][Bugfix] Compilation passes fix (#22202) Gregory Shtrasberg 2025-08-04 22:12:28 -04:00
• 6ad6b8e115 [FEAT] Refactor ROPE into module (#22192) TJian 2025-08-04 19:12:16 -07:00
• f4f4e7ef27 [V0 deprecation][P/D] Deprecate v0 KVConnectorBase code (1/2) (#21785) lkchen 2025-08-04 19:11:33 -07:00
• 5ea71ff46f [V1] reduce block size for tree attention correctness test to fix 'ou… (#22207) Giancarlo Delfin 2025-08-04 19:11:06 -07:00
• 7175817637 Revert "[Bugfix] V1 Fix the cursor leakage issue during request scheduling." (#22223) Woosuk Kwon 2025-08-04 18:37:06 -07:00
• 2dffac464c [Bugfix] V1 Fix the cursor leakage issue during request scheduling. (#21173) PiteXChen 2025-08-05 09:34:10 +08:00
• bdcb42e45d [NVIDIA] Auto detect modelopt quant and fix DSR1-FP4 weight loading (#22073) Po-Han Huang (NVIDIA) 2025-08-05 09:02:55 +08:00
• c09efff976 [Bugfix][V1][P/D]Fix the uneven polling issue in the toy proxy for P2pNcclConnector (#21819) Zhonghua Deng 2025-08-05 04:17:05 +08:00
• 309c1bb822 [Bug] Update auto_tune.sh to separate benchmarking and profiling. (#21629) ericehanley 2025-08-04 10:12:06 -05:00
• 9af654cc38 [Responses API] Ignore store=True and process the request by default (#22185) Woosuk Kwon 2025-08-04 05:12:48 -07:00
• a5fff3bd49 Fix Arcee model weight loading: Add custom load_weights (#21725) Raghav Ravishankar 2025-08-04 16:39:56 +05:30
• 1539ced93a [Doc] Update pooling model docs (#22186) Cyrus Leung 2025-08-04 18:37:06 +08:00
• 54de71d0df [Sampler] Support returning all logprobs or logits (#21792) 22quinn 2025-08-04 03:04:12 -07:00
• fed5849d3f [Bugfix] Fix failing GGUF models test (#22174) Isotr0py 2025-08-04 16:27:02 +08:00
• c1b4eb048a [feat] move WEIGHT_SCALE_SUPPORTED into raise block to accelerate RLHF weight loading (#21164) Weixiao Huang 2025-08-04 15:43:06 +08:00
• a7b8788d2c [Misc] Modify the organization of GLM series (#22171) Jee Jee Li 2025-08-04 14:51:20 +08:00
• 8ecb3e9e93 [CI Bugfix] Fix wNa16 kernel not found for test_shared_storage_connector_hashes (#22163) Tyler Michael Smith 2025-08-04 01:19:04 -04:00
• e5949e5ae0 Remove index_put from MM embeddings merging (#22105) Chenxi YangandChenxi Yang 2025-08-03 22:15:14 -07:00
• 49bcd893e7 [refactor] improve ConstantList exception specificity (#22156) ZiTian.Zhao 2025-08-04 13:14:49 +08:00
• aa7012eb6d Add tree attention backend for v1 (part 1) (#20401) Giancarlo Delfin 2025-08-03 22:13:26 -07:00
• c2e75b3c11 remove duplicate code within cleanup_dist_env_and_memory (#22147) Ning Xie 2025-08-04 11:03:58 +08:00
• 0d7db16a92 [PD] add test for chat completions endpoint (#21925) Abirdcfly 2025-08-04 10:57:03 +08:00
• 845420ac2c [RLHF] Fix torch.dtype not serializable in example (#22158) 22quinn 2025-08-03 19:43:33 -07:00
• e27d25a0dc [fix] fix correct assertion syntax error in attention utils. (#22154) ZiTian.Zhao 2025-08-04 10:24:02 +08:00
• 6f5478298d Use aiohttp connection pool for benchmarking (#21981) Seiji Eicher 2025-08-03 19:23:32 -07:00
• 6a39ba85fe [Bugfix] Fix failing multimodal standard test (#22153) Isotr0py 2025-08-04 03:04:38 +08:00