Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• d3c18c9cb0 fuse fp32 for GLM-4.5 e_score_correction_bias (#22143) Yuxuan Zhang 2025-08-04 00:04:54 +08:00
• 83f7bbb318 Add chat doc in quick start (#21213) TankNeeandCyrus Leung 2025-08-03 22:47:55 +08:00
• b5dfb94fa0 [CI/Build][Bugfix] Fix Qwen2.5 tests in CPU CI via fallback silu_and_mul to torch native implementation (#22145) Li, Jiang 2025-08-03 20:34:04 +08:00
• 6d98843b31 [Responses API] Disable response store by default (#22137) Woosuk Kwon 2025-08-03 04:04:21 -07:00
• aefeea0fde [V1] [P/D] Refactor KV Connector Path (#21980) David Ben-DavidandDavid Ben-David 2025-08-03 14:03:40 +03:00
• 24d1dffbeb [executor] feat: add supports_pp attr to executors (#21786) H 2025-08-03 03:04:45 -07:00
• 7de45db9a5 [Misc] update doc comment for send (#22026) Ning Xie 2025-08-03 15:55:20 +08:00
• 789562c28c Support CUTLASS NVFP4 (w4a4) for Blackwell Geforce GPUs (SM120) (#21309) Roberto L. Castro 2025-08-03 09:54:22 +02:00
• 3f36c325fa [Benchmark] Support ready check timeout in vllm bench serve (#21696) Ye (Charlotte) QiandRoger Wang 2025-08-03 00:52:38 -07:00
• 3dddbf1f25 [Misc] Add tensor schema test coverage for multimodal models (#21754) Isotr0py 2025-08-03 15:52:14 +08:00
• 337eb23bcc [Fix] Fix llama4 modelopt weight loading error (#22107) jiahancandmgoin 2025-08-03 00:50:34 -07:00
• 2ff46b8826 [Misc] Bump ray to 2.48.0 (#22123) Rui Qiao 2025-08-02 19:42:00 -07:00
• 554df8a6a2 Revert "[compile][startup] Disable C++ compilation of symbolic shapes" (#22122) Xiao 2025-08-02 09:03:30 -07:00
• 73e1b9b1d4 [xpu]support moe models on XPU platform (#21643) Yan Ma 2025-08-02 22:49:08 +08:00
• 4abfd8796f [V1] [Hybrid] Validate compatibility of attention backend batch reordering at init time (#21557) Thomas Parnell 2025-08-02 14:29:40 +02:00
• f5d0f4784f [Frontend] Improve error message for too many mm items (#22114) Cyrus Leung 2025-08-02 17:20:38 +08:00
• b690e34824 [Model] Mamba2 preallocate SSM output tensor to avoid d2d copy overhead (#21075) Chih-Chieh Yang 2025-08-02 04:59:34 -04:00
• 25373b6c6c for glm-4.1V update (#22000) Yuxuan ZhangandIsotr0py 2025-08-02 16:46:57 +08:00
• 58eee5f2e0 [PERF] Use faster way of decode in tokenizer: avoid useless list-to-list conversion (#20000) Vadim Gimpelson 2025-08-02 12:43:52 +04:00
• 067c34a155 docs: remove deprecated disable-log-requests flag (#22113) Roger Wang 2025-08-02 00:19:48 -07:00
• c64861d63c [Bugfix] Mamba2 remove bugged initial state condition in chunk scan (#22034) Chih-Chieh Yang 2025-08-02 02:55:57 -04:00
• 8564dc9448 Fix test_kv_sharing_fast_prefill flakiness (#22038) Yong Hoon Shin 2025-08-01 23:55:34 -07:00
• 4ac8437352 [Misc] Getting and passing ray runtime_env to workers (#22040) Rui Qiao 2025-08-01 23:54:40 -07:00
• d3a6f2120b [FEAT][ROCm] Enable running Flash Attention as ViT attn backend for Qwen-VL models on ROCm platform. (#22069) vllmellmandtjtanaavllm 2025-08-02 14:53:18 +08:00
• 0edaf752d7 [Attention][DBO] Add support for "splitting" the CommonAttentionMetadata (#21153) Sage Moore 2025-08-01 19:47:53 -07:00
• 6e8d8c4afb [Test] Add Unit Test for Batched DeepGEMM (#21559) Wentao Ye 2025-08-01 22:45:46 -04:00
• 8d524ce79f [BugFix] Improve internal DP load balancing (#21617) Nick Hill 2025-08-02 03:45:27 +01:00
• 9f9c38c392 [Speculators][Speculative Decoding] Add Qwen Eagle3 Support (#21835) Dipika Sikka 2025-08-01 22:43:37 -04:00
• a65f46be5e [Misc] DeepGemmExperts : Avoid JIT generation in the hot-path (#21955) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-08-02 08:12:03 +05:30
• 57393715e8 [Misc] VLLM_TARGET_DEVICE.lower() (#22101) Nicolò Lucchesi 2025-08-02 04:41:40 +02:00
• ee2eb6ecd8 [Model] Qwen2.5 VL SiLU-and-Mul (#22066) vllmellmandkf 2025-08-02 10:34:37 +08:00
• 23322431c8 [V1][CUDA] Full cudagraph support for FlashInfer (#21367) fhl2000 2025-08-02 09:49:34 +08:00
• 3654847db5 feat: Add Support GPTQ Quantization MOE on ROCM vllm serve (#21733) JartX 2025-08-02 03:12:19 +02:00
• eefbf4a68b [Perf] Optimize reshape_and_cache_flash CUDA Kernel (#22036) Wentao Ye 2025-08-01 19:18:51 -04:00
• 88faa466d7 [CI] Initial tests for SM100 Blackwell runner (#21877) Michael Goin 2025-08-01 19:18:38 -04:00
• 881e1af43a [BugFix] Harden distributed DP startup (#21538) Nick Hill 2025-08-01 22:40:45 +01:00
• d84b97a3e3 Add lora test for tp>1 case for TPU. (#21970) XiongfeiWei 2025-08-01 11:56:08 -07:00
• d331759488 Introduce RayPPCommunicator for ray-based PP (#21660) Rui Qiao 2025-08-01 11:50:58 -07:00
• 9659bc7f27 [compile][startup] Disable C++ compilation of symbolic shapes (#20836) Animesh Jain 2025-08-01 10:38:52 -07:00
• 3277e8f9e1 Fix pre-commit failure for SECURTIY.md (#22102) Michael Goin 2025-08-01 13:36:07 -04:00
• 8d705996df [Misc] Minor enhancement of benchmark_moe (#22068) Jee Jee Li 2025-08-02 01:35:30 +08:00
• 38c8bce8b6 Enable headless models for pooling in the Transformers backend (#21767) Harry Mellor 2025-08-01 18:31:29 +01:00
• ac45c44d98 [Bugfix] [Performance] DeepEPHighThroughput + DeepSeek : Quant before Dispatch (#21837) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-08-01 22:44:38 +05:30
• d6664664b4 security policy: take 1 (#21119) 3 people 2025-08-01 21:09:49 +04:00
• b879ecd6e2 [Bugfix] fix when skip tokenizer init (#21922) rongfu.leng 2025-08-02 01:09:36 +08:00
• 3f8e952179 [Bugfix] Fix glm4.1v video inference issue (#22067) Isotr0py 2025-08-02 00:33:30 +08:00
• 326a1b001d Improve documentation of ModelConfig.try_get_generation_config to prevent future confusion (#21526) Harry Mellor 2025-08-01 17:32:27 +01:00
• 2d7b09b998 Deprecate --disable-log-requests and replace with --enable-log-requests (#21739) Harry Mellor 2025-08-01 17:16:37 +01:00
• 97608dc276 [Docs] use uv in CPU installation docs (#22089) David Xia 2025-08-01 10:55:55 -04:00
• 3146519add [BugFix] Don't change title of top-level process (#22032) Nick Hill 2025-08-01 15:37:55 +01:00
• 8026a335a1 [BugFix] Update AttnFusionPass cache key (#21947) Richard Zou 2025-08-01 10:11:29 -04:00
• a59cd9d9f7 [Refactor] Fix Compile Warning #1444-D (#21462) Wentao Ye 2025-08-01 09:10:30 -04:00
• 5c54d9759d [Bugfix][PD] set max_completion_tokens=1 if req has this value (#21841) Abirdcfly 2025-08-01 21:08:45 +08:00
• 0a6d305e0f feat(multimodal): Add customizable background color for RGBA to RGB conversion (#22052) GamhangandJinheng Li 2025-08-01 21:07:33 +08:00
• f81c1bb055 [Bugfix] Check NVIDIA artifactory is accessible before using flashinfer cubin kernels (#21893) Michael Goin 2025-08-01 08:28:45 -04:00
• fb0e0d46fc Fix get_kwargs for case where type hint is list[Union[str, type]] (#22016) Harry Mellor 2025-08-01 13:26:42 +01:00
• 26b5f7bd2a [BUG] [ROCm] Fix import bug on ROCm (#22083) TJian 2025-08-01 05:25:20 -07:00
• dfbc1f8880 [Speculative Decoding] Add speculators config support (#21345) Dipika Sikka 2025-08-01 08:25:18 -04:00
• 87c94bc879 Revert "Update sampling_metadata.py (#21937)" (#22088) Harry Mellor 2025-08-01 13:24:46 +01:00
• 28b18cc741 [Quantization] Enable BNB support for InternS1 (#21953) Jee Jee Li 2025-08-01 19:09:54 +08:00
• 4931486988 [Doc] Added warning of speculating with draft model (#22047) WeiQing ChenandDilute-l 2025-08-01 17:11:56 +08:00
• 0f81b310db [Misc] Remove upper bound in openai package version (#22060) Woosuk Kwon 2025-08-01 02:11:40 -07:00
• e6680f9e25 [Bugfix] Add log prefix in non-dp mode engine core (#21889) wuhang 2025-08-01 17:04:16 +08:00
• 27a145e893 [Doc] Add example for Step3-VL (#22061) Roger Wang 2025-08-01 01:35:49 -07:00
• da31f6ad3d Revert precompile wheel changes (#22055) Simon Mo 2025-08-01 01:26:24 -07:00
• 98df153abf [Frontend] Align tool_choice="required" behavior with OpenAI when tools is empty (#21052) Sungyoon Jeong 2025-08-01 16:54:17 +09:00
• e0f63e4a35 [Core] Avoid repeated len(block_token_ids) check in hash_request_tokens (#21781) Zebing Lin 2025-08-01 03:23:29 -04:00
• b4e081cb15 [Bugfix] Disable multi-modal preprocessor cache for DP (#21896) Cyrus Leung 2025-08-01 15:03:56 +08:00
• 79731a79f0 [Doc] Fix a syntax error of example code in structured_outputs.md (#22045) Hongsheng Liuandwangzi 2025-08-01 15:01:22 +08:00
• 53d7c39271 Update sampling_metadata.py (#21937) Aviad Rossmann 2025-08-01 09:23:18 +03:00
• 61dcc280fa [Doc] Add Voxtral to Supported Models page (#22059) Cyrus Leung 2025-08-01 14:10:56 +08:00
• 0f46a780d4 [Model] [Quantization] Support quantization for Gemma3n (#21974) Kyle Sayers 2025-08-01 01:45:15 -04:00
• e1a7fe4af5 [BugFix] fix: aot passes kvcache dtype information (#19750) Mickaël Seznec 2025-08-01 07:45:02 +02:00
• 82de9b9d46 [Misc] Automatically resolve HF processor init kwargs (#22005) Cyrus Leung 2025-08-01 13:44:10 +08:00
• ad57f23f6a [Bugfix] Fix: Fix multi loras with tp >=2 and LRU cache (#20873) Charent 2025-08-01 10:48:13 +08:00
• 3700642013 [Refactor] Remove Duplicate per_block_cast_to_fp8, Remove Dependencies of DeepGEMM (#21787) Wentao Ye 2025-07-31 21:13:27 -04:00
• 0bd409cf01 Move flashinfer-python to optional extra vllm[flashinfer] (#21959) Michael Goin 2025-07-31 21:02:11 -04:00
• e360316ab9 Add DeepGEMM to Dockerfile in vllm-base image (#21533) Matthew Bonanniandmgoin 2025-07-31 21:01:55 -04:00
• c3e0e9337e [Feature] Add Flashinfer MoE Support for Compressed Tensor NVFP4 (#21639) Wentao Ye 2025-07-31 18:26:11 -04:00
• 6e672daf62 Add FlashInfer allreduce RMSNorm Quant fusion (#21069) Ilya Markovandilmarkov 2025-07-31 22:58:38 +02:00
• 2dff2e21d9 [Bugfix] Fix MTP weight loading (#21941) Benjamin Chislett 2025-07-31 16:33:53 -04:00
• 71470bc4af [Misc] Add unit tests for chunked local attention (#21692) Yong Hoon Shin 2025-07-31 11:39:16 -07:00
• 9e0726e5bf [Meta] Official Eagle mm support, first enablement on llama4 (#20788) zhiweizandRoger Wang 2025-07-31 10:35:07 -07:00
• 53c21e492e Update torch_xla pin to 20250730 (#21956) XiongfeiWei 2025-07-31 10:26:43 -07:00
• 0780bb5783 Removing amdproduction Tests (#22027) Alexei-V-Ivanov-AMD 2025-07-31 11:53:27 -05:00
• 58bb902186 fix(setup): improve precompiled wheel setup for Docker builds (#22025) Doug Smith 2025-07-31 12:52:48 -04:00
• 7349d5268b [ez] Remove a trailing space from compilation/decorators.py (#22028) Zhengxu Chen 2025-07-31 12:46:07 -04:00
• 9484641616 [Model] Add step3 vl (#21998) Songandoliveryuan 2025-07-31 23:19:06 +08:00
• 207b750e19 [NVIDIA] Add SM100 Flashinfer MoE per tensor scale fp8 backend (#21458) amirkl94andmgoin 2025-07-31 16:00:01 +03:00
• 5daffe7cf6 [BugFix] Fix case where collective_rpc returns None (#22006) Nick Hill 2025-07-31 13:51:37 +01:00
• 2836dd73f1 [Model][CI] Let more pooling models support v1 (#21747) wang.yuqi 2025-07-31 16:51:15 +08:00
• d2aab336ad [CI/Build] get rid of unused VLLM_FA_CMAKE_GPU_ARCHES (#21599) Daniele 2025-07-31 09:00:08 +02:00
• 9532a6d563 [Deprecation] Remove deprecated args and methods (#21907) Cyrus Leung 2025-07-31 14:46:38 +08:00
• 3e36fcbee6 [Bugfix]: fix metadata file copy in test_sharded_state_loader (#21830) Ning Xie 2025-07-31 14:22:11 +08:00
• 055bd3978e [CI Bugfix] Fix CI OOM for test_shared_storage_connector_hashes (#21973) Michael Goin 2025-07-30 23:45:29 -04:00
• 0f7919fca0 [Misc] Expand SUPPORTED_HIDDEN_SIZES for DeepEP low-latency kernels (#21818) Jee Jee Li 2025-07-31 11:41:12 +08:00
• 61445453df [UX] Rename CUTLASS_MLA_VLLM_V1 to CUTLASS_MLA (#21966) Michael Goin 2025-07-30 23:40:34 -04:00
• ec02e536df [Bugfix] Relax lang pin for voxtral (#21833) Sanchit Gandhiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-07-31 04:38:52 +01:00
• 9cb497bfa3 [Example] Add async_llm_streaming.py example for AsyncLLM streaming in python (#21763) Michael Goin 2025-07-30 20:39:46 -04:00
• ca9e2be3ed [Core] Move EngineCoreRequest to Request conversion out of EngineCore (#21627) Zebing Lin 2025-07-30 18:00:54 -04:00