Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• d1b649f1ef [Quant] Aria SupportsQuant (#13416) Kyle Sayers 2025-02-18 00:51:09 -05:00
• ac19b519ed [core] fix sleep mode in pytorch 2.6 (#13456) youkaichao 2025-02-18 13:48:10 +08:00
• a1074b3efe [Bugfix] Only print out chat template when supplied (#13444) Yuan Tang 2025-02-18 00:43:31 -05:00
• 00294e1bc6 [Quant] Arctic SupportsQuant (#13366) Kyle Sayers 2025-02-18 00:35:09 -05:00
• 88787bce1d [Quant] Molmo SupportsQuant (#13336) Kyle Sayers 2025-02-18 00:34:47 -05:00
• 932b51cedd [v1] fix parallel config rank (#13445) youkaichao 2025-02-18 12:33:45 +08:00
• 7c7adf81fc [ROCm] fix get_device_name for rocm (#13438) Divakar Verma 2025-02-17 22:07:12 -06:00
• 67ef8f666a [Model] Enable quantization support for transformers backend (#12960) Isotr0py 2025-02-18 11:52:47 +08:00
• efbe854448 [Misc] Remove dangling references to SamplingType.BEAM (#13402) Harry Mellor 2025-02-18 03:52:35 +00:00
• b3942e157e [Bugfix][CI][V1] Work around V1 + CUDA Graph + torch._scaled_mm fallback issue (#13425) Tyler Michael Smith 2025-02-17 19:32:48 -05:00
• cd4a72a28d [V1][Spec decode] Move drafter to model runner (#13363) Woosuk Kwon 2025-02-17 15:40:12 -08:00
• 6ac485a953 [V1][PP] Fix intermediate tensor values (#13417) Cody Yu 2025-02-17 13:37:45 -08:00
• 4c21ce9eba [V1] Get input tokens from scheduler (#13339) Woosuk Kwon 2025-02-17 11:01:07 -08:00
• ce77eb9410 [Bugfix] Fix VLLM_USE_MODELSCOPE issue (#13384) r.4ntix 2025-02-17 22:22:01 +08:00
• 30513d1cb6 [Bugfix] fix xpu communicator (#13368) Yan Ma 2025-02-17 20:59:18 +08:00
• 1f69c4a892 [Model] Support Mamba2 (Codestral Mamba) (#9292) Tyler Michael SmithandYu Chin Fabian Lim 2025-02-17 07:17:50 -05:00
• 7b623fca0b [VLM] Check required fields before initializing field config in DictEmbeddingItems (#13380) Cyrus Leung 2025-02-17 17:36:07 +08:00
• 238dfc8ac3 [MISC] tiny fixes (#13378) Mengqing Cao 2025-02-17 16:57:13 +08:00
• 45186834a0 Run v1 benchmark and integrate with PyTorch OSS benchmark database (#13068) Huy Do 2025-02-17 00:16:32 -08:00
• f857311d13 Fix spelling error in index.md (#13369) yankooo 2025-02-17 14:53:20 +08:00
• 46cdd59577 [Feature][Spec Decode] Simplify the use of Eagle Spec Decode (#12304) shangmingc 2025-02-17 11:32:26 +08:00
• 2010f04c17 [V1][Misc] Avoid unnecessary log output (#13289) Jee Jee Li 2025-02-17 11:26:24 +08:00
• 69e1d23e1e [V1][BugFix] Clean up rejection sampler & Fix warning msg (#13362) Woosuk Kwon 2025-02-16 12:25:29 -08:00
• d67cc21b78 [Bugfix][Platform][CPU] Fix cuda platform detection on CPU backend edge case (#13358) Isotr0py 2025-02-17 02:55:27 +08:00
• e18227b04a [V1][PP] Cache Intermediate Tensors (#13353) Woosuk Kwon 2025-02-16 10:02:27 -08:00
• 7b89386553 [V1][BugFix] Add __init__.py to v1/spec_decode/ (#13359) Woosuk Kwon 2025-02-16 09:39:08 -08:00
• da833b0aee [Docs] Change myenv to vllm. Update python_env_setup.inc.md (#13325) 凌 2025-02-17 00:04:21 +08:00
• 5d2965b7d7 [Bugfix] Fix 2 Node and Spec Decode tests (#13341) Cyrus Leung 2025-02-16 22:20:22 +08:00
• a0231b7c25 [platform] add base class for communicators (#13208) youkaichao 2025-02-16 22:14:22 +08:00
• 124776ebd5 [ci] skip failed tests for flashinfer (#13352) youkaichao 2025-02-16 22:09:15 +08:00
• b7d309860e [V1] Update doc and examples for H2O-VL (#13349) Roger Wang 2025-02-16 02:35:54 -08:00
• dc0f7ccf8b [BugFix] Enhance test_pos_encoding to support execution on multi-devices (#13187) wchen61 2025-02-16 16:59:49 +08:00
• d3d547e057 [Bugfix] Pin xgrammar to 0.1.11 (#13338) Michael Goin 2025-02-15 22:42:25 -05:00
• 12913d17ba [Quant] Add SupportsQuant to phi3 and clip (#13104) Kyle Sayers 2025-02-15 22:28:33 -05:00
• 80f63a3966 [V1][Spec Decode] Ngram Spec Decode (#12193) Lily Liu 2025-02-15 18:05:11 -08:00
• 367cb8ce8c [Doc] [2/N] Add Fuyu E2E example for multimodal processor (#13331) Cyrus Leung 2025-02-15 23:06:23 +08:00
• 54ed913f34 [ci/build] update flashinfer (#13323) youkaichao 2025-02-15 21:33:13 +08:00
• 9206b3d7ec [V1][PP] Run engine busy loop with batch queue (#13064) Cody Yu 2025-02-15 03:59:01 -08:00
• ed0de3e4b8 [AMD] [Model] DeepSeek tunings (#13199) rasmith 2025-02-15 05:58:09 -06:00
• 2ad1bc7afe [V1][Metrics] Add iteration_tokens_total histogram from V0 (#13288) Mark McLoughlin 2025-02-15 11:56:19 +00:00
• 7fdaaf48ef [Bugfix] Fix qwen2.5-vl image processor (#13286) Isotr0py 2025-02-15 19:00:11 +08:00
• 067fa2255b [Bugfix]Fix search start_index of stop_checker (#13280) Xu Song 2025-02-15 13:39:42 +08:00
• 9076325677 [BugFix] Don't scan entire cache dir when loading model (#13302) Nick Hill 2025-02-14 21:33:31 -08:00
• 97a3d6d995 [Bugfix] Massage MLA's usage of flash attn for RoCM (#13310) Tyler Michael Smith 2025-02-15 00:33:25 -05:00
• 579d7a63b2 [Bugfix][Docs] Fix offline Whisper (#13274) Nicolò Lucchesi 2025-02-15 06:32:37 +01:00
• c9f9d5b397 [Bugfix][AMD] Update torch_bindings so that scaled_fp4_quant isn't build on ROCm (#13235) Sage Moore 2025-02-14 20:30:42 -08:00
• 0c73026844 [V1][PP] Fix memory profiling in PP (#13315) Woosuk Kwon 2025-02-14 20:17:25 -08:00
• 6a854c7a2b [V1][Sampler] Don't apply temp for greedy-only (#13311) Nick Hill 2025-02-14 18:10:53 -08:00
• e7eea5a520 [V1][CI] Fix failed v1-test because of min_p (#13316) Woosuk Kwon 2025-02-14 17:29:51 -08:00
• a12934d3ec [V1][Core] min_p sampling support (#13191) AoyuandAoyu 2025-02-15 07:50:05 +08:00
• 3bcb8c75da [Core] Reduce TTFT with concurrent partial prefills (#10235) 3 people 2025-02-14 16:36:07 -07:00
• 5e5c8e091e [Quant][Perf] Use moe_wna16 kernel by default for MoEs with many experts (#13236) Michael Goin 2025-02-14 15:53:42 -05:00
• c9e2d644e7 [Hardware][Gaudi][Bugfix] Fix error for guided decoding (#12317) Yu-Zhou 2025-02-14 20:36:49 +08:00
• 7734e9a291 [Core] choice-based structured output with xgrammar (#12632) Russell Bryant 2025-02-14 07:36:05 -05:00
• 6224a9f620 Support logit_bias in v1 Sampler (#13079) Lu Fang 2025-02-14 04:34:59 -08:00
• 085b7b2d6c [V1] Simplify GPUModelRunner._update_states check (#13265) Nick Hill 2025-02-14 04:33:43 -08:00
• 4da1f667e9 [VLM] Keep track of whether prompt replacements have been applied (#13215) Cyrus Leung 2025-02-14 20:20:46 +08:00
• 556ef7f714 [Misc] Log time consumption of sleep and wake-up (#13115) Jun Duan 2025-02-14 07:10:21 -05:00
• 83481ceb49 [Bugfix] Fix missing parentheses (#13263) Xu Song 2025-02-14 17:07:10 +08:00
• 185cc19f92 [Frontend] Optionally remove memory buffer used for uploading to URLs in run_batch (#12927) Pooya Davoodi 2025-02-14 00:22:42 -08:00
• 45f90bcbba [WIP] TPU V1 Support Refactored (#13049) Alexander Matveev 2025-02-14 03:21:53 -05:00
• b0ccfc565a [Bugfix][V1] GPUModelRunner._update_states should return True when there is a finished request in batch (#13126) Kero Liang 2025-02-14 14:39:20 +08:00
• ba59b78a9c [ROCm][V1] Add intial ROCm support to V1 (#12790) Sage Moore 2025-02-13 22:21:50 -08:00
• cbc40128eb [V1] LoRA - Enable Serving Usecase (#12883) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-02-14 11:51:12 +05:30
• f0b2da72a8 Expand MLA to support most types of quantization (#13181) Michael Goin 2025-02-14 01:19:22 -05:00
• f2b20fe491 Consolidate Llama model usage in tests (#13094) Harry Mellor 2025-02-14 06:18:03 +00:00
• 40932d7a05 [Misc] Remove redundant statements in scheduler.py (#13229) Wang Ran (汪然) 2025-02-14 14:07:25 +08:00
• 84683fa271 [Bugfix] Offline example of disaggregated prefill (#13214) XiaobingZhang 2025-02-14 12:20:47 +08:00
• 067678262a [Bugfix][CI] Inherit codespell settings from pyproject.toml in the pre-commit-config (#13237) Tyler Michael Smith 2025-02-13 23:19:43 -05:00
• 09545c0a94 [Bugfix/CI] Turn test_compressed_tensors_2of4_sparse back on (#13250) Tyler Michael Smith 2025-02-13 23:19:25 -05:00
• dd5ede4440 [V1] Consolidate MM cache size to vllm.envs (#13239) Roger Wang 2025-02-13 20:19:03 -08:00
• 8c32b08a86 [Kernel] Fix awq error when n is not divisable by 128 (#13227) Jinzhen Lin 2025-02-14 12:07:05 +08:00
• 410886950a [ROCm] Avoid using the default stream on ROCm (#13238) Gregory Shtrasberg 2025-02-13 20:29:26 -05:00
• e38be640e6 Revert "Add label if pre-commit passes" (#13242) Harry Mellor 2025-02-14 00:12:32 +00:00
• c1e37bf71b [Kernel][Bugfix] Refactor and Fix CUTLASS 2:4 Sparse Kernels (#13198) Tyler Michael Smith 2025-02-13 19:01:14 -05:00
• 2344192a55 Optimize moe_align_block_size for deepseek_v3 (#12850) Michael Goin 2025-02-13 18:43:37 -05:00
• bffddd9a05 Add label if pre-commit passes (#12527) Harry Mellor 2025-02-13 20:51:30 +00:00
• d84cef76eb [Frontend] Add /v1/audio/transcriptions OpenAI API endpoint (#12909) Nicolò Lucchesi 2025-02-13 16:23:45 +01:00
• 37dfa60037 [Bugfix] Missing Content Type returns 500 Internal Server Error (#13193) Vaibhav Jain 2025-02-13 20:22:22 +05:30
• 1bc3b5e71b [VLM] Separate text-only and vision variants of the same model architecture (#13157) Cyrus Leung 2025-02-13 22:19:15 +08:00
• 02ed8a1fbe [Misc] Qwen2.5-VL Optimization (#13155) 燃 2025-02-13 22:17:57 +08:00
• 2092a6fa7d [V1][Core] Add worker_base for v1 worker (#12816) 3 people 2025-02-13 20:35:18 +08:00
• c9d3ecf016 [VLM] Merged multi-modal processor for Molmo (#12966) Cyrus Leung 2025-02-13 20:34:00 +08:00
• fdcf64d3c6 [V1] Clarify input processing and multimodal feature caching logic (#13211) Roger Wang 2025-02-13 03:43:24 -08:00
• 578087e56c [Frontend] Pass pre-created socket to uvicorn (#13113) Russell Bryant 2025-02-13 03:51:46 -05:00
• fa253f1a70 [VLM] Remove input processor from clip and siglip (#13165) Isotr0py 2025-02-13 16:31:37 +08:00
• 9605c1256e [V1][core] Implement pipeline parallel on Ray (#12996) Rui Qiao 2025-02-13 00:02:46 -08:00
• 0ccd8769fb [CI/Build] Allow ruff to auto-fix some issues (#13180) Russell Bryant 2025-02-13 02:45:38 -05:00
• cb944d5818 Allow Unsloth Dynamic 4bit BnB quants to work (#12974) Daniel Han 2025-02-12 23:13:08 -08:00
• d46d490c27 [Frontend] Move CLI code into vllm.cmd package (#12971) Russell Bryant 2025-02-13 02:12:21 -05:00
• 04f50ad9d1 [Bugfix] deepseek_r1_reasoning_parser put reason content in wrong field in certain edge case (#13097) LikeSundayLikeRain 2025-02-13 02:11:26 -05:00
• 60c68df6d1 [Build] Automatically use the wheel of the base commit with Python-only build (#13178) Cody Yu 2025-02-12 23:10:28 -08:00
• 009439caeb Simplify logic of locating CUDART so file path (#13203) Lu Fang 2025-02-12 21:52:41 -08:00
• bc55d13070 [VLM] Implement merged multimodal processor for Mllama (#11427) Isotr0py 2025-02-13 12:26:21 +08:00
• d88c8666a1 [Bugfix][Example] Fix GCed profiling server for TPU (#12792) Michael Goin 2025-02-12 22:52:11 -05:00
• 4fc5c23bb6 [NVIDIA] Support nvfp4 quantization (#12784) Kaixi Hou 2025-02-12 19:51:51 -08:00
• 9f9704dca6 [perf-benchmark] cleanup unused Docker images and volumes in H100 benchmark instance (#12706) Kevin H. Luu 2025-02-12 19:51:33 -08:00
• 8eafe5eaea [CI/Build] Ignore ruff warning up007 (#13182) Russell Bryant 2025-02-12 22:48:31 -05:00
• 4c0d93f4b2 [V1][Bugfix] Copy encoder input ids to fix set iteration issue during VLM abort (#13173) Murali Andoorveedu 2025-02-12 12:58:11 -08:00
• 14b7899d10 [CI] Fix failing FP8 cpu offload test (#13170) Michael Goin 2025-02-12 14:16:06 -05:00