Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 73deea2fdb [Frontend] track server_load (#13950) daniel-salib 2025-03-14 09:53:17 -07:00
• 9d2b4a70f4 [V1][Metrics] Updated list of deprecated metrics in v0.8 (#14695) Mark McLoughlin 2025-03-14 16:45:25 +00:00
• 0b0d6421b2 [Frontend] Fix log message to use http vs https (#14774) Russell Bryant 2025-03-14 12:21:09 -04:00
• 1140991a7b [V1] Fix vocab size calculation for structured output (#14826) Russell Bryant 2025-03-14 12:18:38 -04:00
• 613c5bb945 [Bugfix] Fix Aria test loading (#14823) Cyrus Leung 2025-03-15 00:11:23 +08:00
• fd8e055ffb [BugFix]: properly catch templating error when preprocess input (#13976) Guillaume Calmettes 2025-03-14 08:58:34 -04:00
• ab93f1360f [VLM] Various cleanup and fixes (#14806) Cyrus Leung 2025-03-14 20:58:19 +08:00
• 40253bab44 [Bugfix][W8A8] fixed cutlass block fp8 binding (#14796) DefTruth 2025-03-14 18:32:42 +08:00
• c77620d22d [V1][Minor] Minor code cleanup for scheduling metrics (#14800) Woosuk Kwon 2025-03-14 01:21:28 -07:00
• 989ecd2007 [Misc] Gemma3ForConditionalGeneration supports LoRA (#14797) Jee Jee Li 2025-03-14 16:07:30 +08:00
• 54cc46f3eb [Bugfix] Fix small typo in the example of Streaming delimiter (#14793) WeiCheng 2025-03-14 16:05:17 +08:00
• 601bd3268e [Misc] Clean up type annotation for SupportsMultiModal (#14794) Cyrus Leung 2025-03-14 15:59:56 +08:00
• 09269b3127 [BugFix]Fix performance serving benchmark when enable profiling (#14737) Li Wang 2025-03-14 15:02:05 +08:00
• 27b50f1fe6 [Bugfix][Kernel][CPU] Fix num_tokens in CPU rotary embedding kernel (#14667) Thien Tran 2025-03-14 14:47:49 +08:00
• 9532c49836 [Attention] MLA get rid of materialization (#14770) Lucas Wilkinson 2025-03-14 02:39:02 -04:00
• 0c2af17c76 [CI] Fix missing example model id in processor test (#14787) Roger Wang 2025-03-13 22:52:15 -07:00
• a6e0d096dd [Feature] Add visionarena offline support for benchmark_throughput (#14654) 4 people 2025-03-13 21:07:54 -07:00
• d3d4956261 [Neuron] flatten test parameterization for neuron attention kernels (#14712) Liangfu Chen 2025-03-13 20:46:56 -07:00
• 4059adc31b [Misc][Minor] Simplify SamplingParams.__post_init__() (#14772) Nick Hill 2025-03-13 23:44:20 -04:00
• f1f632d9ec [ci] Reduce number of tests in fastcheck (#14782) Kevin H. Luu 2025-03-13 20:43:45 -07:00
• 95d680b862 [Bugfix][IPEX] Add VLLM_CPU_MOE_PREPACK to allow disabling MoE prepack when CPU does not support it (#14681) Thien Tran 2025-03-14 11:43:18 +08:00
• fb4c7f8ef0 [Kernel] [V1] Further optimizations to ROCm (Triton) Backend to better handle GQA. (#14431) 4 people 2025-03-14 04:42:27 +01:00
• 0b1cfa6180 [Kernel] LoRA - Enable CUDAGraphs for V1 (#14626) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-03-13 23:42:04 -04:00
• 32ef4983cd [V1] Temporarily disable FlashInfer Rejection Sampler (#14788) Woosuk Kwon 2025-03-13 20:40:35 -07:00
• ad19c8a003 [V1] Move OOM check into sampler run (#14728) Roger WangandSimon Mo 2025-03-13 20:40:23 -07:00
• 2a602b055a forward fix PR 14245, restore build on ROCm 6.2 (#14709) Jeff Daily 2025-03-13 20:40:15 -07:00
• 7888e1d0a3 [V1] TPU - Enable prefix caching by default (#14773) Alexander Matveev 2025-03-13 23:40:05 -04:00
• 60c872d4b6 [Doc] Fix small typo in Transformers fallback (#14791) Chen Zhang 2025-03-14 11:33:12 +08:00
• 3fb17d26c8 [Doc] Fix typo in documentation (#14783) yasu52 2025-03-13 20:33:09 -07:00
• d47807ba08 [Attention] Remove slow setattr in MLA (#14769) Lucas Wilkinson 2025-03-13 17:31:14 -04:00
• 02fcaa3d0a [V1] Detokenizer: Respect Stop Tokens + not include_stop_str_in_output (#14624) afeldman-nm 2025-03-13 15:07:34 -04:00
• 8a4a2efc6f [V1][Core] using cached vocab_size for Structured Outputs (#14630) Aaron Pham 2025-03-13 14:39:28 -04:00
• 8e9ffd37d6 [Misc] Clean up processor tests (#14771) Cyrus Leung 2025-03-14 02:25:37 +08:00
• 01b3fd0af7 [V1][Minor] Minor enhancements on scheduler (#14732) Woosuk Kwon 2025-03-13 08:53:22 -07:00
• f53a0586b9 [Bugfix] Fix prompt format of GLM4V (#14539) Cyrus Leung 2025-03-13 19:37:17 +08:00
• b1cc4dfef5 [VLM] Support loading InternVideo2.5 models as original InternVLChatModel (#14738) Isotr0py 2025-03-13 18:10:02 +08:00
• 382403921f [VLM] Support pan-and-scan for Gemma3 multi-modal processor (#14672) 3 people 2025-03-13 17:23:12 +08:00
• a73122de96 [Bugfix] fix benchmark moe (#14653) Jee Jee Li 2025-03-13 16:12:42 +08:00
• bd44b812cb [CI/Build] Delete ultravox LoRA test (#14730) Jee Jee Li 2025-03-13 15:57:39 +08:00
• 55211b01e8 [Bugfix] Fix chunked prefill for GGUF (#14666) Szymon Ożóg 2025-03-13 08:19:03 +01:00
• 5d043c1685 [Quant] Bamba SupportsQuant (#14698) Kyle Sayers 2025-03-13 00:57:05 -04:00
• 36d1ccb286 [Quant] BartModel SupportsQuant (#14699) Kyle Sayers 2025-03-13 00:55:59 -04:00
• 1bc3b739c4 [V1][TPU] Add assertion on multi-step-scheduler (#14707) Siyuan Liu 2025-03-12 21:37:58 -07:00
• 1bd32bc8dd [Config][Disaggregated] Add timeout configuration for the torch.store and add KVTransferConfig.kv_connector_extra_config (#14367) Mathis Felardos 2025-03-13 04:15:20 +01:00
• 128bf75283 [BugFix][TritonMLA] Process weights after model loading for GGUF (#14555) TY-AMD 2025-03-13 11:14:36 +08:00
• a94a699c3f [ROCm][FP8] Fix for adjustments needed only for fnuz (#14689) Gregory Shtrasberg 2025-03-12 23:14:04 -04:00
• ab426ec9c0 Add ray[data] as tpu dependency (#14691) Richard Liu 2025-03-12 20:13:48 -07:00
• 165290d357 [bugfix] fixup warning message for plugged schedulers for v1 (#14700) Joe Runde 2025-03-12 21:12:13 -06:00
• ce20124671 [release] Add force remove for TPU logs (#14697) Kevin H. Luu 2025-03-12 15:35:18 -07:00
• 53be4a8634 [V1] Allow sliding window + prefix caching (#13069) Woosuk Kwon 2025-03-12 11:21:19 -07:00
• f5d3acd474 [BugFix][V1] Fix parallel sampling finishing/aborts (#14512) Nick Hill 2025-03-12 13:29:48 -04:00
• 916836bbfb [FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash Attention to enable embedding models. (#14664) TJian 2025-03-13 00:31:19 +08:00
• d9f83d6206 [ROCm] Enable chunked prefill/paged attention in MLA on ROCm (#14316) Sage Moore 2025-03-12 08:51:20 -07:00
• 4a754fcf15 [Bugfix] Missing thumbnail from NVLM-D processor (#14633) ameyanjarlekar 2025-03-12 08:50:49 -07:00
• c0c25e25fa [Model] Add support for Gemma 3 (#14660) 3 people 2025-03-12 08:36:33 -07:00
• 45f3f3f59e [ROCm][Bugfix] Ensure that the moe_wna16_gemm kernel is not built on ROCm platforms. (#14629) Sage Moore 2025-03-12 05:00:28 -07:00
• ff47aab056 [CPU] Upgrade CPU backend to torch-2.6 (#13381) Li, JiangandIsotr0py 2025-03-12 18:41:13 +08:00
• debd6bbf09 [Kernel] Add ModelOpt FP4 Checkpoint Support (#12520) Pavani Majety 2025-03-11 22:13:11 -07:00
• 5c538c37b2 [V1][Bugfix][Spec Decode] Fix incorrect outputs in V1 speculative decoding due to batch indexing (#14645) Benjamin Chislett 2025-03-12 01:12:41 -04:00
• e22ee1e7a2 [Kernel] GGUF MoE kernel (#14613) Szymon Ożóg 2025-03-12 04:33:27 +01:00
• e392d85831 [Core] Refactor QKVCrossParallelLinear implementation to support BNB 4-bit quantization (#14545) Isotr0py 2025-03-12 11:12:52 +08:00
• 77a318bd01 [V1][Core] Support MistralTokenizer for Structured Output (#14625) Aaron Pham 2025-03-11 22:40:09 -04:00
• 80e78d02ac [Model] Extend Ultravox to accept audio longer than 30s (#13631) Farzad Abdolhosseini 2025-03-11 19:27:10 -07:00
• 4a42b9f5d6 [Doc] Update benchmarks README (#14646) 3 people 2025-03-11 19:23:04 -07:00
• 47532cd9f4 [core][V1] pluggable scheduler (#14466) Joe Runde 2025-03-11 19:15:15 -06:00
• 36e0c8f7da [Feature] Add vllm bench CLI (#13993) Randy ChenandCody Yu 2025-03-11 17:31:48 -07:00
• 9f583e360c [release] Add commands to clean up logs on TPU release node (#14642) Kevin H. Luu 2025-03-11 17:14:50 -07:00
• b706d898af [Bugfix][V1][PP] Only warmup sampler at last PP rank (#14643) Cody Yu 2025-03-11 16:40:07 -07:00
• 863d315c86 [V1][TPU] Pad the block_table.shape[1] so the ragged paged attention can handle correctly (#14597) iefgnoix 2025-03-11 16:12:26 -07:00
• d374f04a33 Fix run_tpu_test (#14641) Richard Liu 2025-03-11 14:14:33 -07:00
• 61a01b27a7 [V1] Delay all xgrammar usage until needed (#14616) Russell Bryant 2025-03-11 16:21:33 -04:00
• 53056731fd fix some typos : supported_head_sizes (#14627) Yang.Tao 2025-03-12 01:38:24 +08:00
• 4cbf286794 [V1] Remove cache from StructuredOutputManager (#14622) Russell Bryant 2025-03-11 13:36:07 -04:00
• c6e14a61ab [Hardware][Intel GPU] upgrade IPEX dependency to 2.6.10. (#14564) Kunshang Ji 2025-03-11 10:11:47 -07:00
• 07b4b7a37f [BugFix/Build] Fix sparse kernels not getting built on hopper (#14572) Lucas Wilkinson 2025-03-11 13:09:03 -04:00
• 07964e2f30 docs: Add documentation for s390x cpu implementation (#14198) Dilip Gowda BhagavanandHarry Mellor 2025-03-11 22:32:17 +05:30
• 4bf82d4b90 [V1] Add regex structured output support with xgrammar (#14590) Russell Bryant 2025-03-11 11:03:44 -04:00
• 9ab326713f Uninstall dependencies before installing requirements/tpu.txt (#14586) Richard Liu 2025-03-11 08:01:35 -07:00
• af295e9b01 [Bugfix] Update --hf-overrides for Alibaba-NLP/gte-Qwen2 (#14609) Cyrus Leung 2025-03-11 22:59:43 +08:00
• a1c8f3796c dynamic distpatch of fp8 kernels (#14245) Jeff Daily 2025-03-11 07:54:56 -07:00
• 08a1a1121d benchmarks: simplify test jsonschema (#14567) Russell Bryant 2025-03-11 09:39:30 -04:00
• 1477ffc381 [VLM] Cleanup siglip legacy code and fix broken paligemma multimodal processor (#14602) Isotr0py 2025-03-11 19:27:36 +08:00
• 70b808fe1a [Perf]:Optimize qwen2-vl to reduce cudaMemcpyAsync (#14377) yexin(叶鑫) 2025-03-11 15:39:56 +08:00
• 63d635d179 [Misc] Correct deepseek-vl2 chat template (#14558) Isotr0py 2025-03-11 12:37:11 +08:00
• 1fc973c0b5 [V1][Core] Fix memory issue with logits & sampling (#14508) Roger WangandVarun Sundar Rabindranath 2025-03-10 21:03:41 -07:00
• c982ac5722 [Bugfix] Fix FP16 overflow for DeepSeek V2 (#13232) Concurrensee 2025-03-10 22:46:59 -05:00
• 4290b704ff [V1][PP] Do not block engine core when no requests to schedule (#14585) Cody Yu 2025-03-10 19:48:24 -07:00
• c91b64f749 [neuron] add reshape_and_cache (#14391) Liangfu Chen 2025-03-10 18:37:29 -07:00
• d6123170d5 [Neuron] Add Neuron device communicator for vLLM v1 (#14085) gnovack 2025-03-10 18:37:04 -07:00
• 485afdd3cb [MISC][V1] Handle exception of current_platform.get_device_name() in arg_utils (#14379) Cody Yu 2025-03-10 17:42:11 -07:00
• 90e88ab756 [Kernel] moe wna16 cuda kernel (#13321) Jinzhen Linandmgoin 2025-03-11 08:12:40 +08:00
• 04421dff8a [V1] Prevent xgrammar from breaking TPU support (#14575) Russell Bryant 2025-03-10 19:06:19 -04:00
• 432d6dad15 Fix typo in benchmark_serving_structured_output.py (#14566) Russell Bryant 2025-03-10 17:58:58 -04:00
• 5ff0d32580 [V1] LoRA - Add triton kernels for V1 (#13096) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-03-10 17:27:53 -04:00
• 0967110e42 [Minor] Update the tqdm bar for parallel sampling (#14571) Woosuk Kwon 2025-03-10 14:23:48 -07:00
• fb0acb6c72 [Perf] Improve MLA on V1 (#14540) Simon Mo 2025-03-10 12:06:58 -07:00
• 92b0ce2ac7 [Bugfix][v1] fixed llava-hf/llava-1.5-7b-hf is broken on V1 (#14554) ChaunceyandDarkLight1337 2025-03-11 02:24:51 +08:00
• bc2d4473bf [Docs] Make installation URLs nicer (#14556) Harry Mellor 2025-03-10 18:43:08 +01:00
• 3b352a2f92 Correct capitalisation: VLLM -> vLLM (#14562) Harry Mellor 2025-03-10 17:36:21 +01:00
• dea985aef0 [V1][Bugfix] Fix handing of second_per_grid_ts for Qwen2-VL & Qwen2.5-VL (#14548) Roger Wang 2025-03-10 09:03:11 -07:00