Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 8c946cecca Update deprecated type hinting in vllm/transformers_utils (#18058) Harry Mellor 2025-05-13 12:34:37 +01:00
• ff334ca1cd Update deprecated type hinting in vllm/profiler (#18057) Harry Mellor 2025-05-13 12:34:34 +01:00
• 6223dd8114 Update deprecated type hinting in model_executor/layers (#18056) Harry Mellor 2025-05-13 12:17:23 +01:00
• 906f0598fc [doc] add download/list/delete HF model CLI usage (#17940) Reidandreidliu41 2025-05-13 19:15:51 +08:00
• cb528d0585 [Fix] check to make sure processor has chat templates (#18047) Aaron Pham 2025-05-13 06:04:10 -04:00
• 98fcba1575 Convert .buildkite to ruff format (#17656) Harry Mellor 2025-05-13 10:28:31 +01:00
• 23b3134eb5 [Benchmarks] Refactor run_structured_output_benchmarks.sh (#17722) Russell Bryant 2025-05-13 04:47:29 -04:00
• ea6ae8cb45 [Bugfix] Fix marlin moe fallback logic for llama4 (#18042) Michael Goin 2025-05-13 03:53:28 -04:00
• 2ff297dce9 [BugFix] Set default random seed to 0 for V1 (#17929) Woosuk Kwon 2025-05-13 00:52:19 -07:00
• 8dd0671bac [Bugfix][V1] Only get input embeddings w/ multi-modal models if first PP (#17916) Jin HuangandJin Huang 2025-05-13 03:10:07 -04:00
• f0d610a8ae [v1][KVCacheManager] Avoid full cache hit by controlling max_length (#17999) Chen ZhangandWoosuk Kwon 2025-05-13 14:50:38 +08:00
• e57e4d6e9e Fix Broken macro for cutlass moe (#18049) Driss Guessous 2025-05-12 23:31:06 -07:00
• ee5be834e7 [BugFix] Fix 4-GPU RLHF tests (#18007) Nick Hill 2025-05-12 23:03:55 -07:00
• 48545728d8 cleanup invalid prints (#18050) Calvin Chen 2025-05-13 14:01:57 +08:00
• dc1a821768 [Feature][V1] Support tool_choice: required when using Xgrammar as the StructuredOutputBackend. (#17845) Chauncey 2025-05-13 14:01:31 +08:00
• 61e0a506a3 [Bugfix] Avoid repeatedly creating dummy data during engine startup (#17935) Cyrus Leung 2025-05-13 13:40:19 +08:00
• 1df491c522 [Bugfix] Fixes for new marlin moe usage (#18017) Michael Goin 2025-05-12 23:50:04 -04:00
• d8487ef557 [ROCm]: Fix build from source failure with gcc14 and ROCm 6.3 (#13779) Arjun Kathuria 2025-05-13 09:06:33 +05:30
• c06af9a959 [Misc] Slight spelling modification (#18039) Jee Jee Li 2025-05-13 11:36:27 +08:00
• 60f7624334 Implements dual-chunk-flash-attn backend for dual chunk attention with sparse attention support (#11844) Tao He 2025-05-13 10:52:47 +08:00
• f6518b2b48 [ROCm] Skip tests for quantizations incompatible with ROCm (#17905) hissu-hyvarinen 2025-05-13 03:39:28 +03:00
• d67085c2c8 Remove noisy warnings from SchedulerConfig (#17995) Harry Mellor 2025-05-13 01:33:45 +01:00
• 307939f299 Use NVFP4 Marlin for CompressedTensorsW4A16Fp4 (#18000) Michael GoinandDipika 2025-05-12 20:07:34 -04:00
• 9d7ea9dbbf Update some more deprecated type hinting (#17998) Harry Mellor 2025-05-13 00:49:33 +01:00
• acee8f48aa [Model] Support MiMo-7B inference with MTP (#17433) bwshen-miandwangpeng66 2025-05-13 07:25:33 +08:00
• f065de4e88 Fix FBGEMM integration (#18002) Michael Goin 2025-05-12 19:02:07 -04:00
• dc9905368d [V1][Spec Decode] Eagle unit tests (#17350) wwl2755 2025-05-12 16:01:17 -07:00
• ebab1ac37c [CI] Make JSON output tests less likely to fail (#17859) Russell Bryant 2025-05-12 18:31:54 -04:00
• 2b0db9b0e2 Enable standard language model for torhc nightly (#18004) Yang Wang 2025-05-12 14:00:04 -07:00
• 195adb47c0 [Chore] Remove unused method (#18024) Robert Shaw 2025-05-12 16:59:47 -04:00
• 302f3aca7e [v1][KVCacheManager] Change prefix caching metric from counting blocks to counting tokens (#18003) Chen Zhang 2025-05-13 04:46:12 +08:00
• e9c730c9bd Enabling "Weight Loading Multiple GPU Test - Large Models" (#18020) Alexei-V-Ivanov-AMD 2025-05-12 15:05:33 -05:00
• 289199feb6 [Core] Use platform-agnostic device control for DP engine core (#17245) Jade Zheng 2025-05-13 03:09:16 +08:00
• b9fd0d7a69 [CI/Build] Fix TPU V1 Test mixed use of & and && across tests (#17968) Carol Zheng 2025-05-12 12:06:59 -07:00
• 72a3f6b898 Construct KVTransferConfig properly from Python instead of using JSON blobs without CLI (#17994) Harry Mellor 2025-05-12 19:25:33 +01:00
• 98ea35601c [Lora][Frontend]Add default local directory LoRA resolver plugin. (#16855) Jonathan Berkhahn 2025-05-12 10:39:10 -07:00
• d19110204c [P/D] NIXL Integration (#17751) 8 people 2025-05-12 12:46:16 -04:00
• 05a4324f8e Initialize the delta tool call fields explicitly (#17340) Maximilien de Bayserandigmainc 2025-05-12 10:28:58 -03:00
• 7ea6cb28b2 [Misc] Improve modelscope import error (#17983) Jee Jee Li 2025-05-12 18:46:45 +08:00
• 9fbf2bfbd5 Correcting testcases in builkite job for IBM Power (#17675) Aaruni Aggarwal 2025-05-12 13:41:55 +05:30
• 3a5ea75129 [Feature] Support DeepSeekV3 Function Call (#17784) Xu Wenqing 2025-05-12 15:45:21 +08:00
• 891b9d33de [Fix] Benchmark "EngineClient" has no attribute "model_config" (#17976) Brayden Zhong 2025-05-12 01:55:53 -04:00
• 430783018c [Bugfix][TPU] Use np array when updating cache slot_mapping (#17971) Siyuan Liu 2025-05-11 21:58:33 -07:00
• 19a3c78d1f [Bugfix] Fix pydantic.errors.PydanticUserError (#17962) Li Wang 2025-05-12 12:58:23 +08:00
• ada50aa295 [bugfix] fix the wrong parser (#17958) Reidandreidliu41 2025-05-12 12:58:02 +08:00
• 08bf784078 [Bugfix] validate grammar and throw 400 error instead of crashing the engine when xgrammar validation fails (#17623) Cheng Kuan Yong JasonandRussell Bryant 2025-05-12 09:06:10 +08:00
• d45fe333fb [misc] add instructions on how to install nvshmem/pplx/deepep (#17964) youkaichao 2025-05-12 09:02:39 +08:00
• 021c16c7ca [Model] Broadcast Ovis2 implementation to fit Ovis1.6 (#17861) Isotr0py 2025-05-12 08:56:30 +08:00
• 7de18d541b [BUG] [ROCm] [MLA] Fix variable name bug due to change in variable name in PR #17483 (#17961) TJian 2025-05-12 00:14:30 +08:00
• a810b5b088 [BugFix] [ROCm]: Bugfix and handle addition case of input for rocm_aiter_rms_norm (#17857) TJian 2025-05-11 19:17:11 +08:00
• 009b3d5382 [Misc] not show --model in vllm serve --help (#16691) Reidandreidliu41 2025-05-11 16:47:58 +08:00
• e4b8713380 [New Model]: nomic-embed-text-v2-moe (#17785) wang.yuqi 2025-05-11 15:59:43 +08:00
• 06c0922a69 [FP8][ROCm][Attention] Enable FP8 KV cache on ROCm for V1 (#17870) Gregory Shtrasberg 2025-05-11 03:58:45 -04:00
• cd3edfc908 [Misc] Add compressed-tensors NVFP4A16 emulation support (#17914) Dipika Sikka 2025-05-11 03:58:38 -04:00
• 9cea90eab4 [Frontend] Add /classify endpoint (#17032) Frieda Huang 2025-05-11 03:57:07 -04:00
• d1110f5b5a [doc] update lora doc (#17936) Reidandreidliu41 2025-05-11 15:56:21 +08:00
• 8132365b74 [Bugfix]: v1 engine - consider lora adapters in allowed_token_ids (#17855) Ben Browning 2025-05-11 03:53:58 -04:00
• eea22a56ab fix amd triton mla path (#17871) Shiyan Deng 2025-05-11 00:53:31 -07:00
• 9112155283 [Perf] Use small max_num_batched_tokens for A100 (#17885) Kuntai Du 2025-05-11 00:53:23 -07:00
• 90d0a74b60 [Bugfix] Add revision to transformers.Auto*.from_pretrained processors (#17948) xinli-centml 2025-05-11 03:52:44 -04:00
• d74e5f37bc [Kernel] fp4 marlin kernel (#17687) Jinzhen Lin 2025-05-11 10:58:49 +08:00
• ca66a1674c [v1] Rename specialized_manager.py to single_type_kv_cache_manager.py (#17946) Chen Zhang 2025-05-11 07:14:12 +08:00
• 950751a987 [v1] Pass BlockTable and KVCacheSpec to AttentionMetadataBuilders (#17483) Chen Zhang 2025-05-11 07:12:04 +08:00
• 4c31218f80 [Misc] remove --model from vllm serve usage (#17944) Reidandreidliu41 2025-05-10 21:23:31 +08:00
• 68311891f5 Don't default construct ModelConfig when default constructing VllmConfig (#17943) Harry Mellor 2025-05-10 14:23:00 +01:00
• fc4441a4ee Add missing content type headers to /ping and /health (#17036) (#17786) Ximo GuanterandHarry Mellor 2025-05-10 08:13:32 +02:00
• 246e3e0a36 fix broken test vllm:test_kernels - test_attention_selector.py::test_flash_attn (#17873) tracelogfbandStephen Chen 2025-05-09 19:46:54 -07:00
• 7042cc96b0 [V1][Spec Decoding] Log accumulated metrics after system goes idle (#17913) Mark McLoughlin 2025-05-10 02:23:07 +01:00
• 0c0fdae84f [Hardware/NVIDIA/Kernel] Enable nvidia/DeepSeek-R1-FP4 Model (#16362) Pavani Majety 2025-05-09 16:24:41 -07:00
• 3b602cdea7 AMD conditional all test execution // new test groups (#17556) Alexei-V-Ivanov-AMD 2025-05-09 17:35:58 -05:00
• 4b2ed7926a Improve configs - the rest! (#17562) Harry Mellor 2025-05-09 23:18:44 +01:00
• 7e3571134f [V1][Spec Decoding] Include bonus tokens in mean acceptance length (#17908) Mark McLoughlin 2025-05-09 21:32:36 +01:00
• ea2236bf95 Add option to use torch._inductor.standalone_compile (#17057) Richard Zou 2025-05-09 15:59:04 -04:00
• 7d4aedae7c Handle error when str passed to /v1/audio/transcriptions (#17909) Harry Mellor 2025-05-09 20:23:59 +01:00
• 22481fbfa3 Update CT WNA16MarlinMoE integration (#16666) Michael Goin 2025-05-09 11:19:45 -06:00
• 5c4c08f6f1 [Misc] Auto fallback to float16 for pre-Ampere GPUs when detected bfloat16 config (#17265) Isotr0py 2025-05-10 01:16:12 +08:00
• c44c384b1c [Misc] Add references in ray_serve_deepseek example (#17907) Rui Qiao 2025-05-09 09:59:36 -07:00
• 85b72cb7b1 Revert "[BugFix][AMD] Compatible patch for latest AITER(05/07/2025)" (#17910) Michael Goin 2025-05-09 09:58:18 -06:00
• 6e5595ca39 [CI/Build] Automatically retry flaky tests (#17856) Cyrus Leung 2025-05-09 23:55:17 +08:00
• 200da9a517 [v1] Move block management logic from KVCacheManager to SpecializedManager (#17474) Chen Zhang 2025-05-09 23:25:34 +08:00
• 9f64e93415 [BugFix][AMD] Compatible patch for latest AITER(05/07/2025) (#17864) qli88 2025-05-09 09:59:36 -05:00
• ec61ea20a8 [Misc] add dify integration (#17895) Reidandreidliu41 2025-05-09 18:42:39 +08:00
• c6798baa9c Change top_k to be disabled with 0 (still accept -1 for now) (#17773) Harry Mellor 2025-05-09 11:01:49 +01:00
• 5b2dcbf0b8 Fix Whisper crash caused by invalid`` max_num_batched_tokens`` config (#17853) inkcherry 2025-05-09 17:16:26 +08:00
• 6e4a93e3f7 [Bugfix][CPU] Fix broken AVX2 CPU TP support (#17252) Isotr0py 2025-05-09 16:55:14 +08:00
• 217db4baa6 [Bugfix][ROCm] Fix AITER MLA V1 (#17880) vllmellm 2025-05-09 16:38:21 +08:00
• ff8c400502 [Doc] remove visible token in doc (#17884) Yan Ma 2025-05-09 16:21:31 +08:00
• 89a0315f4c [Doc] Update several links in reasoning_outputs.md (#17846) Michael Yao 2025-05-09 16:20:55 +08:00
• 3d1e387652 [Docs] Add Slides from NYC Meetup (#17879) Simon Mo 2025-05-08 21:46:54 -07:00
• d310e6de98 [BUGFIX]: return fast when request requires prompt logprobs (#17251) Ning Xie 2025-05-09 12:25:41 +08:00
• 5e6f939484 [Attention] MLA move rotary embedding to cuda-graph region (#17668) Lucas Wilkinson 2025-05-08 23:14:42 -04:00
• 760e3ecc8f [V1][Structured Output] Update llguidance (>= 0.7.11) to avoid AttributeError (no StructTag) (#17839) Shanshan Shen 2025-05-09 11:14:18 +08:00
• 3c9396a64f [FEAT][ROCm]: Support AITER MLA on V1 Engine (#17523) 3 people 2025-05-09 10:42:05 +08:00
• 376786fac1 Add cutlass support for blackwell fp8 blockwise gemm (#14383) Shu Wang 2025-05-08 17:09:55 -05:00
• 4f605a6de5 Fix noisy warning for uncalibrated q_scale/p_scale (#17414) Michael Goin 2025-05-08 15:56:59 -04:00
• 8342e3abd1 [CI] Prune down lm-eval small tests (#17012) Michael Goin 2025-05-08 15:00:26 -04:00
• a83a0f92b5 [Test] Attempt all TPU V1 tests, even if some of them fail. (#17334) yarongmu-google 2025-05-08 10:20:54 -07:00
• 226a4272cf [V1] Improve VLLM_ALLOW_INSECURE_SERIALIZATION logging (#17860) Russell Bryant 2025-05-08 12:57:35 -04:00
• ec54d73c31 [CI] Fix test_collective_rpc (#17858) Russell Bryant 2025-05-08 12:47:12 -04:00
• a944f8ede7 [Misc] Delete LoRA-related redundancy code (#17841) Jee Jee Li 2025-05-08 21:02:21 +08:00