Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 322d2a27d6 [BugFix] Minor: logger import in attention backend (#13706) Andy Lo 2025-02-23 00:51:13 +00:00
• 82e0d601fc [CI/Build] Fix pre-commit errors from #13571 (#13709) Roger Wang 2025-02-22 16:50:38 -08:00
• 78ac0f591d [CI/Build] fix uv caching in Dockerfile (#13611) Daniele 2025-02-22 17:25:20 +01:00
• b56155e7f3 [XPU]fix setuptools version for xpu (#13548) Yan Ma 2025-02-23 00:05:35 +08:00
• 382f66fb08 [Bugfix] Fix boolean conversion for OpenVINO env variable (#13615) Helena Kloosterman 2025-02-22 17:04:12 +01:00
• 8354f6640c [Doc] Dockerfile instructions for optional dependencies and dev transformers (#13699) Cyrus Leung 2025-02-22 22:04:31 +08:00
• c904fdddf6 [ROCm] Apply FP8 weights padding to values not divisible by 512 bytes on ROCm (#13231) Gregory Shtrasberg 2025-02-22 08:54:38 -05:00
• 558db8083c [V1][Kernel] Refactor the prefix_prefill kernel so that the caller no longer has to pass in the context lengths (#13095) Sage Moore 2025-02-22 05:25:41 -08:00
• e109e598c7 [NVIDIA] Support nvfp4 cutlass gemm (#13571) Kaixi Hou 2025-02-22 05:24:05 -08:00
• 8db1b9d0a1 Support SSL Key Rotation in HTTP Server (#13495) Keyun Tong 2025-02-22 05:17:44 -08:00
• 2382ad29d1 [ci] fix linter (#13701) youkaichao 2025-02-22 20:28:59 +08:00
• 3e472d882a [core] set up data parallel communication (#13591) youkaichao 2025-02-22 19:28:59 +08:00
• 7f6bae561c [CI/Build] Fix pre-commit errors (#13696) Cyrus Leung 2025-02-22 16:31:26 +08:00
• 105b8ce4c0 [Misc] Reduce LoRA-related static variable (#13166) Jee Jee Li 2025-02-22 16:21:30 +08:00
• 2cb8c1540e [Metrics] Add --show-hidden-metrics-for-version CLI arg (#13295) Mark McLoughlin 2025-02-22 08:20:45 +00:00
• 1cd981da4f [V1][Metrics] Support vllm:cache_config_info (#13299) Mark McLoughlin 2025-02-22 08:20:00 +00:00
• fca20841c2 Correction to TP logic for Mamba Mixer 2 when Num Groups not divisible by TP Size (#13660) Yu Chin Fabian Lim 2025-02-22 16:19:10 +08:00
• da31b5333e [Bugfix] V1 Memory Profiling: V0 Sampler Integration without Rejection Sampler (#13594) Jennifer ZhaoandRoger Wang 2025-02-22 00:08:29 -08:00
• bb78fb318e [v1] Support allowed_token_ids in v1 Sampler (#13210) Lu Fang 2025-02-21 22:13:05 -08:00
• 8aca27fa11 [Bugfix] Fix benchmark script bug: inaccurate stats for vllm backend when max_model_len < input_len + output_len (#13691) Robin 2025-02-22 14:10:38 +08:00
• 95c617e04b [Misc] Bump compressed-tensors (#13619) Dipika Sikka 2025-02-22 01:09:04 -05:00
• 9a1f1da5d1 [Bugfix][Model] OLMo 2: split qkv correctly for GQA and MQA (#13687) Shane A 2025-02-21 22:07:45 -08:00
• 68d630a0c7 [ROCM] fix native attention function call (#13650) Gordon Wong 2025-02-22 14:07:04 +08:00
• 68d535ef44 [Misc] Capture and log the time of loading weights (#13666) Jun Duan 2025-02-22 01:06:34 -05:00
• c6ed93860f [Bugfix][API Server] Fix invalid usage of 'ge' and 'le' in port valid… (#13672) Robin 2025-02-22 14:05:28 +08:00
• 0ffdf8ce0c [HTTP Server] Make model param optional in request (#13568) Keyun Tong 2025-02-21 21:55:50 -08:00
• 8c0dd3d4df docs: Add a note on full CI run in contributing guide (#13646) Yuan Tang 2025-02-22 00:53:59 -05:00
• ada7c780d5 [Misc] Fix yapf linting tools etc not running on pre-commit (#13695) Isotr0py 2025-02-22 13:10:43 +08:00
• 288cc6c234 [Attention] MLA with chunked prefill (#12639) 4 people 2025-02-21 18:30:12 -05:00
• 900edbfa48 fix typo of grafana dashboard, with correct datasource (#13668) John Zheng 2025-02-22 02:21:05 +08:00
• b2c3fc5d65 [Bugfix][CPU] Fix cpu all-reduce using native pytorch implementation (#13586) Isotr0py 2025-02-21 14:24:17 +08:00
• 839b27c6cc [Kernel]Add streamK for block-quantized CUTLASS kernels (#12978) leoneo 2025-02-21 14:14:24 +08:00
• 34ad27fe83 [ci] Fix metrics test model path (#13635) Kevin H. Luu 2025-02-20 22:12:10 -08:00
• 1c3c975766 [FEATURE] Enables /score endpoint for embedding models (#12846) Gabriel Marinho 2025-02-21 03:09:47 -03:00
• 1cdc88614a Missing comment explaining VDR variable in GGUF kernels (#13290) Szymon Ożóg 2025-02-21 07:06:54 +01:00
• 31aa045c11 [V1][Sampler] Avoid an operation during temperature application (#13587) Nick Hill 2025-02-20 22:05:56 -08:00
• a30c093502 [Bugfix] Add mm_processor_kwargs to chat-related protocols (#13644) Roger Wang 2025-02-20 22:04:33 -08:00
• c7b07a95a6 Use pre-commit to update requirements-test.txt (#13617) Harry Mellor 2025-02-21 06:03:27 +00:00
• 27a09dc52c [NVIDIA] Fix an issue to use current stream for the nvfp4 quant (#13632) Kaixi Hou 2025-02-20 22:01:48 -08:00
• 981f3c831e [Misc] Adding script to setup ray for multi-node vllm deployments (#12913) Edwin Hernandez 2025-02-20 21:16:40 -08:00
• 44c33f01f3 Add llmaz as another integration (#13643) Kante Yin 2025-02-21 11:52:40 +08:00
• 33170081f1 [Neuron][Kernel] Vectorize KV cache load in FlashPagedAttention to maximize DMA bandwidth (#13245) Lingfan Yu 2025-02-20 17:45:45 -08:00
• 71face8540 [Bugfix] Fix max_num_batched_tokens for MLA (#13620) Michael Goin 2025-02-20 20:45:20 -05:00
• bfbc0b32c6 [Frontend] Add backend-specific options for guided decoding (#13505) Joe Runde 2025-02-20 13:07:58 -07:00
• 6a417b8600 fix neuron performance issue (#13589) ajayvohra2005 2025-02-20 13:59:36 -05:00
• d3ea50113c [V1][Minor] Print KV cache size in token counts (#13596) Woosuk Kwon 2025-02-20 09:24:31 -08:00
• 34aad515c8 Update pre-commit's isort version to remove warnings (#13614) Harry Mellor 2025-02-20 16:00:14 +00:00
• ed6e9075d3 [Bugfix] Fix deepseekv3 grouped topk error (#13474) chenxiaobing 2025-02-20 22:47:01 +08:00
• 992e5c3d34 Merge similar examples in offline_inference into single basic example (#12737) Harry Mellor 2025-02-20 12:53:51 +00:00
• b69692a2d8 [Kernel] LoRA - Refactor sgmv kernels (#13110) Varun Sundar Rabindranath 2025-02-20 17:58:06 +05:30
• a64a84433d [2/n][ci] S3: Use full model path (#13564) Kevin H. Luu 2025-02-20 01:20:15 -08:00
• aa1e62d0db [ci] Fix spec decode test (#13600) Kevin H. Luu 2025-02-20 00:56:00 -08:00
• 497bc83124 [CI/Build] Use uv in the Dockerfile (#13566) Michael Goin 2025-02-20 02:05:44 -05:00
• 3738e6fa80 [API Server] Add port number range validation (#13506) Yuan Tang 2025-02-20 02:05:13 -05:00
• 0023cd2b9d [ROCm] MI300A compile targets deprecation (#13560) Gregory Shtrasberg 2025-02-20 02:05:00 -05:00
• 041e294716 [Misc] add mm_processor_kwargs to extra_body for Qwen2.5-VL (#13533) 燃 2025-02-20 15:04:30 +08:00
• 9621667874 [Misc] Warn if the vLLM version can't be retrieved (#13501) Alex Brooks 2025-02-19 23:24:48 -07:00
• 8c755c3b6d [bugfix] spec decode worker get tp group only when initialized (#13578) Simon Mo 2025-02-19 20:46:28 -08:00
• ba81163997 [core] add sleep and wake up endpoint and v1 support (#12987) youkaichaoandcennn 2025-02-20 12:41:17 +08:00
• 0d243f2a54 [ROCm][MoE] mi300 mixtral8x7B perf for specific BS (#13577) Divakar Verma 2025-02-19 22:01:02 -06:00
• 88f6ba3281 [ci] Add AWS creds for AMD (#13572) Kevin H. Luu 2025-02-19 19:56:06 -08:00
• 512368e34a [Misc] Qwen2.5 VL support LoRA (#13261) Jee Jee Li 2025-02-20 10:37:55 +08:00
• 473f51cfd9 [3/n][CI] Load Quantization test models with S3 (#13570) Kevin H. LuuandEC2 Default User 2025-02-19 18:12:30 -08:00
• a4c402a756 [BugFix] Avoid error traceback in logs when V1 LLM terminates (#13565) Nick Hill 2025-02-19 16:49:01 -08:00
• 550d97eb58 [Misc] Avoid calling unnecessary hf_list_repo_files for local model path (#13348) Isotr0py 2025-02-20 02:57:48 +08:00
• fbbe1fbac6 [MISC] Logging the message about Ray teardown (#13502) Cody YuandRui Qiao 2025-02-19 09:40:50 -08:00
• 01c184b8f3 Fix copyright year to auto get current year (#13561) Wilson Wu 2025-02-20 00:55:34 +08:00
• ad5a35c21b [doc] clarify multi-node serving doc (#13558) youkaichao 2025-02-19 22:32:17 +08:00
• 5ae9f26a5a [Bugfix] Fix device ordinal for multi-node spec decode (#13269) shangmingc 2025-02-19 22:13:15 +08:00
• 377d10bd14 [VLM][Bugfix] Pass processor kwargs properly on init (#13516) Cyrus Leung 2025-02-19 21:13:50 +08:00
• 52ce14d31f [doc] clarify profiling is only for developers (#13554) youkaichao 2025-02-19 20:55:58 +08:00
• 81dabf24a8 [CI/Build] force writing version file (#13544) Daniele 2025-02-19 11:48:03 +01:00
• 423330263b [Feature] Pluggable platform-specific scheduler (#13161) Yannick Schnider 2025-02-19 10:16:38 +01:00
• caf7ff4456 [V1][Core] Generic mechanism for handling engine utility (#13060) Nick Hill 2025-02-19 01:09:22 -08:00
• f525c0be8b [Model][Speculative Decoding] DeepSeek MTP spec decode (#12755) Lucia FangandLiuXiaoxuanPKU 2025-02-19 01:06:23 -08:00
• 983a40a8bb [Bugfix] Fix Positive Feature Layers in Llava Models (#13514) Alex Brooks 2025-02-19 01:50:07 -07:00
• fdc5df6f54 use device param in load_model method (#13037) Zhe Zhang 2025-02-19 16:05:02 +08:00
• 3b05cd4555 [perf-benchmark] Fix ECR path for premerge benchmark (#13512) Kevin H. LuuandEC2 Default User 2025-02-18 23:56:11 -08:00
• d5d214ac7f [1/n][CI] Load models in CI from S3 instead of HF (#13205) Kevin H. LuuandEC2 Default User 2025-02-18 23:34:59 -08:00
• fd84857f64 [Doc] Add clarification note regarding paligemma (#13511) Roger Wang 2025-02-18 22:24:03 -08:00
• 8aada19dfc [ROCm][MoE configs] mi325 mixtral & mi300 qwen_moe (#13503) Divakar Verma 2025-02-19 00:23:24 -06:00
• 9aa95b0e6a [perf-benchmark] Allow premerge ECR (#13509) Kevin H. LuuandEC2 Default User 2025-02-18 21:13:41 -08:00
• d0a7a2769d [Hardware][Gaudi][Feature] Support Contiguous Cache Fetch (#12139) Yu-ZhouandCody Yu 2025-02-19 11:40:19 +08:00
• 00b69c2d27 [Misc] Remove dangling references to --use-v2-block-manager (#13492) Harry Mellor 2025-02-19 03:37:26 +00:00
• 4c82229898 [V1][Spec Decode] Optimize N-gram matching with Numba (#13365) Woosuk Kwon 2025-02-18 13:19:58 -08:00
• c8d70e2437 Pin Ray version to 2.40.0 (#13490) Woosuk Kwon 2025-02-18 12:50:31 -08:00
• 30172b4947 [V1] Optimize handling of sampling metadata and req_ids list (#13244) Nick Hill 2025-02-18 12:15:33 -08:00
• a4d577b379 [V1][Tests] Adding additional testing for multimodal models to V1 (#13308) Murali Andoorveedu 2025-02-18 09:53:14 -08:00
• 7b203b7694 [misc] fix debugging code (#13487) youkaichao 2025-02-19 01:37:11 +08:00
• 4fb8142a0e [V1][PP] Enable true PP with Ray executor (#13472) Woosuk Kwon 2025-02-18 09:15:32 -08:00
• a02c86b4dd [CI/Build] migrate static project metadata from setup.py to pyproject.toml (#8772) Daniele 2025-02-18 17:02:49 +01:00
• 3809458456 [Bugfix] Fix invalid rotary embedding unit test (#13431) Liangfu Chen 2025-02-18 03:52:03 -08:00
• d3231cb436 [Bugfix] Handle content type with optional parameters (#13383) zifeitong 2025-02-18 03:29:13 -08:00
• 435b502a6e [ROCm] Make amdsmi import optional for other platforms (#13460) Cyrus Leung 2025-02-18 19:15:56 +08:00
• 29fc5772c4 [Bugfix] Remove noisy error logging during local model loading (#13458) Isotr0py 2025-02-18 19:15:48 +08:00
• 2358ca527b [Doc]: Improve feature tables (#13224) Harry Mellor 2025-02-18 10:52:39 +00:00
• 8cf97f8661 [Bugfix] Fix failing transformers dynamic module resolving with spawn multiproc method (#13403) Isotr0py 2025-02-18 18:25:53 +08:00
• e2603fefb8 [Bugfix] Ensure LoRA path from the request can be included in err msg (#13450) Yuan Tang 2025-02-18 03:19:15 -05:00
• b53d79983c Add outlines fallback when JSON schema has enum (#13449) Michael Goin 2025-02-18 01:49:41 -05:00
• 9915912f7f [V1][PP] Fix & Pin Ray version in requirements-cuda.txt (#13436) Woosuk Kwon 2025-02-17 21:58:06 -08:00