Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• b9e41734c5 [Bugfix][Disaggregated] patch the inflight batching on the decode node in SimpleConnector to avoid hangs in SimpleBuffer (nccl based) (#13987) Mathis Felardos 2025-02-28 08:53:45 +01:00
• 1088f06242 [Doc] Move multimodal Embedding API example to Online Serving page (#14017) Cyrus Leung 2025-02-28 15:12:04 +08:00
• 73e0225ee9 [Bugfix] Check that number of images matches number of <|image|> tokens with mllama (#13911) Travis Johnson 2025-02-27 21:00:45 -07:00
• 6c85da3a18 [V1]SupportsV0Only protocol for model definitions (#13959) Roger Wang 2025-02-27 17:02:15 -08:00
• 67fc426845 [Misc] Print FusedMoE detail info (#13974) Jee Jee Li 2025-02-28 07:53:13 +08:00
• 9804145cac [Model][Speculative Decoding] Expand DeepSeek MTP code to support k > n_predict (#13626) Benjamin Chislett 2025-02-27 18:28:08 -05:00
• 2e94b9cfbb [Attention] Flash MLA for V1 (#13867) Lucas WilkinsonandYang Chen 2025-02-27 18:03:41 -05:00
• 8294773e48 [core] Perf improvement for DSv3 on AMD GPUs (#13718) qli88 2025-02-27 16:14:30 -06:00
• cd813c6d4d [V1][Minor] Minor cleanup for GPU Model Runner (#13983) Woosuk Kwon 2025-02-27 13:11:40 -08:00
• 38acae6e97 [ROCm] Fix the Kernels, Core, and Prefix Caching AMD CI groups (#13970) Sage Moore 2025-02-27 12:31:47 -08:00
• a2dd48c386 [VLM] Deprecate legacy input mapper for OOT multimodal models (#13979) Cyrus Leung 2025-02-28 03:14:55 +08:00
• 126f6beeb4 Bump azure/setup-helm from 4.2.0 to 4.3.0 (#13742) dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 2025-02-27 19:04:10 +00:00
• 58d1b2aa77 [Attention] MLA support for V1 (#13789) Yang Chen 2025-02-27 10:14:17 -08:00
• f1579b229d [VLM] Generalized prompt updates for multi-modal processor (#13964) Cyrus Leung 2025-02-28 01:44:25 +08:00
• 7864875879 [Bugfix] Fix qwen2.5-vl overflow issue (#13968) Isotr0py 2025-02-28 01:30:39 +08:00
• 1dd422b64a Update LMFE version to v0.10.11 to support new versions of transforme… (#13930) Noam Gat 2025-02-27 19:16:12 +02:00
• 06c8f8d885 [bugfix] Fix profiling for RayDistributedExecutor (#13945) Rui Qiao 2025-02-27 09:01:21 -08:00
• 5677c9bb3e Deduplicate .pre-commit-config.yaml's exclude (#13967) Harry Mellor 2025-02-27 16:27:47 +00:00
• 512d77d582 Update quickstart.md (#13958) 王博伟 2025-02-28 00:05:11 +08:00
• 7f0be2aa24 [Model] Deepseek GGUF support (#13167) Szymon Ożóg 2025-02-27 11:08:35 +01:00
• edf309ebbe [VLM] Support multimodal inputs for Florence-2 models (#13320) Isotr0py 2025-02-27 18:06:41 +08:00
• 788f284b53 Fix test_block_fp8.py test for MoE (#13915) Michael Goin 2025-02-27 05:00:00 -05:00
• 4b1d141f49 [PP] Correct cache size check (#13873) Yang Zheng 2025-02-27 17:47:29 +08:00
• 10c3b8c1cf [Misc] fixed 'required' is an invalid argument for positionals (#13948) Chauncey 2025-02-27 17:06:49 +08:00
• a7f37314b7 [CI/Build] Add examples/ directory to be labelled by mergify (#13944) Brayden Zhong 2025-02-27 03:24:11 -05:00
• cd711c48b2 [V1][Metrics] Handle preemptions (#13169) Mark McLoughlin 2025-02-27 04:04:59 +00:00
• 378b3ef6f8 [ROCm][V1] Update reshape_and_cache to properly work with CUDA graph padding (#13922) Sage Moore 2025-02-26 20:04:12 -08:00
• c9944acbf9 [misc] Rename Ray ADAG to Compiled Graph (#13928) Rui Qiao 2025-02-26 20:03:28 -08:00
• ca377cf1b9 Use CUDA 12.4 as default for release and nightly wheels (#12098) Michael Goin 2025-02-26 22:06:37 -05:00
• a31614e386 [ROCm][Quantization][Kernel] Use FP8 FNUZ when OCP flag is 0 or undefined (#13851) ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 2025-02-27 04:39:10 +02:00
• f95903909f [Kernel] FlashMLA integration (#13747) Lucas Wilkinson 2025-02-26 21:35:08 -05:00
• b382a7f28f [BugFix] Make FP8 Linear compatible with torch.compile (#13918) Woosuk Kwon 2025-02-26 13:48:55 -08:00
• 4cb6fa0a9c [Bugfix] Backend option to disable xgrammar any_whitespace (#12744) Wallas HenriqueandJoe Runde 2025-02-26 15:52:34 -03:00
• d08b285adf [Misc] fixed qwen_vl_utils parameter error (#13906) Chauncey 2025-02-27 00:31:53 +08:00
• b27122acc2 [TPU] use torch2.6 with whl package (#13860) Chenyaaang 2025-02-26 05:18:54 -08:00
• 934bb99c71 [Bugfix] Update expected token counts for Ultravox tests (#13895) Cyrus Leung 2025-02-26 20:56:50 +08:00
• 3f808cc044 [Bugfix] Do not crash V0 engine on input errors (#13101) Joe Runde 2025-02-26 04:07:29 -07:00
• ec8a5e5386 [Misc]: Add support for goodput on guided benchmarking + TPOT calculation refactor (#13736) Brayden Zhong 2025-02-26 06:06:47 -05:00
• 215bf150a6 [Bugfix] Handle None parameters in Mistral function calls. (#13786) Florian Greinacher 2025-02-26 12:06:21 +01:00
• 0ecdd98031 Add comments on accessing kv_cache and attn_metadata (#13887) Harry Mellor 2025-02-26 10:41:02 +00:00
• 7b700ec8c8 [Bugfix] Add test example for Ultravox v0.5 (#13890) Cyrus Leung 2025-02-26 18:31:43 +08:00
• 7ca1da020f [Misc] Fix input processing for Ultravox (#13871) Roger Wang 2025-02-25 23:56:34 -08:00
• 5157338ed9 [Misc] Improve LoRA spelling (#13831) Jee Jee Li 2025-02-26 15:43:01 +08:00
• e206b54331 [v0][Core] Use xgrammar shared context to avoid copy overhead for offline engine (#13837) Seth Kimmel 2025-02-25 22:58:24 -08:00
• 1d35662e6d [ROCm] Disable chunked prefill/prefix caching when running MLA on non-cuda platforms (#13844) Sage Moore 2025-02-25 22:56:58 -08:00
• e656f638de [Doc] fix the incorrect module path of tensorize_vllm_model (#13863) Albert 2025-02-26 14:56:19 +08:00
• 145944cb94 Improve pipeline partitioning (#13839) Harry Mellor 2025-02-26 02:53:56 +00:00
• 094b7d9496 [Kernel][Build/CI] Bump CUTLASS to 3.8 and add initializers for cutlass epilogues (#13797) Henry Tsang 2025-02-25 18:52:03 -08:00
• e1fe7591f2 [Misc]Code Cleanup (#13859) Chenguang Liandnoemotiovon 2025-02-26 10:44:30 +08:00
• 5629f26df7 [V1][Spec Decode] Change Spec Decode Rejection Sampling API (#13729) Lily Liu 2025-02-25 18:14:48 -08:00
• 9ba28043b5 [misc] Show driver IP info when Ray fails to allocate driver worker (#13858) Rui Qiao 2025-02-25 17:53:43 -08:00
• 24679788ed DeepSeek V2/V3/R1 only place lm_head on last pp rank (#13833) Harry Mellor 2025-02-26 01:24:57 +00:00
• 07c4353057 [Model] Support Grok1 (#13795) Michael Goin 2025-02-25 20:07:12 -05:00
• 34e3494e70 Fix failing MyGemma2Embedding test (#13820) Harry Mellor 2025-02-25 20:33:03 +00:00
• f75aa72732 [Neuron] Add custom_ops for neuron backend (#13246) 3 people 2025-02-25 11:47:49 -08:00
• 340e39e387 Fix string parsing error (#13825) Chen1022 2025-02-26 00:20:29 +08:00
• f4133ce4e5 [Bugfix] Revert inspection code in #13743 (#13832) Cyrus Leung 2025-02-26 00:18:50 +08:00
• 6522d55b6f Fix /v1/audio/transcriptions Bad Request Error (#13811) Wen Sun 2025-02-25 22:03:33 +08:00
• 6ff518626c [Bugfix] Fix deepseek-vl2 inference with more than 2 images (#13818) Isotr0py 2025-02-25 22:03:02 +08:00
• fa82074167 [Bugfix] Flush TunableOp results before worker processes are destroyed. (#13623) Nichols A. Romero 2025-02-25 05:08:20 -06:00
• 75e9d49796 [Bugfix] Initialize attention bias on the same device as Query/Key/Value (#13468) Junlin Zhou 2025-02-25 18:13:09 +08:00
• 32c3b6bfd1 [Misc]Clarify Error Handling for Non-existent Model Paths and HF Repo IDs (#13724) Chen1022andMichael Goin 2025-02-25 18:12:19 +08:00
• 37b6cb4985 [CI/Build] Fix V1 LoRA failure (#13767) Jee Jee Li 2025-02-25 18:01:15 +08:00
• aabeb2688f [ROCm][Quantization][Kernel] Using HIP FP8 header (#12593) Gregory Shtrasberg 2025-02-25 03:39:59 -05:00
• 2f42a4888c [Feature] Support KV cache offloading and disagg prefill with LMCache connector. (#12953) Jiayi Yao 2025-02-25 02:38:42 -06:00
• 3173c3b34e [misc] Clean up ray compiled graph type hints (#13731) Rui Qiao 2025-02-25 00:37:08 -08:00
• 2d87d7d1ac [Bugfix] Modify modelscope api usage in transformer_utils (#13807) Shanshan Shen 2025-02-25 16:36:07 +08:00
• aab392774b [Core] xgrammar: Expand list of unsupported jsonschema keywords (#13783) Russell Bryant 2025-02-25 03:21:25 -05:00
• 6724e79164 [Misc] Check that the model can be inspected upon registration (#13743) Cyrus Leung 2025-02-25 16:18:19 +08:00
• 03f48b3db6 [Core] LoRA V1 - Add add/pin/list/remove_lora functions (#13705) Varun Sundar Rabindranath 2025-02-25 13:48:02 +05:30
• 4d251ad00e Fix CompressedTensorsWNA16MoE with grouped scales (#13769) Michael Goin 2025-02-25 03:17:14 -05:00
• 18e505930d [Bugfix] Support MLA for CompressedTensorsWNA16 (#13725) Michael Goin 2025-02-25 01:10:31 -05:00
• 4a8cfc7551 [Bugfix] Fix deepseek-v2 error: "missing 1 required positional argument: 'residual'" (#13802) Lucas Wilkinson 2025-02-24 23:33:59 -05:00
• bc32bc73aa [V1][Metrics] Implement vllm:lora_requests_info metric (#13504) Mark McLoughlin 2025-02-25 04:01:33 +00:00
• ab1091d5f2 [Misc][Attention][Quantization] init property earlier (#13733) wangxiyuan 2025-02-25 11:19:30 +08:00
• 1e15aaef56 [Bugfix][Quantization] Fix FP8 + EP (#13784) Tyler Michael Smith 2025-02-24 21:54:17 -05:00
• 51010a1807 [Misc] set single whitespace between log sentences (#13771) cjackal 2025-02-25 11:26:12 +09:00
• 7196a3b1db [Doc] arg_utils.py: fixed a typo (#13785) Eli Boyarski 2025-02-25 04:23:04 +02:00
• cdc1fa12eb Remove unused kwargs from model definitions (#13555) Harry Mellor 2025-02-25 01:13:52 +00:00
• f61528d46d [Misc][Chore] Clean Up AsyncOutputProcessing Logs (#13780) Robert Shaw 2025-02-24 19:39:07 -05:00
• 1f0ae3ed0a [Misc] Clean Up EngineArgs.create_engine_config (#13734) Robert Shaw 2025-02-24 13:52:21 -05:00
• db986c19ea Fix precommit fail in fused_moe intermediate_cache2 chunking (#13772) Michael Goin 2025-02-24 12:25:47 -05:00
• 227578480d Revert "[V1][Core] Fix memory issue with logits & sampling" (#13775) Roger Wang 2025-02-24 09:16:05 -08:00
• befc402d34 [V1] V1 engine implements parallel sampling (AsyncLLM and LLMEngine) (#10980) afeldman-nmandNick Hill 2025-02-24 11:29:41 -05:00
• 444b0f0f62 [Misc][Docs] Raise error when flashinfer is not installed and VLLM_ATTENTION_BACKEND is set (#12513) Nicolò Lucchesi 2025-02-24 16:43:21 +01:00
• ccc00515fd [BugFix] Illegal memory access for MoE On H20 (#13693) Zhonghua Deng 2025-02-24 23:37:32 +08:00
• 781096e385 Expert Parallelism (EP) Support for DeepSeek V2 (#12583) Jongseok Park 2025-02-24 07:33:20 -08:00
• 7940d8a6a7 [CI/Build] add python-json-logger to requirements-common (#12842) Roger Meier 2025-02-24 15:10:33 +01:00
• c0e3ecd6d2 [Bugfix] fix(logging): add missing opening square bracket (#13011) Roger Meier 2025-02-24 15:10:25 +01:00
• 23eca9cf68 [model][refactor] remove cuda hard code in models and layers (#13658) Mengqing Cao 2025-02-24 22:10:14 +08:00
• 437b76ff59 [V1][Core] Fix memory issue with logits & sampling (#13721) Roger Wang 2025-02-24 06:10:06 -08:00
• f90a375593 [ci] Add logic to change model to S3 path only when S3 CI env var is on (#13727) Kevin H. LuuandEC2 Default User 2025-02-23 22:32:11 -08:00
• e7ef74e26e Fix some issues with benchmark data output (#13641) Huy Do 2025-02-23 18:23:18 -08:00
• cbae7af552 [V1][BugFix] Fix engine core client shutdown hangs (#13298) Nick Hill 2025-02-23 13:07:43 -08:00
• eb24dc4a45 [v1] torchrun compatibility (#13642) youkaichao 2025-02-23 22:47:24 +08:00
• 9bebc9512f [Misc] Deprecate --dataset from benchmark_serving.py (#13708) Roger Wang 2025-02-23 05:32:20 -08:00
• 5a2ba16f5c [Core][Distributed] Use IPC (domain socket) ZMQ socket for local comms (#13688) Nick Hill 2025-02-23 02:54:29 -08:00
• ba5106e519 [LMM] Implement merged multimodal processor for whisper (#13278) Isotr0py 2025-02-23 17:46:03 +08:00
• d5ca2110f1 [Quant] BaiChuan SupportsQuant (#13710) Kyle Sayers 2025-02-22 22:21:15 -05:00
• 2c5e637b57 [ci] Use env var to control whether to use S3 bucket in CI (#13634) Kevin H. Luu 2025-02-22 19:19:45 -08:00