Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• c56ed8bb0e [Bugfix][Nixl] Fix full prefix cache hit bug (#18632) Robert ShawandNick Hill 2025-06-04 22:07:32 -04:00
• 78dcf56cb3 [doc] small fix (#19167) Reidandreidliu41 2025-06-05 09:13:50 +08:00
• b2fac67130 [P/D] Heterogeneous TP (#18833) Nicolò Lucchesi 2025-06-05 01:25:34 +02:00
• 23027e2daf [Misc] refactor: simplify EngineCoreClient.make_async_mp_client in AsyncLLM (#18817) CYJiang 2025-06-05 06:37:25 +08:00
• c3fd4d669a [Kernel] Integrate batched/masked deepgemm kernel (#19111) Varun Sundar RabindranathandVarun 2025-06-04 17:59:18 -04:00
• ef3f98b59f [Bugfix] fix v1 cpu worker fails on macOS (#19121) Kebe 2025-06-05 04:17:38 +08:00
• 7ee2590478 [TPU] Update dynamo dump file name in compilation test (#19108) Siyuan Liu 2025-06-04 13:13:43 -07:00
• 53a5a0ce30 [Perf] Tunings for SM100 FP8 CUTLASS kernel (#18778) Michael Goin 2025-06-04 13:46:28 -04:00
• d459fae0a2 [Bugfix][EP+DP] Fix internode check (#19112) Tyler Michael Smith 2025-06-04 11:39:23 -04:00
• c8dcc15921 Allow AsyncLLMEngine.generate to target a specific DP rank (#19102) jmswen 2025-06-04 08:26:47 -07:00
• 8f4ffbd373 [Doc] Update V1 Guide for embedding models (#19141) Cyrus Leung 2025-06-04 22:57:55 +08:00
• 5f2cd251d2 Sm100 blockwise fp8 swap ab (#18564) Lain 2025-06-04 07:48:45 -07:00
• 02658c2dfe Add DeepSeek-R1-0528 function call chat template (#18874) Xu Wenqing 2025-06-04 21:24:18 +08:00
• 01dc9a76db [CI/Build][Bugfix] Ensure compatibility with transformers 4.52 (#18678) Cyrus Leung 2025-06-04 19:49:20 +08:00
• 35cf32df30 Improve the output precision of embedding models (#19092) wang.yuqi 2025-06-04 19:48:57 +08:00
• 8711bc5e68 [Misc] Add packages for benchmark as extra dependency (#19089) Isotr0py 2025-06-04 19:18:48 +08:00
• 2669a0d7b5 Fix ValueError: Missing value for tag key(s): model_name,engine. (#19113) Seiji Eicher 2025-06-04 02:10:45 -07:00
• 8e972d9c44 [TPU] Skip hanging tests (#19115) Siyuan Liu 2025-06-04 01:43:00 -07:00
• 3336c8cfbe Fix #19130 (#19132) 汪志鹏 2025-06-04 16:42:06 +08:00
• b124e1085b [Bugfix] Fix FA3 full cuda graph correctness (#19106) Woosuk Kwon 2025-06-03 23:10:15 -07:00
• 41aa578428 [NVIDIA] Add Cutlass MLA backend (#17625) Kaixi Hou 2025-06-04 12:40:26 +08:00
• 8d646c2e53 [Cleanup][v1]:remote guided-decoding-backend for example (#19059) Calvin Chen 2025-06-04 12:23:26 +08:00
• 5d6d1adf15 [KERNEL] Sampler. CUDA kernel for applying repetition penalty (#18437) Vadim Gimpelson 2025-06-04 08:13:01 +04:00
• 1409ef9134 [Core] Cast multimodal input in hf processor (#18862) Lukas Geiger 2025-06-04 04:24:56 +01:00
• 4555143ea7 [CPU] V1 support for the CPU backend (#16441) Li, Jiang 2025-06-04 09:43:01 +08:00
• 52dceb172d [Docs] Add developer doc about CI failures (#18782) 3 people 2025-06-03 21:09:13 -04:00
• abd7df2fca [Misc] Fix path and python alias errors in disagg_prefill exmaples (#18919) Jiaxin Shan 2025-06-03 17:15:18 -07:00
• b712be98c7 feat: add data parallel rank to KVEventBatch (#18925) Yan Ru Pei 2025-06-03 17:14:20 -07:00
• a8da78eac9 [Bugfix] Max concurrency estimation and check_enough_kv_cache_memory for models with sliding window layers (#19029) Chen Zhang 2025-06-04 08:14:06 +08:00
• 5d96533e22 [Bugfix][P/D] Fix Prefix Cache Bug (#18411) Nicolò LucchesiandRobert Shaw 2025-06-04 01:53:16 +02:00
• 4de790fcad [Bugfix]: Fix the incompatibility issue with tool_choice 'required' when Thinking is enabled (#19075) Chauncey 2025-06-04 07:27:24 +08:00
• b5fd9506c1 [Bugfix] get_num_blocks_to_allocate with null_block (#19031) Chen Zhang 2025-06-04 06:30:55 +08:00
• 135cf55cd1 [V1][Spec Decode][Ngram] 1.35x gain -> 1.95x gain on InstructCoder with prompt fix (#18971) Ekagra Ranjan 2025-06-03 18:26:33 -04:00
• 6cac54f4d1 [v1] Re-init input batch for multiple kv cache groups (#18654) Chen Zhang 2025-06-04 05:41:36 +08:00
• 6865fe0074 Fix interaction between Optional and Annotated in CLI typing (#19093) Harry MellorandYikun Jiang 2025-06-03 22:07:19 +01:00
• e31446b6c8 [Perf] Tune scaled_fp8_quant by increasing vectorization (#18844) Michael Goin 2025-06-03 16:48:25 -04:00
• bdf13965ab [V1] Support cross-layer KV sharing (#18212) Yong Hoon Shin 2025-06-03 13:33:07 -07:00
• fa98d77773 [Kernel] DeepEP dispatch-combine kernel integration (#18434) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-06-03 15:30:02 -04:00
• 01eee40536 [doc] update docker version (#19074) Reidandreidliu41 2025-06-04 03:08:21 +08:00
• 19bdaf32b1 [Doc] Readme standardization (#18695) SorenDreanoandSoren Dreano 2025-06-03 20:50:55 +02:00
• 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) Simon Mo 2025-06-03 11:20:17 -07:00
• d054da1992 [Misc] fix: add miss best_of param validation (#18555) CYJiang 2025-06-04 02:02:07 +08:00
• 4b7817c119 [Misc] Add missing _Backend enums (#19081) Nicolò Lucchesi 2025-06-03 18:15:16 +02:00
• d00dd65cd4 [Doc] Improve the Pull Request template with key components (#19086) Lu Fang 2025-06-03 23:44:34 +08:00
• d81edded69 [Bugfix] disable processor cache (#19068) Raushan Turganbay 2025-06-03 17:06:04 +02:00
• 476844d44c Fix underscores in dict keys passed via CLI (#19030) Harry Mellor 2025-06-03 15:39:24 +01:00
• 4e68ae5e59 [CI/Build] Remove V0 LoRA test (#19066) Jee Jee Li 2025-06-03 22:30:18 +08:00
• 4e88723f32 [doc] clarify windows support (#19088) youkaichao 2025-06-03 21:42:17 +08:00
• 118ff92111 [Doc] Update V1 user guide for embedding and enc-dec models (#19060) Cyrus Leung 2025-06-03 17:29:41 +08:00
• ec2dcd80bc [Misc] Update WeightsMapper for qwen2-vl/qwen2.5-vl (#19054) Isotr0py 2025-06-03 17:08:20 +08:00
• 42243fbda0 [Doc] Add InternVL LoRA support (#19055) Jee Jee Li 2025-06-03 17:08:03 +08:00
• 6d18ed2a2e Update docker docs with ARM CUDA cross-compile (#19037) Michael Goin 2025-06-03 04:21:53 -04:00
• f32fcd9444 [v1][KVCacheManager] Rename BlockHashType to BlockHash (#19015) Chen Zhang 2025-06-03 16:01:48 +08:00
• d32aa2e670 [Bugfix] Use cmake 3.26.1 instead of 3.26 to avoid build failure (#19019) Lu Fang 2025-06-03 15:16:17 +08:00
• cc977286e7 Reduce logs in CLI scripts and plugin loader (#18970) Michael Goin 2025-06-03 02:00:45 -04:00
• 17430e3653 [bugfix] small fix logic issue (#18999) Reidandreidliu41 2025-06-03 13:35:12 +08:00
• 1282bd812e Add tarsier model support (#18985) 汪志鹏 2025-06-03 13:13:13 +08:00
• bdce64f236 [V1] Support DP with Ray (#18779) Rui Qiao 2025-06-02 21:15:13 -07:00
• 9e6f61e8c3 [ROCm][Build] Clean up the ROCm build (#19040) Gregory Shtrasberg 2025-06-02 23:47:47 -04:00
• 8655f47f37 [CPU][CI] Re-enable the CPU CI tests (#19046) Li, Jiang 2025-06-03 11:46:47 +08:00
• 4ce42f9204 Adding "LoRA Test %N" to AMD production tests (#18929) Concurrensee 2025-06-02 22:46:44 -05:00
• 8a57872b2a [Bugfix][EP+DP] Use pplx-kernel internode instead of intranode (#19034) Tyler Michael Smith 2025-06-02 23:36:51 -04:00
• 5bc1ad6cee [Doc] Remove duplicate TOCs during MkDocs migration (#19021) Hyogeun Oh (오효근) 2025-06-03 11:49:48 +09:00
• 9112b443a0 [Hardware][TPU] Initial support of model parallelism with single worker using SPMD (#18011) 3 people 2025-06-02 17:06:20 -07:00
• c57d577e8d add an absolute path for run.sh (#18258) Calvin Chen 2025-06-03 03:38:23 +08:00
• ca2f6b9c30 [Bugfix][Model] Attempt to fix eagle in V0. (#18978) Gregory Shtrasberg 2025-06-02 11:15:53 -04:00
• 20133cfee2 [Frontend] enable custom logging for the uvicorn server (OpenAI API server) (#18403) FrαnçoisandCyrus Leung 2025-06-02 17:04:23 +02:00
• ebb1ec9318 [Model] enable data parallel for Llama4 vision encoder (#18368) 3 people 2025-06-02 04:22:54 -07:00
• 5b168b6d7a [doc] add pytest tips (#19010) Reidandreidliu41 2025-06-02 19:07:26 +08:00
• 9760fd8f6a [Core] Support inplace model weights loading (#18745) 22quinn 2025-06-02 02:38:50 -07:00
• b9f61e1387 [Bugfix][Nixl] Fix DP Metadata Handshake (#19008) Robert Shaw 2025-06-01 23:30:41 -04:00
• d6fd3a33b8 [Misc] reuse num_tokens_across_dp of get_dp_padding to avoid unnecessary dp all reduce in set_forward_context (#18935) 3 people 2025-06-02 03:41:18 +08:00
• 432ec9926e [doc] wrong output (#19000) Reidandreidliu41 2025-06-01 19:26:14 +08:00
• 2b102d51ad [BugFix] Fix incorrect metrics shutdown error log message (#18992) Nick Hill 2025-05-31 20:42:23 -07:00
• aa54a7bf7b [BugFix] fix data parallel construct ipv6 url addres (#18991) rongfu.leng 2025-06-01 11:42:10 +08:00
• 2ad6194a02 Let max_num_batched_tokens use human_readable_int for large numbers (#18968) Michael Goin 2025-05-31 23:41:29 -04:00
• c594cbf565 [doc] small fix - mkdocs (#18996) Reidandreidliu41 2025-06-01 11:23:43 +08:00
• a35ca765a5 [LoRA] Support dynamically initialize packed_modules_mapping for VLM with arbitrary components (#18987) Isotr0py 2025-06-01 11:06:57 +08:00
• 6aa8f9a4e7 [Core] Rework dtype resolution (#18751) Cyrus Leung 2025-06-01 11:04:23 +08:00
• 1bc86a3da1 [Bugfix] Fix EAGLE3 broken logits (#18909) Benjamin Chislett 2025-05-31 22:58:07 -04:00
• bbfa0c61d1 [Misc][Benchmark] Add support for CustomDataset (#18511) Ekagra Ranjan 2025-05-31 15:07:38 -04:00
• 20079c6e36 [Misc] add return token strs for tokenize (#18941) Reidandreidliu41 2025-06-01 02:00:11 +08:00
• 9a1b9b99d7 [BugFix] Fix multi-node offline data-parallel (#18981) Nick HillandYizhou Liu 2025-05-31 08:34:52 -07:00
• 8bf507d766 [P/D] NixlConnector use cache device index for memory registration (#18969) ptarasiewiczNV 2025-05-31 17:19:18 +02:00
• 306d60401d [ROCm][Kernel] Add gfx950 support for skinny gemms (#18010) Charlie Fu 2025-05-31 09:40:05 -05:00
• f2c3f66d59 [Bugfix] Fix for issue 17396 (#18773) Fred Reiss 2025-05-31 04:58:17 -07:00
• 0f5e0d567e [FEAT][ROCm] Add AITER grouped topk for DeepSeekV2 (#18825) vllmellm 2025-05-31 18:39:31 +08:00
• c55d804672 [BugFix] Pydantic part 2 (#18911) Luka Govedič 2025-05-31 06:39:28 -04:00
• 749f5bdd38 [doc] fix the list rendering issue - security.md (#18982) Reidandreidliu41 2025-05-31 18:39:21 +08:00
• 2a50ef5760 [Neuron] Add Multi-Modal model support for Neuron (#18921) 5 people 2025-05-31 03:39:11 -07:00
• b8b904795d fix security issue of logging llm output (#18980) Lucia FangandLucia Fang 2025-05-31 03:38:56 -07:00
• ba5111f237 [Bugfix]: Fix the incompatibility issue with Structured Outputs when Thinking is disabled (#18879) Chauncey 2025-05-31 17:20:54 +08:00
• 1e123529d7 [Misc] Fix estimated max model len msg (#18966) Yong Hoon Shin 2025-05-31 01:43:44 -07:00
• dff80b0e42 [Frontend] Add rerank support to run_batch endpoint (#16278) Pooya Davoodi 2025-05-31 00:40:01 -07:00
• 7782464a17 create util function for batched arange (#18937) Yu Guo 2025-05-30 22:50:38 -07:00
• 0f71e24034 [Docs] Correct multiprocessing design doc (#18964) Lukas Geiger 2025-05-31 02:30:15 +01:00
• 1dab4d5718 Tool parser regex timeout handling (#18960) Will Eaton 2025-05-30 17:02:54 -04:00
• 7f21e8052b [Misc] add group_size is -1 in awq quantization (#18910) rongfu.leng 2025-05-31 01:34:22 +08:00
• 5a8641638a [VLM] Add PP support and fix GPTQ inference for Ovis models (#18958) Isotr0py 2025-05-31 01:11:44 +08:00
• f49239cb45 Benchmark script for fp8 vs bf16 gemm (#17126) Michael Goin 2025-05-30 12:56:11 -04:00