Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 222a9dc350 [Benchmark] More accurate TPOT calc in benchmark_serving.py (#12288) Nick Hill 2025-01-21 21:46:14 -08:00
• cbdc4ad5a5 [Ci/Build] Fix mypy errors on main (#12296) Cyrus Leung 2025-01-22 12:06:54 +08:00
• 016e3676e7 [CI] add docker volume prune to neuron CI (#12291) Liangfu Chen 2025-01-21 18:47:49 -08:00
• 64ea24d0b3 [ci/lint] Add back default arg for pre-commit (#12279) Kevin H. Luu 2025-01-21 17:15:27 -08:00
• df76e5af26 [VLM] Simplify post-processing of replacement info (#12269) Cyrus Leung 2025-01-22 08:48:13 +08:00
• 09ccc9c8f7 [Documentation][AMD] Add information about prebuilt ROCm vLLM docker for perf validation purpose (#12281) Hongxia Yang 2025-01-21 18:49:22 -05:00
• 69196a9bc7 [BUGFIX] When skip_tokenize_init and multistep are set, execution crashes (#12277) Aleksandr Malyshevandmaleksan85 2025-01-21 15:30:46 -08:00
• 2acba47d9b [bugfix] moe tuning. rm is_navi() (#12273) Divakar Verma 2025-01-21 16:47:32 -06:00
• 9c485d9e25 [Core] Free CPU pinned memory on environment cleanup (#10477) Jani Monoses 2025-01-21 21:56:41 +02:00
• fa9ee08121 [Misc] Set default backend to SDPA for get_vit_attn_backend (#12235) wangxiyuan 2025-01-22 03:52:11 +08:00
• 347eeebe3b [Misc] Remove experimental dep from tracing.py (#12007) Adrian Cole 2025-01-21 11:51:55 -08:00
• 18fd4a8331 [Bugfix] Multi-sequence broken (#11898) Andy Lo 2025-01-21 19:51:35 +00:00
• 132a132100 [v1][stats][1/n] Add RequestStatsUpdate and RequestStats types (#10907) Ricky Xu 2025-01-21 11:51:13 -08:00
• 1e60f87bb3 [Kernel] fix moe_align_block_size error condition (#12239) Jinzhen Lin 2025-01-22 02:30:28 +08:00
• 9705b90bcf [Bugfix] fix race condition that leads to wrong order of token returned (#10802) Jannis Schönleber 2025-01-21 18:47:04 +01:00
• 3aec49e56f [ci/build] update nightly torch for gh200 test (#12270) youkaichao 2025-01-21 23:03:17 +08:00
• c64612802b [Platform] improve platforms getattr (#12264) Mengqing Cao 2025-01-21 22:42:41 +08:00
• 9a7c3a0042 Remove pytorch comments for outlines + compressed-tensors (#12260) Thomas Parnell 2025-01-21 14:49:08 +01:00
• b197a5ccfd [V1][Bugfix] Fix data item ordering in mixed-modality inference (#12259) Roger Wang 2025-01-21 05:18:43 -08:00
• c81081fece [torch.compile] transparent compilation with more logging (#12246) youkaichao 2025-01-21 19:32:55 +08:00
• a94eee4456 [Bugfix] Fix mm_limits access for merged multi-modal processor (#12252) Cyrus Leung 2025-01-21 18:09:39 +08:00
• f2e9f2a3be [Misc] Remove redundant TypeVar from base model (#12248) Cyrus Leung 2025-01-21 16:40:39 +08:00
• 1f1542afa9 [Misc]Add BNB quantization for PaliGemmaForConditionalGeneration (#12237) Jee Jee Li 2025-01-21 15:49:08 +08:00
• 96912550c8 [Misc] Rename MultiModalInputsV2 -> MultiModalInputs (#12244) Cyrus Leung 2025-01-21 15:31:19 +08:00
• 2fc6944c5e [ci/build] disable failed and flaky tests (#12240) youkaichao 2025-01-21 13:25:03 +08:00
• 5fe6bf29d6 [BugFix] Fix GGUF tp>1 when vocab_size is not divisible by 64 (#12230) Nicolò Lucchesi 2025-01-21 05:23:14 +01:00
• d4b62d4641 [AMD][Build] Porting dockerfiles from the ROCm/vllm fork (#11777) Gregory Shtrasberg 2025-01-20 23:22:23 -05:00
• ecf67814f1 Add quantization and guided decoding CODEOWNERS (#12228) Michael Goin 2025-01-20 20:23:40 -05:00
• 750f4cabfa [Kernel] optimize moe_align_block_size for cuda graph and large num_experts (e.g. DeepSeek-V3) (#12222) 3 people 2025-01-21 08:42:16 +08:00
• 06a760d6e8 [bugfix] catch xgrammar unsupported array constraints (#12210) Cheng Kuan Yong Jason 2025-01-21 08:42:02 +08:00
• da7512215f [misc] add cuda runtime version to usage data (#12190) youkaichaoandRoger Wang 2025-01-21 08:31:01 +08:00
• af69a6aded fix: update platform detection for M-series arm based MacBook processors (#12227) Işık 2025-01-20 22:23:28 +00:00
• 7bd3630067 [Misc] Update CODEOWNERS (#12229) Roger Wang 2025-01-20 14:19:09 -08:00
• 96663699b2 [CI] Pass local python version explicitly to pre-commit mypy.sh (#12224) Chen Zhang 2025-01-20 23:49:18 +08:00
• 18572e3384 [Bugfix] Fix HfExampleModels.find_hf_info (#12223) Cyrus Leung 2025-01-20 23:35:36 +08:00
• 86bfb6dba7 [Misc] Pass attention to impl backend (#12218) wangxiyuan 2025-01-20 23:25:28 +08:00
• 5f0ec3935a [V1] Remove _get_cache_block_size (#12214) Chen Zhang 2025-01-20 21:54:16 +08:00
• c222f47992 [core][bugfix] configure env var during import vllm (#12209) youkaichao 2025-01-20 19:35:59 +08:00
• 170eb35079 [misc] print a message to suggest how to bypass commit hooks (#12217) youkaichao 2025-01-20 18:06:24 +08:00
• b37d82791e [Model] Upgrade Aria to transformers 4.48 (#12203) Cyrus Leung 2025-01-20 17:58:48 +08:00
• 3127e975fb [CI/Build] Make pre-commit faster (#12212) Cyrus Leung 2025-01-20 17:36:24 +08:00
• 4001ea1266 [CI/Build] Remove dummy CI steps (#12208) Cyrus Leung 2025-01-20 16:41:57 +08:00
• 5c89a29c22 [misc] add placeholder format.sh (#12206) youkaichao 2025-01-20 16:04:49 +08:00
• 59a0192fb9 [Core] Interface for accessing model from VllmRunner (#10353) Cyrus Leung 2025-01-20 15:00:59 +08:00
• 83609791d2 [Model] Add Qwen2 PRM model support (#12202) Isotr0py 2025-01-20 14:59:46 +08:00
• 0974c9bc5c [Bugfix] Fix incorrect types in LayerwiseProfileResults (#12196) Yuan Tang 2025-01-20 01:59:20 -05:00
• d2643128f7 [DOC] Add missing docstring in LLMEngine.add_request() (#12195) Yuan Tang 2025-01-20 01:59:00 -05:00
• c5c06209ec [DOC] Fix typo in docstring and assert message (#12194) Yuan Tang 2025-01-20 01:58:29 -05:00
• 3ea7b94523 Move linting to pre-commit (#11975) Harry Mellor 2025-01-20 06:58:01 +00:00
• 51ef828f10 [torch.compile] fix sym_tensor_indices (#12191) youkaichao 2025-01-20 11:37:50 +08:00
• df450aa567 [Bugfix] Fix num_heads value for simple connector when tp enabled (#12074) shangmingc 2025-01-20 10:56:43 +08:00
• bbe5f9de7d [Model] Support for fairseq2 Llama (#11442) Martin Gleizeandmgleize user 2025-01-19 19:40:40 +01:00
• 81763c58a0 [V1] Add V1 support of Qwen2-VL (#12128) 3 people 2025-01-19 03:52:13 -08:00
• edaae198e7 [Misc] Add BNB support to GLM4-V model (#12184) Isotr0py 2025-01-19 19:49:22 +08:00
• 936db119ed benchmark_serving support --served-model-name param (#12109) gujingandRoger Wang 2025-01-19 17:59:56 +08:00
• e66faf4809 [torch.compile] store inductor compiled Python file (#12182) youkaichao 2025-01-19 16:27:26 +08:00
• 630eb5b5ce [Bugfix] Fix multi-modal processors for transformers 4.48 (#12187) Cyrus Leung 2025-01-19 11:16:34 +08:00
• 4e94951bb1 [BUGFIX] Move scores to float32 in case of running xgrammar on cpu (#12152) Michal Adamczyk 2025-01-19 04:12:05 +01:00
• 7a8a48d51e [V1] Collect env var for usage stats (#12115) Simon Mo 2025-01-18 19:07:15 -08:00
• 32eb0da808 [Misc] Support register quantization method out-of-tree (#11969) yancong 2025-01-19 08:13:16 +08:00
• 6d0e3d3724 [core] clean up executor class hierarchy between v1 and v0 (#12171) youkaichao 2025-01-18 14:35:15 +08:00
• 02798ecabe [Model] Port deepseek-vl2 processor, remove dependency (#12169) Isotr0py 2025-01-18 13:59:39 +08:00
• 813f249f02 [Docs] Fix broken link in SECURITY.md (#12175) Russell Bryant 2025-01-17 23:35:21 -05:00
• da02cb4b27 [core] further polish memory profiling (#12126) youkaichao 2025-01-18 12:25:08 +08:00
• c09503ddd6 [AMD][CI/Build][Bugfix] use pytorch stale wheel (#12172) Hongxia Yang 2025-01-17 22:15:53 -05:00
• 2b83503227 [misc] fix cross-node TP (#12166) youkaichao 2025-01-18 10:53:27 +08:00
• 7b98a65ae6 [torch.compile] disable logging when cache is disabled (#12043) youkaichao 2025-01-18 04:29:31 +08:00
• b5b57e301e [AMD][FP8] Using MI300 FP8 format on ROCm for block_quant (#12134) Gregory Shtrasberg 2025-01-17 12:12:26 -05:00
• 54cacf008f [Bugfix] Mistral tokenizer encode accept list of str (#12149) Kunshang Ji 2025-01-18 00:47:53 +08:00
• 58fd57ff1d [Bugfix] Fix score api for missing max_model_len validation (#12119) Wallas Henrique 2025-01-17 13:24:22 -03:00
• 87a0c076af [core] allow callable in collective_rpc (#12151) youkaichao 2025-01-17 20:47:01 +08:00
• d4e6194570 [CI/Build][CPU][Bugfix] Fix CPU CI (#12150) Li, Jiang 2025-01-17 19:39:52 +08:00
• 07934cc237 [Misc][LoRA] Improve the readability of LoRA error messages (#12102) Jee Jee Li 2025-01-17 19:32:28 +08:00
• 69d765f5a5 [V1] Move more control of kv cache initialization from model_executor to EngineCore (#11960) Chen ZhangandCody Yu 2025-01-17 15:39:35 +08:00
• 8027a72461 [ROCm][MoE] moe tuning support for rocm (#12049) Divakar Verma 2025-01-17 00:49:16 -06:00
• d75ab55f10 [Misc] Add deepseek_vl2 chat template (#12143) Isotr0py 2025-01-17 14:34:48 +08:00
• d1adb9b403 [BugFix] add more is not None check in VllmConfig.__post_init__ (#12138) Chen Zhang 2025-01-17 13:33:22 +08:00
• b8bfa46a18 [Bugfix] Fix issues in CPU build Dockerfile (#12135) Yuan Tang 2025-01-16 23:54:01 -05:00
• 1475847a14 [Doc] Add instructions on using Podman when SELinux is active (#12136) Yuan Tang 2025-01-16 23:45:36 -05:00
• fead53ba78 [CI]add genai-perf benchmark in nightly benchmark (#10704) Kunshang Ji 2025-01-17 12:15:09 +08:00
• ebc73f2828 [Bugfix] Fix a path bug in disaggregated prefill example script. (#12121) Kuntai Du 2025-01-17 11:12:41 +08:00
• d06e824006 [Bugfix] Set enforce_eager automatically for mllama (#12127) Chen Zhang 2025-01-17 04:30:08 +08:00
• 62b06ba23d [Model] Add support for deepseek-vl2-tiny model (#12068) Isotr0py 2025-01-17 01:14:48 +08:00
• 5fd24ec02e [misc] Add LoRA kernel micro benchmarks (#11579) Varun Sundar Rabindranath 2025-01-16 21:21:40 +05:30
• 874f7c292a [Bugfix] Fix max image feature size for Llava-one-vision (#12104) Roger Wang 2025-01-16 06:54:06 -08:00
• 92e793d91a [core] LLM.collective_rpc interface and RLHF example (#12084) youkaichao 2025-01-16 20:19:52 +08:00
• bf53e0c70b Support torchrun and SPMD-style offline inference (#12071) youkaichao 2025-01-16 19:58:53 +08:00
• dd7c9ad870 [Bugfix] Remove hardcoded head_size=256 for Deepseek v2 and v3 (#12067) Isotr0py 2025-01-16 18:11:54 +08:00
• 9aa1519f08 Various cosmetic/comment fixes (#12089) Michael Goin 2025-01-16 04:59:06 -05:00
• f8ef146f03 [Doc] Add documentation for specifying model architecture (#12105) Cyrus Leung 2025-01-16 15:53:43 +08:00
• fa0050db08 [Core] Default to using per_token quantization for fp8 when cutlass is supported. (#8651) 3 people 2025-01-15 20:31:27 -08:00
• cd9d06fb8d Allow hip sources to be directly included when compiling for rocm. (#12087) tvirolai-amd 2025-01-15 23:46:03 +02:00
• ebd8c669ef [Bugfix] Fix _get_lora_device for HQQ marlin (#12090) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-01-16 01:29:42 +05:30
• 70755e819e [V1][Core] Autotune encoder cache budget (#11895) Roger Wang 2025-01-15 11:29:00 -08:00
• edce722eaa [Bugfix] use right truncation for non-generative tasks (#12050) Joe Runde 2025-01-15 09:31:01 -07:00
• 57e729e874 [Doc]: Update OpenAI-Compatible Server documents (#12082) maang-h 2025-01-16 00:07:45 +08:00
• de0526f668 [Misc][Quark] Upstream Quark format to VLLM (#10765) 3 people 2025-01-16 00:05:15 +08:00
• 5ecf3e0aaf Misc: allow to use proxy in HTTPConnection (#12042) Yuan 2025-01-15 21:16:40 +08:00
• 97eb97b5a4 [Model]: Support internlm3 (#12037) RunningLeon 2025-01-15 19:35:17 +08:00
• 3adf0ffda8 [Platform] Do not raise error if _Backend is not found (#12023) wangxiyuanandMengqing Cao 2025-01-15 18:14:15 +08:00