Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 0408efc6d0 [Misc] Improve error message for incorrect pynvml (#12809) youkaichao 2025-02-06 15:23:50 +08:00
• 449d1bce02 [Misc] Remove duplicated DeepSeek V2/V3 model definition (#12793) Michael Goin 2025-02-06 02:16:20 -05:00
• 1a6fcad4c9 Improve TransformersModel UX (#12785) Harry Mellor 2025-02-06 06:24:57 +00:00
• 56534cd577 [Bugfix] Fix the test_ultravox.py's license (#12806) Lu Fang 2025-02-05 21:25:54 -08:00
• d88506dda4 [Model] LoRA Support for Ultravox model (#11253) Sumit Vij 2025-02-05 19:54:13 -08:00
• 9cdea30b4f [Misc][Easy] Remove the space from the file name Lu Fang 2025-02-05 19:23:35 -08:00
• 76abd0c881 [Bugfix] Better FP8 supported defaults Lucas Wilkinson 2025-02-05 22:22:19 -05:00
• 5b19b93082 [ROCm][Kernel] Using the correct warp_size value Gregory Shtrasberg 2025-02-05 22:15:08 -05:00
• 75404d041b [VLM] Update compatibility with transformers 4.49 Cyrus Leung 2025-02-06 11:09:45 +08:00
• bf3b79efb8 [VLM] Qwen2.5-VL Roger Wang 2025-02-05 13:31:38 -08:00
• 9a5b1554b4 [Docs] Drop duplicate [source] links Russell Bryant 2025-02-05 16:30:50 -05:00
• a4ce74c14a [VLM] Use shared field to pass token ids to model Cyrus Leung 2025-02-06 05:30:46 +08:00
• 3b2005e1db Add: Support for Sparse24Bitmask Compressed Models Rahul Tuli 2025-02-05 15:30:43 -06:00
• af8486de49 [Hardware][Intel-Gaudi] Enable FusedSDPA support for Intel Gaudi (HPU) Sanju C Sudhakaran 2025-02-06 02:59:45 +05:30
• 4c3aac51e1 Merging PR #12536 Chen Zhang 2025-02-06 05:24:26 +08:00
• bc1bdecebf [core][distributed] exact ray placement control (#12732) youkaichao 2025-02-06 02:03:19 +08:00
• 022bcc701a [Bugfix] Fix 'ModuleNotFoundError: No module named 'intel_extension_for_pytorch'' for --tensor-parallel-size more than 1 (#12546) Akash kaothalkar 2025-02-05 12:41:02 +05:30
• c53dc466b1 [Doc] Remove performance warning for auto_awq.md (#12743) Michael Goin 2025-02-05 01:43:11 -05:00
• 3d09e592a8 [V1][Misc] Shorten FinishReason enum and use constant strings (#12760) Nick Hill 2025-02-04 22:43:02 -08:00
• fcf2e3d7fc [Bugfix] Fix OpenVINO model runner (#12750) Harry Mellor 2025-02-05 06:42:46 +00:00
• 58b218d7ae [Doc] Update PR Reminder with link to Developer Slack (#12748) Michael Goin 2025-02-05 01:42:09 -05:00
• 7ff7a638b6 [Model][Quant] Fix GLM, Fix fused module mappings for quantization (#12634) Kyle Sayersandmgoin 2025-02-05 00:32:06 -05:00
• 686006a220 [Misc] Bump the compressed-tensors version (#12736) Dipika Sikka 2025-02-04 23:44:48 -05:00
• 98fd089fc9 [VLM] Add MLA with pure RoPE support for deepseek-vl2 models (#12729) Isotr0py 2025-02-05 12:44:26 +08:00
• 249824c3bf Refactor Linear handling in TransformersModel (#12727) Harry Mellor 2025-02-05 04:31:12 +00:00
• 64862d106e [ROCM][AMD][TRITON] Halving warps number for fw_prefill to reduce spilling (#12713) Aleksandr MalyshevandAleksandr Malyshev 2025-02-04 19:58:22 -08:00
• b3a0d01e45 [Core] add and implement VLLM_LOGITS_PROCESSOR_THREADS (#12368) Aviv Keshet 2025-02-04 18:46:26 -08:00
• 75e94309e8 [Perf] Mem align KV caches for CUDA devices (MLA perf improvement) (#12676) Lucas Wilkinsonandsimon-mo 2025-02-04 21:22:24 -05:00
• 233df6f5c4 [V1][Metrics] Add request_success_total counter, labelled with finish reason (#12579) Mark McLoughlin 2025-02-05 00:46:54 +00:00
• 18016a5e62 [Bugfix] Fix CI failures for InternVL and Mantis models (#12728) Cyrus Leung 2025-02-04 23:54:23 +08:00
• 649550f27e [Build] update requirements of no-device for plugin usage (#12630) Sophie du Couédic 2025-02-04 14:19:12 +01:00
• 62467a834a Avoid unnecessary multi-modal input data copy when len(batch) == 1 (#12722) Kero Liang 2025-02-04 21:03:19 +08:00
• 6469038b14 [Bugfix] Fix loading of fine-tuned models based on Phi-3-Small (#12689) Michael GreenbaumandMichael Greenbaum 2025-02-04 14:58:48 +02:00
• 815079de8e [VLM] merged multimodal processor and V1 support for idefics3 (#12660) Isotr0pyandCyrus Leung 2025-02-04 20:00:51 +08:00
• 18a88fcccc [V1] Remove scheduling constraint on partial requests (#12674) Woosuk Kwon 2025-02-04 02:43:58 -08:00
• d1ca7df84d [VLM] Merged multi-modal processor for InternVL-based models (#12553) Cyrus LeungandIsotr0py 2025-02-04 16:44:52 +08:00
• 96b23621c1 [Misc] Add BNB quantization for Whisper (#12381) Jee Jee Li 2025-02-04 16:27:36 +08:00
• c36ac98d01 [AMD][ROCm] Enable DeepSeek model on ROCm (#12662) Hongxia YangandMatthew Wong 2025-02-04 03:24:11 -05:00
• 4896d0c2dd [Quant] Fix use_mla TypeError and support loading pure-sparsity Compressed Tensors configs (#12711) Kyle Sayers 2025-02-04 02:27:11 -05:00
• bb392af434 [Doc] Replace ibm-fms with ibm-ai-platform (#12709) Thomas Parnell 2025-02-04 02:05:04 -05:00
• 5d98d56089 Support Pixtral-Large HF by using llava multimodal_projector_bias config (#12710) Michael Goin 2025-02-03 22:55:46 -05:00
• 73b35cca7f [Core] Improve hash collision avoidance in prefix caching (#12621) Russell Bryant 2025-02-03 19:28:20 -05:00
• 5095e96606 [V1] Revert uncache_blocks and support recaching full blocks (#12415) Cody Yu 2025-02-03 15:04:53 -08:00
• cf58b9c4ca [MISC] Remove model input dumping when exception (#12582) Cody Yu 2025-02-03 13:34:16 -08:00
• 4797dad3ec [Model] Add Deepseek V3 fp8_w8a8 configs for B200 (#12707) kushanam 2025-02-03 13:30:39 -08:00
• 6dd5e52823 Squelch MLA warning for Compressed-Tensors Models (#12704) Kyle Sayers 2025-02-03 16:29:56 -05:00
• c11de33dad [Bugfix][Kernel] Fix per-token/per-channel quantization for Hopper scaled mm (#12696) Tyler Michael Smith 2025-02-03 16:04:59 -05:00
• 33e0602e59 [Misc] Fix improper placement of SPDX header in scripts (#12694) Russell Bryant 2025-02-03 14:16:59 -05:00
• a1a2aaadb9 [Model]: Add transformers backend support (#11330) 7 people 2025-02-03 14:30:38 +01:00
• 1298a400e8 [ci/build] fix gh200 test (#12681) youkaichao 2025-02-03 15:59:49 +08:00
• ad4a9dc817 [cuda] manually import the correct pynvml module (#12679) youkaichao 2025-02-03 15:58:21 +08:00
• b9986454fe Fix for attention layers to remain unquantized during moe_wn16 quant (#12570) +2434 people 2025-02-02 21:46:19 -08:00
• c5932e5dac Properly check if all fused layers are in the list of targets (#12666) Eldar Kurtic 2025-02-03 06:42:18 +01:00
• 20579c0fae make sure mistral_common not imported for non-mistral models (#12669) youkaichao 2025-02-03 13:40:25 +08:00
• 95460fc513 [Kernel] port sgl moe_align_block_size kernels (#12574) Yang Chen 2025-02-02 21:09:50 -08:00
• 326fcc8b9f [Doc] Deprecate Discord (#12668) Zhuohan Li 2025-02-02 19:19:56 -08:00
• e64330910b [doc][misc] clarify VLLM_HOST_IP for multi-node inference (#12667) youkaichao 2025-02-03 09:32:18 +08:00
• e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) Russell Bryant 2025-02-02 14:58:18 -05:00
• f256ebe4df [Hardware][Intel GPU] add XPU bf16 support (#12392) Kunshang Ji 2025-02-02 18:17:26 +08:00
• f8ece6e17f [Core][v1] Unify allocating slots in prefill and decode in KV cache manager (#12608) Shawn Du 2025-02-02 16:40:58 +08:00
• abfcdcdf27 [V1][Minor] Avoid frequently creating ConstantList (#12653) Woosuk Kwon 2025-02-01 23:43:20 -08:00
• e497f33491 [Core] Silence unnecessary deprecation warnings (#12620) Russell Bryant 2025-02-02 02:35:50 -05:00
• baaa2b24da [Bugfix] fix moe_wna16 get_quant_method (#12648) Jinzhen Lin 2025-02-02 15:29:56 +08:00
• b4e5c03306 doc: fixing minor typo in readme.md (#12643) Vicente Herrera 2025-02-01 18:17:29 +01:00
• 3194039c0e Apply torch.compile to fused_moe/grouped_topk (#12637) Michael Goin 2025-02-01 11:16:19 -05:00
• 4f4d427ac2 Disable chunked prefill and/or prefix caching when MLA is enabled (#12642) Simon Moandmgoin 2025-01-31 23:46:57 -08:00
• 1e3698393f [CI/Build] Add label automation for structured-output, speculative-decoding, v1 (#12280) Russell Bryant 2025-02-01 02:13:10 -05:00
• baeded2569 [Attention] Deepseek v3 MLA support with FP8 compute (#12601) 7 people 2025-02-01 00:52:51 -05:00
• 3e1c76cf3a Fix: Respect sparsity_config.ignore in Cutlass Integration (#12517) Rahul Tuli 2025-01-31 23:41:59 -06:00
• cfa134d247 [Bugfix/CI] Fixup benchmark_moe.py (#12562) Tyler Michael Smith 2025-02-01 00:41:35 -05:00
• 35b7a05507 [ci] Upgrade transformers to 4.48.2 in CI dependencies (#12599) Kevin H. Luu 2025-01-31 21:22:23 -08:00
• 1867c258bd Fix target matching for fused layers with compressed-tensors (#12617) Eldar Kurtic 2025-02-01 06:07:46 +01:00
• cb3e73e4c8 [BugFix] fix wrong output when using lora and num_scheduler_steps=8 (#11161) 3 people 2025-02-01 12:52:07 +08:00
• b1340f9d55 [V1] Bugfix: Validate Model Input Length (#12600) Robert Shaw 2025-01-31 21:32:04 -05:00
• 44bbca78d7 [Doc] int4 w4a16 example (#12585) Brian DellabettaandMichael Goin 2025-01-31 17:38:48 -06:00
• 60808bd4c7 [Doc] Improve installation signposting (#12575) Harry Mellor 2025-01-31 23:38:35 +00:00
• fc542144c4 [Feature] Fix guided decoding blocking bitmask memcpy (#12563) Ryan Nguyen 2025-01-31 18:37:30 -05:00
• eb5741ad42 [Kernel][Quantization] Integrate block-quantized CUTLASS kernels for DeepSeekV3 (#12587) Tyler Michael Smith 2025-01-31 18:29:11 -05:00
• 145c2ff648 [Bugfix] Revert MoE Triton Config Default (#12629) Robert Shaw 2025-01-31 18:28:47 -05:00
• 415f19474d [release] Add input step to ask for Release version (#12631) Kevin H. Luu 2025-01-31 13:39:36 -08:00
• 89003c4082 [v1][Bugfix] Add extra_keys to block_hash for prefix caching (#12603) Chen Zhang 2025-02-01 05:13:04 +08:00
• 60bcef000e [Docs][V1] Prefix caching design (#12598) Cody Yu 2025-01-31 12:30:46 -08:00
• 847f883232 [Git] Automatically sign-off commits (#12595) Cody Yu 2025-01-31 12:30:33 -08:00
• 325f679f32 [BugFix] Fix Torch.Compile For DeepSeek (#12594) Robert Shawandsimon-mo 2025-01-31 15:06:39 -05:00
• e3f7ff65e7 Add favicon to docs (#12611) Harry Mellor 2025-01-31 17:20:34 +00:00
• 7a8987dac5 [Bugfix] Gracefully handle huggingface hub http error (#12571) Roger Wang 2025-01-31 00:19:35 -08:00
• cabaf4eff3 [Attention] MLA decode optimizations (#12528) 8 people 2025-01-31 02:49:37 -05:00
• a1fc18c030 [ROCm][AMD][Model] llama 3.2 support upstreaming (#12421) Aleksandr MalyshevandAleksandr Malyshev 2025-01-30 20:24:28 -08:00
• 9798b2fb00 [Kernel] Update cutlass_scaled_mm to support 2d group (blockwise) scaling (#11868) Lucas Wilkinson 2025-01-30 21:33:00 -05:00
• 4078052f09 [V1][Log] Add max request concurrency log to V1 (#12569) Michael Goin 2025-01-30 18:07:19 -05:00
• bd2107e30a [CPU][PPC] Updated torch, torchvision, torchaudio dependencies (#12555) Nishidha 2025-01-31 02:59:39 +05:30
• 9b0c4bab36 [Kernel] Triton Configs for Fp8 Block Quantization (#11589) 3 people 2025-01-30 14:53:22 -05:00
• 41bf5612f5 [Misc] fix typo: add missing space in lora adapter error message (#12564) Beim 2025-01-31 04:39:22 +13:00
• a2769032ca Set ?device={device} when changing tab in installation guides (#12560) Harry Mellor 2025-01-30 08:05:42 +00:00
• f17f1d4608 [V1][Metrics] Add GPU cache usage % gauge (#12561) Mark McLoughlin 2025-01-30 02:31:01 +00:00
• 1c1bb0bbf2 [Misc][MoE] add Deepseek-V3 moe tuning support (#12558) Divakar Verma 2025-01-29 18:47:30 -06:00
• e0cc5f259a [V1][BugFix] Free encoder cache for aborted requests (#12545) Woosuk Kwon 2025-01-29 13:47:33 -08:00
• 73aa6cfdf7 Revert "[Build/CI] Fix libcuda.so linkage" (#12552) Tyler Michael Smith 2025-01-29 16:12:24 -05:00
• 27b78c73ca [Kernel] add triton fused moe kernel for gptq/awq (#12185) Jinzhen Lin 2025-01-29 22:07:09 +08:00
• b02fd288b2 [Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models. (#11787) Pavani Majetyandmgoin 2025-01-29 01:46:12 -08:00