Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 5b59fe0f08 [Bugfix] Pass json-schema to GuidedDecodingParams and make test stronger (#9530) Chen Zhang 2024-10-19 17:05:02 -07:00
• 8e3e7f2713 [Model][Pixtral] Optimizations for input_processor_for_pixtral_hf (#9514) Michael Goin 2024-10-19 10:44:29 -04:00
• 263d8ee150 [Bugfix] Fix missing task for speculative decoding (#9524) Cyrus Leung 2024-10-19 14:49:40 +08:00
• c5eea3c8ba [Frontend] Support simpler image input format (#9478) Yue Zhang 2024-10-18 23:17:07 -07:00
• 85dc92fc98 [CI/Build] Configure matcher for actionlint workflow (#9511) Russell Bryant 2024-10-19 02:04:18 -04:00
• dfd951ed9b [CI/Build] Add error matching for ruff output (#9513) Russell Bryant 2024-10-19 01:42:20 -04:00
• 82c25151ec [Doc] update gpu-memory-utilization flag docs (#9507) Joe Runde 2024-10-18 22:26:36 -05:00
• 1325872ec8 [Frontend] Avoid creating guided decoding LogitsProcessor unnecessarily (#9521) Nick Hill 2024-10-19 04:21:01 +01:00
• 380e18639f 🐛 fix torch memory profiling (#9516) Joe Runde 2024-10-18 20:25:19 -05:00
• 337ed76671 [Bugfix] Fix offline mode when using mistral_common (#9457) sasha0552 2024-10-19 01:12:32 +00:00
• 0c9a5258f9 [Kernel] Add env variable to force flashinfer backend to enable tensor cores (#9497) 3 people 2024-10-19 02:55:48 +02:00
• d11bf435a0 [MISC] Consolidate cleanup() and refactor offline_inference_with_prefix.py (#9510) Cody Yu 2024-10-18 14:30:55 -07:00
• 9bb10a7d27 [MISC] Add lora requests to metrics (#9477) KunjanandKunjan Patel 2024-10-18 13:50:18 -07:00
• 3921a2f29e [Model] Support Pixtral models in the HF Transformers format (#9036) Michael Goin 2024-10-18 15:29:56 -04:00
• 67a7e5ef38 [CI/Build] Add error matching config for mypy (#9512) Russell Bryant 2024-10-18 15:17:53 -04:00
• 051eaf6db3 [Model] Add user-configurable task for models that support both generation and embedding (#9424) Cyrus Leung 2024-10-19 02:31:58 +08:00
• 7dbe738d65 [Misc] benchmark: Add option to set max concurrency (#9390) Russell Bryant 2024-10-18 14:15:28 -04:00
• ae8b633ba3 [Bugfix] Fix offline_inference_with_prefix.py (#9505) Tyler Michael Smith 2024-10-18 12:59:19 -04:00
• 1bbbcc0b1d [CI/Build] Fix lint errors in mistral tokenizer (#9504) Cyrus Leung 2024-10-19 00:09:35 +08:00
• 25aeb7d4c9 [BugFix] Fix and simplify completion API usage streaming (#9475) Nick Hill 2024-10-18 15:10:26 +01:00
• d2b1bf55ec [Frontend][Feature] Add jamba tool parser (#9154) tomeras91 2024-10-18 13:27:48 +03:00
• 1ffc8a7362 [BugFix] Typing fixes to RequestOutput.prompt and beam search (#9473) Nick Hill 2024-10-18 08:19:53 +01:00
• 944dd8edaf [CI/Build] Use commit hash references for github actions (#9430) Russell Bryant 2024-10-18 00:54:58 -04:00
• 154a8ae880 [Qwen2.5] Support bnb quant for Qwen2.5 (#9467) Haoyu Wang 2024-10-18 12:40:14 +08:00
• de4008e2ab [Bugfix][Core] Use torch.cuda.memory_stats() to profile peak memory usage (#9352) Joe Runde 2024-10-17 21:47:27 -05:00
• 48138a8415 [BugFix] Stop silent failures on compressed-tensors parsing (#9381) Dipika Sikka 2024-10-17 21:54:00 -04:00
• 343f8e0905 Support BERTModel (first encoder-only embedding model) (#9056) 8 people 2024-10-17 19:21:01 -04:00
• bb76538bbd [Hardwware][Neuron] Simplify model load for transformers-neuronx library (#9380) Shashwat Srijan 2024-10-17 15:39:39 -07:00
• d615b5c9f8 [Bugfix] Print warnings related to mistral_common tokenizer only once (#9468) sasha0552 2024-10-17 21:44:20 +00:00
• d65049daab [Bugfix] Add random_seed to sample_hf_requests in benchmark_serving script (#9013) Kai WuandIsotr0py 2024-10-17 14:11:11 -07:00
• eca2c5f7c0 [Bugfix] Fix support for dimension like integers and ScalarType (#9299) bnellnm 2024-10-17 15:08:34 -04:00
• 0f41fbe5a3 [torch.compile] Fine-grained CustomOp enabling mechanism (#9300) Luka Govedič 2024-10-17 14:36:37 -04:00
• 7871659abb [Misc] Remove commit id file (#9470) Cyrus Leung 2024-10-18 01:34:37 +08:00
• a2c71c5405 [CI/Build] remove .github from .dockerignore, add dirty repo check (#9375) Daniele 2024-10-17 19:25:06 +02:00
• 81ede99ca4 [Core] Deprecating block manager v1 and make block manager v2 default (#8704) Kuntai Du 2024-10-17 11:38:15 -05:00
• 5eda21e773 [Hardware][CPU] compressed-tensor INT8 W8A8 AZP support (#9344) Li, Jiang 2024-10-18 00:21:04 +08:00
• 8e1cddcd44 [TPU] Call torch._sync(param) during weight loading (#9437) Woosuk Kwon 2024-10-17 09:00:11 -07:00
• 5e443b594f [Bugfix] Allow prefill of assistant response when using mistral_common (#9446) sasha0552 2024-10-17 15:06:37 +00:00
• 9d30a056e7 [misc] CUDA Time Layerwise Profiler (#8337) 3 people 2024-10-17 10:36:09 -04:00
• 390be74649 [Misc] Print stack trace using logger.exception (#9461) Cyrus Leung 2024-10-17 21:55:48 +08:00
• e312e52b44 [Kernel] Add Exllama as a backend for compressed-tensors (#9395) Lucas Wilkinson 2024-10-17 09:48:26 -04:00
• dbfa8d31d5 Add notes on the use of Slack (#9442) Yuan Tang 2024-10-17 00:46:46 -04:00
• 92d86da217 [BugFix] [Kernel] Fix GPU SEGV occurring in int8 kernels (#9391) rasmith 2024-10-16 20:34:06 -05:00
• c3fab5f769 [Bugfix][Kernel] Prevent integer overflow in fp8 dynamic per-token quantize kernel (#9425) Tyler Michael Smith 2024-10-16 19:46:06 -04:00
• 776dbd74f1 [CI/Build] mypy: Resolve some errors from checking vllm/engine (#9267) Russell Bryant 2024-10-16 18:55:59 -04:00
• 8345045833 [Performance][Spec Decode] Optimize ngram lookup performance (#9333) Lily Liu 2024-10-16 12:37:45 -07:00
• 5b8a1fde84 [Model][Bugfix] Add FATReLU activation and support for openbmb/MiniCPM-S-1B-sft (#9396) Junhao Li 2024-10-16 12:40:24 -04:00
• fb60ae9b91 [Kernel][Model] Improve continuous batching for Jamba and Mamba (#9189) Mor Zusman 2024-10-17 00:12:43 +08:00
• 415f76a9cb Support mistral interleaved attn (#9414) Patrick von Platen 2024-10-16 15:28:30 +02:00
• cf1d62a644 [Model] Support SDPA attention for Molmo vision backbone (#9410) Isotr0py 2024-10-16 19:52:01 +08:00
• 59230ef32b [Misc] Consolidate example usage of OpenAI client for multimodal models (#9412) Roger WangandDarkLight1337 2024-10-16 04:20:51 -07:00
• cee711fdbb [Core] Rename input data types (#8688) Cyrus Leung 2024-10-16 18:49:37 +08:00
• 1de76a0e55 [CI/Build] Test VLM embeddings (#9406) Cyrus Leung 2024-10-16 17:44:30 +08:00
• 7abba39ee6 [Model] VLM2Vec, the first multimodal embedding model in vLLM (#9303) Cyrus Leung 2024-10-16 14:31:00 +08:00
• 7e7eae338d [Misc] Standardize RoPE handling for Qwen2-VL (#9250) Cyrus Leung 2024-10-16 13:56:17 +08:00
• ed920135c8 [Bugfix] Molmo text-only input bug fix (#9397) 4 people 2024-10-15 21:56:09 -07:00
• 717a5f82cd [Bugfix][CI/Build] Fix CUDA 11.8 Build (#9386) Lucas Wilkinson 2024-10-15 20:15:21 -04:00
• ba30942240 [Bugfix] Fix vLLM UsageInfo and logprobs None AssertionError with empty token_ids (#9034) Chang SuandNick Hill 2024-10-15 15:40:43 -07:00
• 22f8a69549 [Misc] Directly use compressed-tensors for checkpoint definitions (#8909) Michael GoinandDarkLight1337 2024-10-15 18:40:25 -04:00
• 5d264f4ab8 pass ignore_eos parameter to all benchmark_serving calls (#9349) Grace Ho 2024-10-15 13:30:44 -07:00
• e9d517f276 [BugFix] Fix chat API continuous usage stats (#9357) Nick Hill 2024-10-15 07:19:48 +01:00
• 55e081fbad [Bugfix] Update InternVL input mapper to support image embeds (#9351) hhzhang16 2024-10-14 21:29:19 -07:00
• 8e836d982a [Doc] Fix code formatting in spec_decode.rst (#9348) Michael Goin 2024-10-15 00:29:11 -04:00
• 44eaa5a5d9 [Frontend] Clarify model_type error messages (#9345) Steve Grubb 2024-10-15 00:29:01 -04:00
• 169b530607 [Bugfix] Clean up some cruft in mamba.py (#9343) Tyler Michael Smith 2024-10-14 20:24:25 -04:00
• f0fe4fe86d [Model] Make llama3.2 support multiple and interleaved images (#9095) Xiang Xu 2024-10-14 15:24:26 -07:00
• 4d31cd424b [Frontend] merge beam search implementations (#9296) Brendan Wong 2024-10-14 15:05:52 -07:00
• 473e7b3606 [TPU] Fix TPU SMEM OOM by Pallas paged attention kernel (#9350) Woosuk Kwon 2024-10-14 15:02:06 -07:00
• fd47e57f4b [Docs] Remove PDF build from Readtehdocs (#9347) Simon Mo 2024-10-14 11:57:47 -07:00
• 203ab8f80f [CI/Build] setuptools-scm fixes (#8900) Daniele 2024-10-14 20:34:47 +02:00
• 4141608c6a [Hardware][intel GPU] add async output process for xpu (#8897) Kunshang Ji 2024-10-15 02:23:33 +08:00
• dfe43a2071 [Model] Molmo vLLM Integration (#9016) 4 people 2024-10-14 07:56:24 -07:00
• 16b24e7dcd [Bugfix] Bandaid fix for speculative decoding tests (#9327) Tyler Michael Smith 2024-10-13 19:02:11 -04:00
• f519902c52 [CI] Fix merge conflict (#9317) Lily Liu 2024-10-12 23:41:23 -07:00
• 250e26a63e [Bugfix]Fix MiniCPM's LoRA bug (#9286) Jee Jee Li 2024-10-13 00:36:47 +08:00
• 2b184ddd4f [Misc][Installation] Improve source installation script and doc (#9309) Yunmengandyoukaichao 2024-10-13 00:36:40 +08:00
• 00298e092c [Bugfix] Fix bug of xformer prefill for encoder-decoder (#9026) Xiang Xu 2024-10-12 00:00:43 -07:00
• 89feb4c84d [SpecDec] Remove Batch Expansion (2/3) (#9298) Lily Liu 2024-10-11 22:13:37 -07:00
• ec10cb8511 [BugFix] Fix tool call finish reason in streaming case (#9209) Maximilien de Bayser 2024-10-11 22:24:26 -03:00
• d11b46f3a5 [bugfix] fix f-string for error (#9295) Prashant Gupta 2024-10-11 17:03:48 -07:00
• c6cf9295e1 [Bugfix] Sets is_first_step_output for TPUModelRunner (#9202) Allen Wang 2024-10-11 15:28:10 -05:00
• de9fb4bef8 [Bugfix][CI/Build] Fix docker build where CUDA archs < 7.0 are being detected (#9254) Lucas Wilkinson 2024-10-11 15:57:39 -04:00
• 8baf85e4e9 [Doc] Compatibility matrix for mutual exclusive features (#8512) Wallas Henrique 2024-10-11 15:18:50 -03:00
• 1a1823871d [Doc] Remove outdated comment to avoid misunderstanding (#9287) homeffjy 2024-10-12 02:02:03 +08:00
• 6cf1167c1a [Model] Add GLM-4v support and meet vllm==0.6.2 (#9242) sixgod 2024-10-12 01:36:13 +08:00
• f710090d8e [Kernel] adding fused moe kernel config for L40S TP4 (#9245) Burkhard Ringlein 2024-10-11 08:54:22 -07:00
• 7342a7d7f8 [Model] Support Mamba (#6484) Tyler Michael Smith 2024-10-11 11:40:06 -04:00
• df3dcdf49d [Bugfix] Fix priority in multiprocessing engine (#9277) Sebastian Schoennenbeck 2024-10-11 17:35:35 +02:00
• 36ea79079b [Misc][LoRA] Support loading LoRA weights for target_modules in reg format (#9275) Jee Jee Li 2024-10-11 20:31:21 +08:00
• e808156f30 [Misc] Collect model support info in a single process per model (#9233) Cyrus Leung 2024-10-11 19:08:11 +08:00
• cbc2ef5529 [misc] hide best_of from engine (#9261) youkaichaoandBrendan Wong 2024-10-10 21:30:44 -07:00
• 94bf9ae4e9 [Misc] Fix sampling from sonnet for long context case (#9235) Andy Dai 2024-10-10 17:33:16 -07:00
• f990bab2a4 [Doc][Neuron] add note to neuron documentation about resolving triton issue (#9257) omrishiv 2024-10-10 16:36:32 -07:00
• e00c094f15 [torch.compile] generic decorators (#9258) youkaichao 2024-10-10 15:54:23 -07:00
• a78c6ba7c8 [ci/build] Add placeholder command for custom models test (#9262) Kevin H. Luu 2024-10-10 15:45:09 -07:00
• fb870fd491 Bump actions/setup-python from 3 to 5 (#9195) dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 2024-10-10 13:30:46 -07:00
• 270953bafb Bump actions/checkout from 3 to 4 (#9196) dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 2024-10-10 13:30:35 -07:00
• 9cc811c4ff Bump actions/github-script from 6 to 7 (#9197) dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 2024-10-10 13:30:24 -07:00
• e4d652ea3e [torch.compile] integration with compilation control (#9058) youkaichao 2024-10-10 12:39:36 -07:00
• 78c0b4166c Suggest codeowners for the core componenets (#9210) Simon Mo 2024-10-10 12:29:24 -07:00