Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 21efb603f5 [CI/Build] Make the Dockerfile.cpu file's PIP_EXTRA_INDEX_URL Configurable as a Build Argument (#9252) jordanyono 2024-10-10 14:18:18 -04:00
• 055f3270d4 [Doc] Improve debugging documentation (#9204) Rafael Vasquez 2024-10-10 13:48:51 -04:00
• 18511aeda6 [Bugfix] Fix Machete unittests failing with NotImplementedError (#9218) Lucas Wilkinson 2024-10-10 13:39:56 -04:00
• 83ea5c72b9 [OpenVINO] Use torch 2.4.0 and newer optimim version (#9121) Ilya LavrenovandDarkLight1337 2024-10-10 21:18:58 +04:00
• 04de9057ab [Model] support input image embedding for minicpmv (#9237) whyiug 2024-10-10 23:00:47 +08:00
• 07c11cf4d4 [Bugfix] Fix lm_head weights tying with lora for llama (#9227) Isotr0py 2024-10-10 21:11:56 +08:00
• f3a507f1d3 [Core] Add an environment variable which needs to be set explicitly to allow BlockSpaceManagerV1 (#9149) sroy745 2024-10-09 23:17:17 -07:00
• a64e7b9407 [Bugfix] Machete garbage results for some models (large K dim) (#9212) Lucas Wilkinson 2024-10-10 02:16:17 -04:00
• ce00231a8b [Bugfix] Fix Weight Loading Multiple GPU Test - Large Models (#9213) Michael Goin 2024-10-10 02:15:40 -04:00
• de895f1697 [misc] improve model support check in another process (#9208) youkaichao 2024-10-09 21:58:27 -07:00
• cf25b93bdd [Core] Fix invalid args to _process_request (#9201) Russell Bryant 2024-10-10 00:10:09 -04:00
• d5fbb8706d [CI/Build] Update Dockerfile install+deploy image to ubuntu 22.04 (#9130) Michael GoinandDarkLight1337 2024-10-09 14:51:47 -04:00
• cdca8994bd [CI/Build] mypy: check vllm/entrypoints (#9194) Russell Bryant 2024-10-09 13:15:28 -04:00
• ca77dd7a44 [Hardware][CPU] Support AWQ for CPU backend (#7515) Li, Jiang 2024-10-10 00:28:08 +08:00
• 7dea289066 Add Dependabot configuration for GitHub Actions updates (#1217) Ewout ter HoevenandDarkLight1337 2024-10-09 17:16:26 +02:00
• cfaa6008e6 [Bugfix] Access get_vocab instead of vocab in tool parsers (#9188) Cyrus Leung 2024-10-09 22:59:57 +08:00
• 21906a6f50 [Bugfix] Fix lora loading for Compressed Tensors in #9120 (#9179) Ahmad Fahadh Ilyas 2024-10-09 05:10:44 -07:00
• dc4aea677a [Doc] Fix VLM prompt placeholder sample bug (#9170) Jiangtao Hu 2024-10-09 16:59:42 +08:00
• c8627cd41b [ci][test] use load dummy for testing (#9165) youkaichao 2024-10-09 00:38:40 -07:00
• 8bfaa4e31e [Bugfix] fix composite weight loading and EAGLE weight loading (#9160) Cyrus Leung 2024-10-09 15:36:55 +08:00
• 0b5b5d767e [Frontend] Log the maximum supported concurrency (#8831) AlpinDale 2024-10-09 07:03:14 +00:00
• cdc72e3c80 [Model] Remap FP8 kv_scale in CommandR and DBRX (#9174) Hui Liu 2024-10-08 23:43:06 -07:00
• 7627172bf4 [Bugfix][Doc] Report neuron error in output (#9159) Joe Rowell 2024-10-09 06:43:34 +01:00
• 480b7f40cf [Misc] Improve validation errors around best_of and n (#9167) Travis Johnson 2024-10-08 22:54:48 -06:00
• acce7630c1 Update link to KServe deployment guide (#9173) Yuan Tang 2024-10-08 23:58:49 -04:00
• ffc4b27ea8 Add classifiers in setup.py (#9171) Yuan Tang 2024-10-08 22:30:48 -04:00
• 2f4117c38e support bitsandbytes quantization with more models (#9148) chenqianfzh 2024-10-08 18:52:19 -07:00
• 9ba0bd6aa6 Add lm-eval directly to requirements-test.txt (#9161) Michael Goin 2024-10-08 21:22:31 -04:00
• 2a131965a8 mypy: check additional directories (#9162) Russell Bryant 2024-10-08 18:08:22 -04:00
• bd37b9fbe2 [Bugfix] Try to handle older versions of pytorch (#9086) bnellnm 2024-10-08 17:28:12 -04:00
• de24046fcd [Doc] Improve contributing and installation documentation (#9132) Rafael Vasquez 2024-10-08 16:22:08 -04:00
• 1874c6a1b0 [Doc] Update vlm.rst to include an example on videos (#9155) Sayak PaulandCyrus Leung 2024-10-08 23:42:29 +05:30
• 9a94ca4a5d [Bugfix] fix OpenAI API server startup with --disable-frontend-multiprocessing (#8537) Daniele 2024-10-08 18:38:40 +02:00
• cfba685bd4 [CI/Build] Add examples folder into Docker image so that we can leverage the templates*.jinja when serving models (#8758) Peter Pan 2024-10-09 00:37:34 +08:00
• 069d3bd8d0 [Frontend] Add Early Validation For Chat Template / Tool Call Parser (#9151) Alex Brooks 2024-10-08 08:31:26 -06:00
• a3691b6b5e [Core][Frontend] Add Support for Inference Time mm_processor_kwargs (#9131) Alex Brooks 2024-10-08 08:12:56 -06:00
• 8c746226c9 [Frontend] API support for beam search for MQLLMEngine (#9117) Brendan Wong 2024-10-07 22:51:43 -07:00
• e1faa2a598 [misc] improve ux on readme (#9147) youkaichao 2024-10-07 22:26:25 -07:00
• 80b57f00d5 [Intel GPU] Fix xpu decode input (#9145) Kunshang Ji 2024-10-08 11:51:14 +08:00
• 04c12f8157 [misc] update utils to support comparing multiple settings (#9140) youkaichao 2024-10-07 19:51:49 -07:00
• 8eeb857084 Add Slack to README (#9137) Simon Mo 2024-10-07 17:06:21 -07:00
• fa45513a51 [misc] fix comment and variable name (#9139) youkaichao 2024-10-07 16:07:05 -07:00
• c0d9a98d0c [Doc] Include performance benchmark in README (#9135) Kuntai Du 2024-10-07 15:04:06 -07:00
• e0dbdb013d [CI/Build] Add linting for github actions workflows (#7876) Russell Bryant 2024-10-07 17:18:10 -04:00
• 93cf74a8a7 [Doc]: Add deploying_with_k8s guide (#8451) TimWang 2024-10-08 04:31:45 +08:00
• 151ef4efd2 [Model] Support NVLM-D and fix QK Norm in InternViT (#9045) 3 people 2024-10-07 19:55:12 +08:00
• f19da64871 [Core] Refactor GGUF parameters packing and forwarding (#8859) Isotr0py 2024-10-07 18:01:46 +08:00
• 4f95ffee6f [Hardware][CPU] Cross-attention and Encoder-Decoder models support on CPU backend (#9089) Isotr0py 2024-10-07 14:50:35 +08:00
• 8c6de96ea1 [Model] Explicit interface for vLLM models and support OOT embedding models (#9108) Cyrus Leung 2024-10-07 14:10:35 +08:00
• 18b296fdb2 [core] remove beam search from the core (#9105) youkaichao 2024-10-06 22:47:04 -07:00
• c8f26bb636 [BugFix][Core] Fix BlockManagerV2 when Encoder Input is None (#9103) sroy745 2024-10-06 20:52:42 -07:00
• 487678d046 [Bugfix][Hardware][CPU] Fix CPU model input for decode (#9044) Isotr0py 2024-10-07 10:14:27 +08:00
• cb3b2b9ba4 [Bugfix] Fix incorrect updates to num_computed_tokens in multi-step scheduling (#9038) Varun Sundar RabindranathandVarun Sundar Rabindranath 2024-10-06 15:48:11 -04:00
• fdf59d30ea [Bugfix] fix tool_parser error handling when serve a model not support it (#8709) Yanyi Liu 2024-10-06 20:51:08 +08:00
• b22b798471 [Model] PP support for embedding models and update docs (#9090) Cyrus LeungandRoger Wang 2024-10-06 16:35:27 +08:00
• f22619fe96 [Misc] Remove user-facing error for removed VLM args (#9104) Cyrus Leung 2024-10-06 16:33:52 +08:00
• 168cab6bbf [Frontend] API support for beam search (#9087) Brendan Wongandyoukaichao 2024-10-05 23:39:03 -07:00
• 23fea8714a [Bugfix] Fix try-catch conditions to import correct Flash Attention Backend in Draft Model (#9101) TJian 2024-10-05 22:00:04 -07:00
• f4dd830e09 [core] use forward context for flash infer (#9097) youkaichao 2024-10-05 19:37:31 -07:00
• 5df1834895 [Bugfix] Fix order of arguments matters in config.yaml (#8960) Andy Dai 2024-10-05 10:35:11 -07:00
• cfadb9c687 [Bugfix] Deprecate registration of custom configs to huggingface (#9083) Chen Zhang 2024-10-05 06:56:40 -07:00
• 15986f598c [Model] Support Gemma2 embedding model (#9004) Xin Yang 2024-10-04 23:57:05 -07:00
• 53b3a33027 [Bugfix] Fixes Phi3v & Ultravox Multimodal EmbeddingInputs (#8979) hhzhang16 2024-10-04 22:05:37 -07:00
• dac914b0d6 [Bugfix] use blockmanagerv1 for encoder-decoder (#9084) Chen ZhangandRoger Wang 2024-10-04 21:45:38 -07:00
• a95354a36e [Doc] Update README.md with Ray summit slides (#9088) Zhuohan Li 2024-10-04 19:54:45 -07:00
• 663874e048 [torch.compile] improve allreduce registration (#9061) youkaichao 2024-10-04 16:43:50 -07:00
• cc90419e89 [Hardware][Neuron] Add on-device sampling support for Neuron (#8746) Chongming NiandAshraf Mahgoub 2024-10-04 16:42:20 -07:00
• 27302dd584 [Misc] Fix CI lint (#9085) Cody Yu 2024-10-04 16:07:54 -07:00
• 0cc566ca8f [Misc] Add random seed for prefix cache benchmark (#9081) Andy Dai 2024-10-04 14:58:57 -07:00
• 05c531be47 [Misc] Improved prefix cache example (#9077) Andy Dai 2024-10-04 14:38:42 -07:00
• fbb74420e7 [CI] Update performance benchmark: upgrade trt-llm to r24.07, and add SGLang (#7412) Kuntai Du 2024-10-04 14:01:44 -07:00
• 05d686432f [Kernel] Zero point support in fused MarlinMoE kernel + AWQ Fused MoE (#8973) 3 people 2024-10-04 20:34:44 +02:00
• 0dcc8cbe5a Adds truncate_prompt_tokens param for embeddings creation (#8999) Flávia Béo 2024-10-04 15:31:40 -03:00
• 26aa325f4f [Core][VLM] Test registration for OOT multimodal models (#8717) Roger WangandDarkLight1337 2024-10-04 10:38:25 -07:00
• e5dc713c23 [Hardware][PowerPC] Make oneDNN dependency optional for Power (#9039) Varad Ahirwadkar 2024-10-04 22:54:42 +05:30
• 36eecfbddb Remove AMD Ray Summit Banner (#9075) Simon Mo 2024-10-04 10:17:16 -07:00
• 9ade8bbc8d [Model] add a bunch of supported lora modules for mixtral (#9008) Prashant Gupta 2024-10-04 09:24:40 -07:00
• 22482e495e [Bugfix] Flash attention arches not getting set properly (#9062) Lucas Wilkinson 2024-10-04 11:43:15 -04:00
• 3d826d2c52 [Bugfix] Reshape the dimensions of the input image embeddings in Qwen2VL (#9071) whyiug 2024-10-04 22:34:58 +08:00
• 0e36fd4909 [Misc] Move registry to its own file (#9064) Cyrus Leung 2024-10-04 18:01:37 +08:00
• 0f6d7a9a34 [Models] Add remaining model PP support (#7168) Murali AndoorveeduandDarkLight1337 2024-10-03 19:56:58 -07:00
• 303d44790a [Misc] Enable multi-step output streaming by default (#9047) Michael Goin 2024-10-03 22:55:42 -04:00
• aeb37c2a72 [CI/Build] Per file CUDA Archs (improve wheel size and dev build times) (#8845) Lucas Wilkinson 2024-10-03 22:55:25 -04:00
• 3dbb215b38 [Frontend][Feature] support tool calling for internlm/internlm2_5-7b-chat model (#8405) 代君 2024-10-04 10:36:39 +08:00
• 2838d6b38e [Bugfix] Weight loading fix for OPT model (#9042) Domen Vrešanddvres 2024-10-04 01:53:29 +02:00
• 91add85ec4 Fix failing spec decode test (#9054) sroy745 2024-10-03 16:07:29 -07:00
• 9aaf14c62e [misc] add forward context for attention (#9029) youkaichao 2024-10-03 12:09:42 -07:00
• 63e39937f9 [Frontend] [Neuron] Parse literals out of override-neuron-config (#8959) xendoandJerzy Zagorski 2024-10-03 20:02:07 +02:00
• f5d72b2fc6 [Core] Make BlockSpaceManagerV2 the default BlockManager to use. (#8678) sroy745 2024-10-03 09:44:21 -07:00
• 83caf35e08 [BugFix] Enforce Mistral ToolCall id constraint when using the Mistral tool call parser (#9020) Guillaume Calmettes 2024-10-03 10:44:52 +02:00
• 01843c89b8 [Misc] log when using default MoE config (#8971) Divakar Verma 2024-10-02 23:31:07 -05:00
• 19a4dd0990 [Bugfix] example template should not add parallel_tool_prompt if tools is none (#9007) Travis Johnson 2024-10-02 21:04:17 -06:00
• 18c2e30c57 [Doc] Update Granite model docs (#9025) Nick Hill 2024-10-03 03:42:24 +01:00
• 19f0d25796 [Model] Adding Granite MoE. (#8206) Shawn TanandNick Hill 2024-10-02 21:33:57 -04:00
• f58d4fccc9 [OpenVINO] Enable GPU support for OpenVINO vLLM backend (#8192) Sergey Shlyapnikov 2024-10-03 01:50:01 +04:00
• afb050b29d [Core] CUDA Graphs for Multi-Step + Chunked-Prefill (#8645) Varun Sundar RabindranathandVarun Sundar Rabindranath 2024-10-02 15:44:39 -04:00
• 7f60520deb [Misc] Update Default Image Mapper Error Log (#8977) Alex BrooksandRoger Wang 2024-10-02 05:44:38 -06:00
• 563649aafe [Core] Combined support for multi-step scheduling, chunked prefill & prefix caching (#8804) 3 people 2024-10-02 03:52:20 -04:00
• 1570203864 [Spec Decode] (1/2) Remove batch expansion (#8839) Lily Liu 2024-10-01 16:04:42 -07:00
• 22f5851b80 Update benchmark_serving.py to read and write json-datasets, results in UTF8, for better compatibility with Windows (#8997) vlsav 2024-10-01 21:07:06 +03:00