Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 628ec6c17b [Docker] bump up neuron sdk v2.21 (#11593) Liangfu Chen 2024-12-29 21:46:14 -08:00
• 3682e33f9f [v1] fix compilation cache (#11598) youkaichao 2024-12-30 12:24:12 +08:00
• 0aa38d16f5 Remove print statement in DeepseekScalingRotaryEmbedding (#11604) Michael Goin 2024-12-29 15:16:46 -05:00
• faef77c0d6 [Misc] KV cache transfer connector registry (#11481) Kuntai Du 2024-12-29 10:08:09 -06:00
• dba4d9dec6 [v1][bugfix] fix cudagraph with inplace buffer assignment (#11596) youkaichao 2024-12-29 17:03:49 +08:00
• 32b4c63f02 [Doc] Convert list tables to MyST (#11594) Cyrus Leung 2024-12-29 15:56:22 +08:00
• 4fb8e329fd [V1] [5/N] API Server: unify Detokenizer and EngineCore input (#11545) Robert Shaw 2024-12-28 15:51:57 -05:00
• 328841d002 [bugfix] interleaving sliding window for cohere2 model (#11583) youkaichao 2024-12-29 00:55:42 +08:00
• d427e5cfda [Doc] Minor documentation fixes (#11580) Cyrus Leung 2024-12-28 21:53:59 +08:00
• 42bb201fd6 [V1][Minor] Set pin_memory=False for token_ids_cpu tensor (#11581) Woosuk Kwon 2024-12-28 22:33:12 +09:00
• 59d6bb4c86 [Hardware][AMD]: Replace HIPCC version with more precise ROCm version (#11515) hj-wei 2024-12-28 19:17:35 +08:00
• b7dcc003dc [Model] Remove hardcoded image tokens ids from Pixtral (#11582) Roger Wang 2024-12-28 02:54:23 -08:00
• d34be24bb1 [Model] Support InternLM2 Reward models (#11571) Isotr0pyandCyrus Leung 2024-12-28 14:14:10 +08:00
• b5cbe8eeb3 [Bugfix] Last token measurement fix (#11376) Rajveer BachkaniwalaandRoger Wang 2024-12-27 22:34:46 -05:00
• df04dffade [V1] [4/N] API Server: ZMQ/MP Utilities (#11541) Robert Shaw 2024-12-27 20:45:08 -05:00
• a60731247f [Doc] Update mllama example based on official doc (#11567) Chen Zhang 2024-12-28 08:31:10 +08:00
• ac79799403 [Bugfix] Fix for ROCM compressed tensor support (#11561) Selali 2024-12-27 12:12:11 -08:00
• dde1fa18c9 [Misc] Improve BNB loader to handle mixture of sharded and merged weights with same suffix (#11566) Isotr0py 2024-12-28 03:45:13 +08:00
• 0240402c46 [Misc]Add BNB quantization for MolmoForCausalLM (#11551) Jee Jee Li 2024-12-28 02:48:24 +08:00
• 55509c2114 [MODEL] LoRA support for Jamba model (#11209) ErezSC42 2024-12-27 19:58:21 +02:00
• 101418096f [VLM] Support caching in merged multi-modal processor (#11396) Cyrus Leung 2024-12-28 01:22:48 +08:00
• 5ce4627a7e [Doc] Add xgrammar in doc (#11549) Chen1022 2024-12-27 21:05:10 +08:00
• 7af553ea30 [Misc] Abstract the logic for reading and writing media content (#11527) Cyrus Leung 2024-12-27 19:21:23 +08:00
• 2c9b8ea2b0 [Bugfix] Fix TeleChat2ForCausalLM weights mapper (#11546) Jee Jee Li 2024-12-27 18:39:15 +08:00
• d003f3ea39 Update deploying_with_k8s.md with AMD ROCm GPU example (#11465) AlexHe99andCyrus Leung 2024-12-27 18:00:04 +08:00
• 6c6f7fe8a8 [Platform] Move model arch check to platform (#11503) Mengqing Cao 2024-12-27 16:45:25 +08:00
• 2339d59f92 [BugFix] Fix quantization for all other methods (#11547) Robert Shaw 2024-12-27 01:23:29 -05:00
• 1b875a0ef3 [V1][3/N] API Server: Reduce Task Switching + Handle Abort Properly (#11534) Robert Shaw 2024-12-27 00:19:21 -05:00
• eb881ed006 [misc] fix typing (#11540) youkaichao 2024-12-27 11:05:08 +08:00
• 46d4359450 [CI] Fix broken CI (#11543) Robert Shaw 2024-12-26 21:49:16 -05:00
• 81b979f2a8 [V1] Fix yapf (#11538) Woosuk Kwon 2024-12-27 09:47:10 +09:00
• 371d04d39b [V1] Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling (#11394) Woosuk Kwon 2024-12-27 09:32:38 +09:00
• 0c0c2015c5 Update openai_compatible_server.md (#11536) Robert ShawandSimon Mo 2024-12-26 19:26:18 -05:00
• 82d24f7aac [Docs] Document Deepseek V3 support (#11535) Simon Mo 2024-12-26 16:21:56 -08:00
• f49777ba62 Deepseek v3 (#11502) 3 people 2024-12-26 16:09:44 -08:00
• 55fb97f7bd [2/N] API Server: Avoid ulimit footgun (#11530) Robert Shaw 2024-12-26 18:43:05 -05:00
• 2072924d14 [Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (#11523) 4 people 2024-12-26 18:33:30 -05:00
• 720b10fdc6 [1/N] API Server (Remove Proxy) (#11529) Robert Shaw 2024-12-26 18:03:43 -05:00
• b85a977822 [Doc] Add video example to openai client for multimodal (#11521) Isotr0pyandCyrus Leung 2024-12-27 01:31:29 +08:00
• eec906d811 [Misc] Add placeholder module (#11501) Cyrus Leung 2024-12-26 21:12:51 +08:00
• f57ee5650d [Model] Modify MolmoForCausalLM MLP (#11510) Jee Jee Li 2024-12-26 21:12:05 +08:00
• dcb1a944d4 [V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (#10681) sroy745andWoosuk Kwon 2024-12-26 02:02:58 -08:00
• 7492a36207 [Doc] Add QVQ and QwQ to the list of supported models (#11509) Roger WangandCyrus Leung 2024-12-26 01:44:32 -08:00
• aa25985bd1 [Misc][LoRA] Fix LoRA weight mapper (#11495) Jee Jee Li 2024-12-26 15:52:48 +08:00
• dbeac95dbb Mypy checking for vllm/compilation (#11496) Lucas Tuckerandlucast2021 2024-12-25 23:04:07 -06:00
• 51a624bf02 [Misc] Move some multimodal utils to modality-specific modules (#11494) Cyrus Leung 2024-12-26 12:23:20 +08:00
• 6ad909fdda [Doc] Improve GitHub links (#11491) Cyrus Leung 2024-12-26 06:49:26 +08:00
• b689ada91e [Frontend] Enable decord to load video from base64 (#11492) Cyrus Leung 2024-12-26 00:33:55 +08:00
• fc601665eb [Misc] Update disaggregation benchmark scripts and test logs (#11456) Jiaxin Shan 2024-12-24 22:58:48 -08:00
• 9832e5572a [V1] Unify VLLM_ENABLE_V1_MULTIPROCESSING handling in RayExecutor (#11472) Rui Qiao 2024-12-24 19:49:46 -08:00
• 3f3e92e1f2 [Model] Automatic conversion of classification and reward models (#11469) Cyrus Leung 2024-12-25 02:22:22 +08:00
• 409475a827 [Bugfix] Fix issues in CPU build Dockerfile. Fixes #9182 (#11435) Yuan Tang 2024-12-24 11:53:28 -05:00
• 196c34b0ac [Misc] Move weights mapper (#11443) Jee Jee Li 2024-12-24 21:05:25 +08:00
• 5c7963249d [attn][tiny fix] fix attn backend in MultiHeadAttention (#11463) Mengqing Cao 2024-12-24 20:39:36 +08:00
• 461cde2080 [OpenVINO] Fixed installation conflicts (#11458) Ilya Lavrenov 2024-12-24 15:38:21 +04:00
• 7a5286cc04 [Bugfix][Hardware][CPU] Fix CPU input_positions creation for text-only inputs with mrope (#11434) Isotr0py 2024-12-24 17:59:51 +08:00
• b1b1038fbd [Bugfix] Fix Qwen2-VL LoRA weight loading (#11430) Jee Jee Li 2024-12-24 17:56:10 +08:00
• 9edca6bf8f [Frontend] Online Pooling API (#11457) Cyrus Leung 2024-12-24 17:54:30 +08:00
• 4f074fbf53 [Misc]Suppress irrelevant exception stack trace information when CUDA… (#11438) dpxaandshiquan <shiquan> 2024-12-24 16:43:39 +08:00
• a491d6f535 [V1] TP Ray executor (#11107) Rui Qiao 2024-12-23 15:00:12 -08:00
• 32aa2059ad [Docs] Convert rST to MyST (Markdown) (#11145) Rafael Vasquez 2024-12-23 17:35:38 -05:00
• 94d545a1a1 [Doc] Fix typo in the help message of '--guided-decoding-backend' (#11440) yansh97 2024-12-24 04:20:44 +08:00
• 60fb4f3bcf [Bugfix] Add kv cache scales to gemma2.py (#11269) Michael Goin 2024-12-23 14:30:45 -05:00
• 63afbe9215 [CI] Expand OpenAI test_chat.py guided decoding tests (#11048) Michael Goin 2024-12-23 13:35:38 -05:00
• 8cef6e02dc [Misc] add w8a8 asym models (#11075) Dipika Sikka 2024-12-23 13:33:20 -05:00
• b866cdbd05 [Misc] Add assertion and helpful message for marlin24 compressed models (#11388) Dipika Sikka 2024-12-23 13:23:38 -05:00
• 2e726680b3 [Bugfix] torch nightly version in ROCm installation guide (#11423) Yuan Tang 2024-12-23 12:20:22 -05:00
• 5bfb30a529 [Bugfix] Fix CFGGuide and use outlines for grammars that can't convert to GBNF (#11389) Michael Goin 2024-12-23 10:06:20 -05:00
• e51719ae72 mypy type checking for vllm/worker (#11418) Lucas Tuckerandlucast2021 2024-12-23 07:55:49 -06:00
• f30581c518 [misc][perf] remove old code (#11425) youkaichao 2024-12-23 00:01:08 -08:00
• 048fc57a0f [CI] Unboock H100 Benchmark (#11419) Simon Mo 2024-12-22 14:17:43 -08:00
• f1d1bf6288 [Bugfix] Fix fully sharded LoRAs with Mixtral (#11390) Jason T. Greene 2024-12-22 09:25:10 -06:00
• 72d9c316d3 [cd][release] fix race conditions (#11407) youkaichao 2024-12-22 00:39:11 -08:00
• 4a9139780a [cd][release] add pypi index for every commit and nightly build (#11404) youkaichaoandRoger Wang 2024-12-21 23:53:44 -08:00
• 29c748930e [CI] Fix flaky entrypoint tests (#11403) Roger Wang 2024-12-21 21:08:44 -08:00
• c2d1b075ba [Bugfix] Fix issues for Pixtral-Large-Instruct-2411 (#11393) Roger Wangandywang96 2024-12-21 02:15:03 -08:00
• 584f0ae40d [V1] Make AsyncLLMEngine v1-v0 opaque (#11383) Ricky Xu 2024-12-20 23:14:08 -08:00
• 51ff216d85 [Bugfix] update should_ignore_layer (#11354) George 2024-12-21 01:36:23 -05:00
• dd2b5633dd [V1][Bugfix] Skip hashing empty or None mm_data (#11386) Woosuk Kwon 2024-12-21 14:22:21 +09:00
• 47a0b615b4 Add ray[default] to wget to run distributed inference out of box (#11265) Jiaxin Shan 2024-12-20 13:54:55 -08:00
• 5d2248d81a [doc] explain nccl requirements for rlhf (#11381) youkaichao 2024-12-20 13:00:56 -08:00
• d573aeadcc [Bugfix] Don't log OpenAI field aliases as ignored (#11378) Michael Goin 2024-12-20 14:03:50 -05:00
• 995f56236b [Core] Loading model from S3 using RunAI Model Streamer as optional loader (#10192) omer-dayan 2024-12-20 18:46:24 +02:00
• 7c7aa37c69 [CI/Build] fix pre-compiled wheel install for exact tag (#11373) Daniele 2024-12-20 17:14:40 +01:00
• 04139ade59 [V1] Fix profiling for models with merged input processor (#11370) Roger Wang 2024-12-20 04:04:21 -08:00
• 1ecc645b8f [doc] backward compatibility for 0.6.4 (#11359) youkaichao 2024-12-19 21:33:53 -08:00
• c954f21ac0 [misc] add early error message for custom ops (#11355) youkaichao 2024-12-19 21:18:25 -08:00
• 86c2d8fd1c [Bugfix] Fix spec decoding when seed is none in a batch (#10863) Wallas Henrique 2024-12-20 02:15:31 -03:00
• b880ffb87e [Misc] Add tqdm progress bar during graph capture (#11349) Michael Goin 2024-12-19 23:35:18 -05:00
• 7801f56ed7 [ci][gh200] dockerfile clean up (#11351) 3 people 2024-12-19 18:13:06 -08:00
• 48edab8041 [Bugfix][Hardware][POWERPC] Fix auto dtype failure in case of POWER10 (#11331) Akash kaothalkar 2024-12-20 07:02:07 +05:30
• a985f7af9f [CI] Adding CPU docker pipeline (#11261) YuanandKevin H. Luu 2024-12-20 03:46:55 +08:00
• e461c262f0 [Misc] Remove unused vllm/block.py (#11336) yangzhibin 2024-12-20 01:54:24 +08:00
• 276738ce0f [Bugfix] Fix broken CPU compressed-tensors test (#11338) Isotr0py 2024-12-20 01:37:31 +08:00
• cdf22afdda [Misc] Clean up and consolidate LRUCache (#11339) Cyrus Leung 2024-12-20 00:59:32 +08:00
• e24113a8fe [Model] Refactor Qwen2-VL to use merged multimodal processor (#11258) 3 people 2024-12-20 00:28:00 +08:00
• 7379b3d4b2 [V1] Fix multimodal profiling for Molmo (#11325) Roger Wangandywang96 2024-12-19 08:27:22 -08:00
• 6c7f881541 [Model] Add JambaForSequenceClassification model (#10860) 3 people 2024-12-19 16:48:06 +02:00
• a0f7d53beb [Bugfix] Cleanup Pixtral HF code (#11333) Cyrus Leung 2024-12-19 21:22:00 +08:00
• 5aef49806d [Feature] Add load generation config from model (#11164) Yanyi LiuandCyrus Leung 2024-12-19 18:50:38 +08:00