Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• eed020f673 [misc] use nvml to get consistent device name (#7582) youkaichao 2024-08-16 21:15:13 -07:00
• 7c0b7ea214 [Bugfix] add >= 1.0 constraint for openai dependency (#7612) Xander Johnson 2024-08-16 20:56:01 -07:00
• 4706eb628e [aDAG] Unflake aDAG + PP tests (#7600) SangBin Cho 2024-08-16 20:49:30 -07:00
• bae888cb8e [Bugfix] Clear engine reference in AsyncEngineRPCServer (#7618) Rui Qiao 2024-08-16 20:44:05 -07:00
• 6bd19551b0 .[Build/CI] Enabling passing AMD tests. (#7610) Alexei-V-Ivanov-AMD 2024-08-16 22:25:32 -05:00
• e680349994 [Bugfix] Fix custom_ar support check (#7617) bnellnm 2024-08-16 22:05:49 -04:00
• 44f26a9466 [Model] Align nemotron config with final HF state and fix lm-eval-small (#7611) Michael Goin 2024-08-16 18:56:34 -04:00
• 37fd47e780 [Kernel] fix types used in aqlm and ggml kernels to support dynamo (#7596) bnellnm 2024-08-16 17:00:11 -04:00
• 7759ae958f [Kernel][Misc] dynamo support for ScalarType (#7594) bnellnm 2024-08-16 16:59:49 -04:00
• 9f69856356 [Kernel] register punica functions as torch ops (#7591) bnellnm 2024-08-16 16:59:38 -04:00
• d4f0f17b02 [Doc] Update quantization supported hardware table (#7595) Michael Goin 2024-08-16 16:59:27 -04:00
• b3f4e17935 [Doc] Add docs for llmcompressor INT8 and FP8 checkpoints (#7444) Michael Goin 2024-08-16 16:59:16 -04:00
• 93478b63d2 [Core] Fix tracking of model forward time in case of PP>1 (#7440) Mahesh Keralapura 2024-08-16 13:46:01 -07:00
• f366f6339b [spec decode] [4/N] Move update_flash_attn_metadata to attn backend (#7571) William LinandCody Yu 2024-08-16 11:41:56 -07:00
• 855866caa9 [Kernel] Add tuned triton configs for ExpertsInt8 (#7601) Michael Goin 2024-08-16 14:37:01 -04:00
• 7fc23be81c [Kernel] W8A16 Int8 inside FusedMoE (#7415) Mor Zusman 2024-08-16 20:06:51 +03:00
• e837b624f2 [Feature][Hardware][Amd] Add fp8 Linear Layer for Rocm (#7210) Charlie Fu 2024-08-16 12:06:30 -05:00
• ec724a725e support tqdm in notebooks (#7510) fzyzcjy 2024-08-17 00:17:50 +08:00
• 0e39a33c6d [Bugfix][Hardware][AMD][Frontend] add quantization param to embedding checking method (#7513) Gordon Wong 2024-08-17 00:05:18 +08:00
• 6fc5b0f249 [CI] Fix crashes of performance benchmark (#7500) Kuntai Du 2024-08-16 08:08:45 -07:00
• 9587b050fb [Core] Use uvloop with zmq-decoupled front-end (#7570) Nick Hill 2024-08-15 22:48:07 -07:00
• 54bd9a03c4 register custom op for flash attn and use from torch.ops (#7536) youkaichao 2024-08-15 22:38:56 -07:00
• 50b8d08dbd [Misc/Testing] Use torch.testing.assert_close (#7324) jon-chuang 2024-08-15 21:24:04 -07:00
• e165528778 [CI] Move quantization cpu offload tests out of fastcheck (#7574) Michael Goin 2024-08-16 00:16:20 -04:00
• 3b19e39dc5 Chat method for offline llm (#5049) 6 people 2024-08-16 09:41:34 +07:00
• 4cd7d47fed [ci/test] rearrange tests and make adag test soft fail (#7572) youkaichao 2024-08-15 19:39:04 -07:00
• f878c8feb0 [Feature]: Add OpenAI server prompt_logprobs support #6508 (#7453) Grant Pinkert 2024-08-16 12:38:08 +10:00
• b67ae00cdb [Misc] Add quantization config support for speculative model. (#7343) shangmingc 2024-08-16 10:34:28 +08:00
• 9c8e2d1161 [Bugfix][Harmless] Fix float16 dtype for model_is_embedding (#7566) Michael Goin 2024-08-15 21:26:19 -04:00
• 21313e09e3 [Bugfix] Fix default weight loading for scalars (#7534) Michael Goin 2024-08-15 16:10:22 -04:00
• f4da5f7b6d [Misc] Update dockerfile for CPU to cover protobuf installation (#7182) PHILO-HE 2024-08-16 01:03:01 +08:00
• 9c1f78d5d6 [Bugfix] update neuron for version > 0.5.0 (#7175) omrishivandCyrus Leung 2024-08-15 09:44:14 -07:00
• fc93e56143 [Bugfix][TPU] Correct env variable for XLA cache path (#7544) Woosuk Kwon 2024-08-15 00:02:29 -07:00
• 22b39e11f2 llama_index serving integration documentation (#6973) Kameshwara Pavan Kumar Manthaandpavanmantha 2024-08-15 04:08:37 +05:30
• f55a9aea45 [Misc] Revert compressed-tensors code reuse (#7521) Kyle Sayers 2024-08-14 18:07:37 -04:00
• 951fdd66d3 [TPU] Set per-rank XLA cache (#7533) Woosuk Kwon 2024-08-14 14:47:51 -07:00
• 2ecf7b1757 [core] [3/N] multi-step args and sequence.py (#7452) William Lin 2024-08-14 12:32:45 -07:00
• 3f674a49b5 [VLM][Core] Support profiling with multiple multi-modal inputs per prompt (#7126) Cyrus Leung 2024-08-15 01:55:42 +08:00
• 70b746efcf [Misc] Deprecation Warning when setting --engine-use-ray (#7424) 4 people 2024-08-14 13:44:27 -03:00
• 67d115db08 [Bugfix][Frontend] Disable embedding API for chat models (#7504) jackandjack 2024-08-15 00:15:19 +08:00
• d3d9cb6e4b [ci] fix model tests (#7507) youkaichao 2024-08-14 01:01:43 -07:00
• c134a46402 Fix empty output when temp is too low (#2937) Chang SuandCyrus Leung 2024-08-13 22:31:44 -07:00
• 199adbb7cf [doc] update test script to include cudagraph (#7501) youkaichao 2024-08-13 21:52:58 -07:00
• dd164d72f3 [Bugfix][Docs] Update list of mock imports (#7493) Cyrus Leung 2024-08-14 11:37:30 +08:00
• ea49e6a3c8 [misc][ci] fix cpu test with plugins (#7489) youkaichao 2024-08-13 19:27:46 -07:00
• 97992802f3 [CI/Build]Reduce the time consumption for LoRA tests (#7396) Jee Jee Li 2024-08-14 08:27:29 +08:00
• 59edd0f134 [Bugfix][CI] Import ray under guard (#7486) Woosuk Kwon 2024-08-13 17:12:58 -07:00
• a08df8322e [TPU] Support multi-host inference (#7457) Woosuk Kwon 2024-08-13 16:31:20 -07:00
• 16422ea76f [misc][plugin] add plugin system implementation (#7426) youkaichao 2024-08-13 16:24:17 -07:00
• 373538f973 [Misc] compressed-tensors code reuse (#7277) Kyle Sayers 2024-08-13 19:05:15 -04:00
• 33e5d7e6b6 [frontend] spawn engine process from api server process (#7484) youkaichao 2024-08-13 15:40:17 -07:00
• c5c7768264 Announce NVIDIA Meetup (#7483) Simon MoandWoosuk Kwon 2024-08-13 14:28:36 -07:00
• b1e5afc3e7 [Misc] Update awq and awq_marlin to use vLLMParameters (#7422) Dipika Sikka 2024-08-13 17:08:20 -04:00
• d3bdfd3ab9 [Misc] Update Fused MoE weight loading (#7334) Dipika Sikka 2024-08-13 14:57:45 -04:00
• fb377d7e74 [Misc] Update gptq_marlin to use new vLLMParameters (#7281) Dipika Sikka 2024-08-13 14:30:11 -04:00
• 181abbc27d [Misc] Update LM Eval Tolerance (#7473) Dipika Sikka 2024-08-13 14:28:14 -04:00
• 00c3d68e45 [Frontend][Core] Add plumbing to support audio language models (#7446) Peter Salas 2024-08-13 10:39:33 -07:00
• e20233d361 Revert "[Doc] Update supported_hardware.rst (#7276)" (#7467) Woosuk Kwon 2024-08-13 01:37:08 -07:00
• d6e634f3d7 [TPU] Suppress import custom_ops warning (#7458) Woosuk Kwon 2024-08-13 00:30:30 -07:00
• 4d2dc5072b [hardware] unify usage of is_tpu to current_platform.is_tpu() (#7102) youkaichao 2024-08-13 00:16:42 -07:00
• 7025b11d94 [Bugfix] Fix weight loading for Chameleon when TP>1 (#7410) Cyrus Leung 2024-08-13 13:33:41 +08:00
• 5469146bcc [ci] Remove fast check cancel workflow (#7455) Kevin H. Luu 2024-08-12 21:19:51 -07:00
• 97a6be95ba [Misc] improve logits processors logging message (#7435) Andrew Wang 2024-08-12 19:29:34 -07:00
• 9ba85bc152 [mypy] Misc. typing improvements (#7417) Cyrus Leung 2024-08-13 09:20:20 +08:00
• 198d6a2898 [Core] Shut down aDAG workers with clean async llm engine exit (#7224) Rui Qiao 2024-08-12 17:57:16 -07:00
• 774cd1d3bf [CI/Build] bump minimum cmake version (#6999) Daniele 2024-08-13 01:29:20 +02:00
• 91294d56e1 [Bugfix] Handle PackageNotFoundError when checking for xpu version (#7398) sasha0552 2024-08-12 23:07:20 +00:00
• a046f86397 [Core/Bugfix] Add FP8 K/V Scale and dtype conversion for prefix/prefill Triton Kernel (#7208) jon-chuangandCody Yu 2024-08-12 15:47:41 -07:00
• 4ddc4743d7 [Core] Consolidate GB constant and enable float GB arguments (#7416) Cyrus Leung 2024-08-13 05:14:14 +08:00
• 6aa33cb2dd [Misc] Use scalar type to dispatch to different gptq_marlin kernels (#7323) Lucas Wilkinson 2024-08-12 14:40:13 -04:00
• 1137f343aa [ci] Cancel fastcheck when PR is ready (#7433) Kevin H. Luu 2024-08-12 10:59:14 -07:00
• 9b3e2edd30 [ci] Cancel fastcheck run when PR is marked ready (#7427) Kevin H. Luu 2024-08-12 10:56:52 -07:00
• 65950e8f58 [ci] Entrypoints run upon changes in vllm/ (#7423) Kevin H. Luu 2024-08-12 10:18:03 -07:00
• cfba4def5d [Bugfix] Fix logit soft cap in flash-attn backend (#7425) Woosuk Kwon 2024-08-12 09:58:28 -07:00
• d2bc4510a4 [CI/Build] bump Dockerfile.neuron image base, use public ECR (#6832) Daniele 2024-08-12 18:53:35 +02:00
• 24154f8618 [Frontend] Disallow passing model as both argument and option (#7347) Cyrus Leung 2024-08-12 20:58:34 +08:00
• e6e42e4b17 [Core][VLM] Support image embeddings as input (#6613) Roger Wang 2024-08-12 01:16:06 -07:00
• ec2affa8ae [Kernel] Flashinfer correctness fix for v0.1.3 (#7319) Lily Liu 2024-08-12 00:59:17 -07:00
• 86ab567bae [CI/Build] Minor refactoring for vLLM assets (#7407) Roger Wang 2024-08-11 19:41:52 -07:00
• f020a6297e [Docs] Update readme (#7316) Simon Mo 2024-08-11 17:13:37 -07:00
• 6c8e595710 [misc] add commit id in collect env (#7405) youkaichao 2024-08-11 15:40:48 -07:00
• 02b1988b9f [Doc] building vLLM with VLLM_TARGET_DEVICE=empty (#7403) tomeras91 2024-08-12 00:38:17 +03:00
• 386087970a [CI/Build] build on empty device for better dev experience (#4773) tomeras91 2024-08-11 23:09:44 +03:00
• c08e2b3086 [core] [2/N] refactor worker_base input preparation for multi-step (#7387) William Lin 2024-08-11 08:50:08 -07:00
• 4fb7b52a2c Updating LM Format Enforcer version to v0.10.6 (#7189) Noam Gat 2024-08-11 15:11:50 +03:00
• 90bab18f24 [TPU] Use mark_dynamic to reduce compilation time (#7340) Woosuk Kwon 2024-08-10 18:12:22 -07:00
• 4c5d8e8ea9 [Bugfix] Fix phi3v batch inference when images have different aspect ratio (#7392) Isotr0py 2024-08-11 00:19:33 +08:00
• baa240252e [Core] Fix edge case in chunked prefill + block manager v2 (#7380) Cade Daniel 2024-08-09 16:48:49 -07:00
• 999ef0b917 [Misc] Add numpy implementation of compute_slot_mapping (#7377) Antoni Baum 2024-08-09 15:52:29 -07:00
• 5c6c54d67a [Bugfix] Fix PerTensorScaleParameter weight loading for fused models (#7376) Dipika Sikka 2024-08-09 17:23:46 -04:00
• 933790c209 [Core] Add span metrics for model_forward, scheduler and sampler time (#7089) Mahesh Keralapura 2024-08-09 13:55:13 -07:00
• 70d268a399 [Bugfix] Fix ITL recording in serving benchmark (#7372) Roger Wang 2024-08-09 10:00:00 -07:00
• 249b88228d [Frontend] Support embeddings in the run_batch API (#7132) Pooya DavoodiandSimon Mo 2024-08-09 09:48:21 -07:00
• 74af2bbd90 [Bugfix] Fix reinit procedure in ModelInputForGPUBuilder (#7360) Alexander Matveev 2024-08-09 12:35:49 -04:00
• fc7b8d1eef [Performance] e2e overheads reduction: Small followup diff (#7364) Alexander Matveev 2024-08-09 11:49:36 -04:00
• 67abdbb42f [VLM][Doc] Add stop_token_ids to InternVL example (#7354) Isotr0py 2024-08-09 22:51:04 +08:00
• 07ab160741 [Model][Jamba] Mamba cache single buffer (#6739) Mor ZusmanandMor Zusman 2024-08-09 17:07:06 +03:00
• b4e9528f95 [Core] Streamline stream termination in AsyncLLMEngine (#7336) Nick Hill 2024-08-09 00:06:36 -07:00
• 57b7be0e1c [Speculative decoding] [Multi-Step] decouple should_modify_greedy_probs_inplace (#6971) William Lin 2024-08-08 22:42:45 -07:00
• 99b4cf5f23 [Bugfix] Fix speculative decoding with MLPSpeculator with padded vocabulary (#7218) Travis Johnson 2024-08-08 23:08:46 -06:00