Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 5fa6e9876e [Bugfix] Fix for multinode crash on 4 PP (#6495) Murali Andoorveedu 2024-07-17 04:25:10 -04:00
• 5bf35a91e4 [Doc][CI/Build] Update docs and tests to use vllm serve (#6431) Cyrus Leung 2024-07-17 15:43:21 +08:00
• a19e8d3726 [Misc][Speculative decoding] Typos and typing fixes (#6467) shangmingcandcaishangming.csm 2024-07-17 15:17:07 +08:00
• 10383887e0 [ROCm] Cleanup Dockerfile and remove outdated patch (#6482) Hongxia Yang 2024-07-17 01:47:02 -04:00
• 1d094fd7c0 [Distributed][PP] only create embedding & lm head when necessary (#6455) Wushi Dong 2024-07-16 19:20:26 -07:00
• ce37be7ba0 [misc][distributed] add seed to dummy weights (#6491) youkaichao 2024-07-16 19:16:34 -07:00
• 7f62077af5 [misc][distributed] improve tests (#6488) youkaichao 2024-07-16 17:35:52 -07:00
• 09c2eb85dd [ci][distributed] add pipeline parallel correctness test (#6410) youkaichao 2024-07-16 15:44:22 -07:00
• 978aed5300 [Kernel][Attention] Separate Attention.kv_scale into k_scale and v_scale (#6081) Michael Goin 2024-07-16 18:31:32 -04:00
• 160e1d8c99 [Misc] Log spec decode metrics (#6454) Cody Yu 2024-07-16 13:37:10 -07:00
• 94162beb9f [Doc] Fix the lora adapter path in server startup script (#6230) Jiaxin Shan 2024-07-16 10:11:04 -07:00
• c467dff24f [Hardware][TPU] Support MoE with Pallas GMM kernel (#6457) Woosuk Kwon 2024-07-16 09:56:28 -07:00
• 9f4ccec761 [doc][misc] remind to cancel debugging environment variables (#6481) youkaichao 2024-07-16 09:45:30 -07:00
• 38ef94888a [CI/Build] Remove "boardwalk" image asset (#6460) Cyrus Leung 2024-07-16 23:59:36 +08:00
• 2bb0489cb3 [Core] Use numpy to speed up padded token processing (#6442) Peng Guanwen 2024-07-16 23:13:25 +08:00
• 7508a3dc34 [Misc] Fix typos in spec. decode metrics logging. (#6470) Thomas Parnell 2024-07-16 15:55:15 +02:00
• 7a3d2a5b95 [Frontend] Support for chat completions input in the tokenize endpoint (#5923) sasha0552 2024-07-16 12:18:09 +00:00
• d97011512e [CI/Build] vLLM cache directory for images (#6444) Cyrus Leung 2024-07-16 14:12:25 +08:00
• 37d776606f [Docs] Announce 5th meetup (#6458) Woosuk Kwon 2024-07-15 21:04:58 -07:00
• d92b3c5cde [Bugfix][CI/Build] Test prompt adapters in openai entrypoint tests (#6419) Joe 2024-07-15 18:54:15 -07:00
• 9ad32dacd9 [BugFix][Model] Jamba - Handle aborted requests, Add tests and fix cleanup bug (#6425) Mor ZusmanandMor Zusman 2024-07-16 04:32:55 +03:00
• d6f3b3d5c4 Pin sphinx-argparse version (#6453) Kevin H. Luu 2024-07-15 18:26:11 -07:00
• 4552e37b55 [CI/Build][TPU] Add TPU CI test (#6277) Woosuk Kwonandkevin 2024-07-15 14:31:16 -07:00
• ec9933f4a5 [Misc] Add CustomOp Interface to UnquantizedFusedMoEMethod (#6289) Woosuk Kwon 2024-07-15 12:02:14 -07:00
• 3dee97b05f [Docs] Add Google Cloud to sponsor list (#6450) Woosuk Kwon 2024-07-15 11:58:10 -07:00
• 4cf256ae7f [misc][distributed] fix pp missing layer condition (#6446) youkaichao 2024-07-15 10:32:35 -07:00
• 64fdc08c72 bump version to v0.5.2 (#6433) Simon Mo 2024-07-15 10:27:40 -07:00
• 4ef95b0f06 [Bugfix] use float32 precision in samplers/test_logprobs.py for comparing with HF (#6409) Thomas Parnell 2024-07-15 19:14:49 +02:00
• eaec4b9153 [Bugfix] Add custom Triton cache manager to resolve MoE MP issue (#6140) Thomas ParnellandChih-Chieh-Yang 2024-07-15 19:12:47 +02:00
• a63a4c6341 [Misc] Use 0.0.9 version for flashinfer (#6447) Pernekhan UtemuratovandPernekhan Utemuratov 2024-07-15 10:10:26 -07:00
• c8fd97f26d [Kernel] Use CUTLASS kernels for the FP8 layers with Bias (#6270) Tyler Michael Smith 2024-07-15 13:05:52 -04:00
• 94b82e8c18 [doc][distributed] add suggestion for distributed inference (#6418) youkaichao 2024-07-15 09:45:51 -07:00
• 6ae1597ddf [VLM] Minor space optimization for ClipVisionModel (#6436) Roger Wang 2024-07-15 02:29:51 -07:00
• 22e79ee8f3 [doc][misc] doc update (#6439) youkaichao 2024-07-14 23:33:25 -07:00
• de19916314 [Bugfix] Convert image to RGB by default (#6430) Cyrus Leung 2024-07-15 13:39:15 +08:00
• 69672f116c [core][distributed] simplify code to support pipeline parallel (#6406) youkaichao 2024-07-14 21:20:51 -07:00
• 44874a0bf9 [Doc] add env docs for flashinfer backend (#6437) DefTruth 2024-07-15 12:16:51 +08:00
• b47008b4d2 [BugFix] BatchResponseData body should be optional (#6345) zifeitongandCyrus Leung 2024-07-14 21:06:09 -07:00
• 9bfece89fd Add FUNDING.yml (#6435) Simon Mo 2024-07-14 20:36:16 -07:00
• 32c9d7f765 Report usage for beam search (#6404) Simon Mo 2024-07-14 19:37:35 -07:00
• ccb20db8bd [Bugfix] Benchmark serving script used global parameter 'args' in function 'sample_random_requests' (#6428) Fish 2024-07-15 10:27:01 +08:00
• a754dc2cb9 [CI/Build] Cross python wheel (#6394) Robert Shaw 2024-07-14 21:54:46 -04:00
• 61e85dbad8 [Doc] xpu backend requires running setvars.sh (#6393) Robert Cohn 2024-07-14 20:10:11 -04:00
• dbfe254eda [Feature] vLLM CLI (#5090) Ethan Xuandsimon-mo 2024-07-14 15:36:43 -07:00
• 73030b7dae [ Misc ] Enable Quantizing All Layers of DeekSeekv2 (#6423) Robert Shaw 2024-07-14 17:38:42 -04:00
• ccd3c04571 [ci][build] fix commit id (#6420) youkaichaoandCyrus Leung 2024-07-14 07:16:21 -07:00
• 9dad5cc859 [Kernel] Turn off CUTLASS scaled_mm for Ada Lovelace (#6384) Tyler Michael Smith 2024-07-14 09:37:19 -04:00
• 6ef3bf912c Remove unnecessary trailing period in spec_decode.rst (#6405) Yuan Tang 2024-07-14 02:58:09 -05:00
• 540c0368b1 [Model] Initialize Fuyu-8B support (#3924) Isotr0pyandRoger Wang 2024-07-14 13:27:14 +08:00
• fb6af8bc08 [ Misc ] Apply MoE Refactor to Deepseekv2 To Support Fp8 (#6417) Robert Shaw 2024-07-13 23:03:58 -04:00
• eeceadaecc [Misc] Add deprecation warning for beam search (#6402) Woosuk Kwon 2024-07-13 11:52:22 -07:00
• babf52dade [ Misc ] More Cleanup of Marlin (#6359) Robert ShawandRobert Shaw 2024-07-13 06:21:37 -04:00
• 9da4aad44b Updating LM Format Enforcer version to v10.3 (#6411) Noam Gat 2024-07-13 13:09:12 +03:00
• 41708e5034 [ci] try to add multi-node tests (#6280) youkaichaoandMuralidhar Andoorveedu 2024-07-12 21:51:48 -07:00
• d80aef3776 [Docs] Clean up latest news (#6401) Woosuk Kwon 2024-07-12 19:36:53 -07:00
• e1684a766a [Bugfix] Fix hard-coded value of x in context_attention_fwd (#6373) Thomas Parnell 2024-07-13 03:30:54 +02:00
• a27f87da34 [Doc] Fix Typo in Doc (#6392) Saliya EkanayakeandSaliya Ekanayake 2024-07-12 17:48:23 -07:00
• 16ff6bd58c [ci] Fix wording for GH bot (#6398) Kevin H. Luu 2024-07-12 16:34:37 -07:00
• f8f9ff57ee [Bugfix][TPU] Fix megacore setting for v5e-litepod (#6397) Woosuk Kwon 2024-07-12 15:59:47 -07:00
• 6bc9710f6e Fix release pipeline's dir permission (#6391) Simon Mo 2024-07-12 15:52:43 -07:00
• 111fc6e7ec [Misc] Add generated git commit hash as vllm.__commit__ (#6386) Michael Goin 2024-07-12 18:52:15 -04:00
• 75f64d8b94 [Bugfix] Fix illegal memory access in FP8 MoE kernel (#6382) Cody Yu 2024-07-12 14:33:33 -07:00
• 21b2dcedab Fix release pipeline's -e flag (#6390) Simon Mo 2024-07-12 14:08:04 -07:00
• 07b35af86d Fix interpolation in release pipeline (#6389) Simon Mo 2024-07-12 14:03:39 -07:00
• bb1a784b05 Fix release-pipeline.yaml (#6388) Simon Mo 2024-07-12 14:00:57 -07:00
• d719ba24c5 Build some nightly wheels by default (#6380) Simon Mo 2024-07-12 13:56:59 -07:00
• aa48e502fb [MISC] Upgrade dependency to PyTorch 2.3.1 (#5327) Cody Yu 2024-07-12 12:04:26 -07:00
• 4dbebd03cc [ci] Add GHA workflows to enable full CI run (#6381) Kevin H. Luu 2024-07-12 11:36:26 -07:00
• b75bce1008 [ci] Add grouped tests & mark tests to run by default for fastcheck pipeline (#6365) Kevin H. Luu 2024-07-12 09:58:38 -07:00
• b039cbbce3 [Misc] add fixture to guided processor tests (#6341) Yihuan Bu 2024-07-12 12:55:39 -04:00
• f9d25c2519 [Build/CI] Checking/Waiting for the GPU's clean state (#6379) Alexei-V-Ivanov-AMD 2024-07-12 11:42:24 -05:00
• 024ad87cdc [Bugfix] Fix dtype mismatch in PaliGemma (#6367) Cyrus Leung 2024-07-12 23:22:18 +08:00
• aea19f0989 [ Misc ] Support Models With Bias in compressed-tensors integration (#6356) Robert Shaw 2024-07-12 11:11:29 -04:00
• f7160d946a [Misc][Bugfix] Update transformers for tokenizer issue (#6364) Roger Wang 2024-07-12 01:40:07 -07:00
• 6047187cd8 [ Misc ] Remove separate bias add (#6353) Robert Shaw 2024-07-12 01:06:09 -04:00
• b6c16cf8ff [ROCm][AMD] unify CUDA_VISIBLE_DEVICES usage in cuda/rocm (#6352) Hongxia Yang 2024-07-12 00:30:46 -04:00
• d26a8b3f1f [CI/Build] (2/2) Switching AMD CI to store images in Docker Hub (#6350) adityagoel14 2024-07-12 00:26:26 -04:00
• d59eb98489 [Model][Phi3-Small] Remove scipy from blocksparse_attention (#6343) Michael Goin 2024-07-11 22:47:17 -04:00
• adf32e0a0f [Bugfix] Fix usage stats logging exception warning with OpenVINO (#6349) Helena Kloosterman 2024-07-12 04:47:00 +02:00
• 2b0fb53481 [distributed][misc] be consistent with pytorch for libcudart.so (#6346) youkaichao 2024-07-11 19:35:17 -07:00
• d6ab528997 [Misc] Remove flashinfer warning, add flashinfer tests to CI (#6351) Lily Liu 2024-07-11 18:32:06 -07:00
• 7ed6a4f0e1 [ BugFix ] Prompt Logprobs Detokenization (#6223) Robert ShawandZifei Tong 2024-07-11 18:02:29 -04:00
• a4feba929b [CI/Build] Add nightly benchmarking for tgi, tensorrt-llm and lmdeploy (#5362) Kuntai Du 2024-07-11 13:28:38 -07:00
• 2d23b42d92 [doc] update pipeline parallel in readme (#6347) youkaichao 2024-07-11 11:38:40 -07:00
• 1df43de9bb [bug fix] Fix llava next feature size calculation. (#6339) xwjiang2010 2024-07-11 10:21:10 -07:00
• 52b7fcb35a Benchmark: add H100 suite (#6047) Simon Mo 2024-07-11 09:17:07 -07:00
• b675069d74 [ Misc ] Refactor Marlin Python Utilities (#6082) Robert ShawandRobert Shaw 2024-07-11 11:40:11 -04:00
• 55f692b46e [BugFix] get_and_reset only when scheduler outputs are not empty (#6266) Mor Zusman 2024-07-11 17:40:20 +03:00
• 8a1415cf77 [Bugfix] GPTBigCodeForCausalLM: Remove lm_head from supported_lora_modules. (#6326) Thomas ParnellandTravis Johnson 2024-07-11 16:05:59 +02:00
• 546b101fa0 [BugFix]: fix engine timeout due to request abort (#6255) pushanandzhangyuntao.dev 2024-07-11 21:46:31 +08:00
• 3963a5335b [Misc] refactor(config): clean up unused code (#6320) aniaan 2024-07-11 17:39:07 +08:00
• c4774eb841 [Bugfix] Fix snapshot download in serving benchmark (#6318) Roger Wang 2024-07-11 00:04:05 -07:00
• fc17110bbe [BugFix]: set outlines pkg version (#6262) Lim Xiang Yang 2024-07-11 12:37:11 +08:00
• 439c84581a [Doc] Update description of vLLM support for CPUs (#6003) Jie Fu (傅杰) 2024-07-11 12:15:29 +08:00
• 99ded1e1c4 [Doc] Remove comments incorrectly copied from another project (#6286) daquexian 2024-07-11 01:05:26 +01:00
• 997df46a32 [Bugfix][Neuron] Fix soft prompt method error in NeuronExecutor (#6313) Woosuk Kwon 2024-07-10 16:39:02 -07:00
• ae151d73be [Speculative Decoding] Enabling bonus token in speculative decoding for KV cache based models (#5765) sroy745 2024-07-10 16:02:47 -07:00
• 44cc76610d [Bugfix] Fix OpenVINOExecutor abstractmethod error (#6296) sangjune.park 2024-07-11 02:03:32 +09:00
• b422d4961a [CI/Build] Enable mypy typing for remaining folders (#6268) Benjamin Muskalla 2024-07-10 16:15:55 +02:00
• c38eba3046 [Bugfix] MLPSpeculator: Use ParallelLMHead in tie_weights=False case. (#6303) Thomas Parnell 2024-07-10 15:04:07 +02:00