Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 18e9e1f7b3 [HotFix] Fix final output truncation with stop string + streaming (#8468) Nick Hill 2024-09-13 19:31:12 +01:00
• f57092c00b [Doc] Add oneDNN installation to CPU backend documentation (#8467) Isotr0py 2024-09-14 02:06:30 +08:00
• a84e598e21 [CI/Build] Reorganize models tests (#7820) Cyrus Leung 2024-09-14 01:20:06 +08:00
• 0a4806f0a9 [plugin][torch.compile] allow to add custom compile backend (#8445) youkaichao 2024-09-13 09:32:42 -07:00
• ecd7a1d5b6 [Installation] Gate FastAPI version for Python 3.8 (#8456) Cyrus Leung 2024-09-14 00:02:26 +08:00
• a2469127db [misc][ci] fix quant test (#8449) youkaichao 2024-09-13 02:20:14 -07:00
• 06311e2956 [Misc] Skip loading extra bias for Qwen2-VL GPTQ-Int8 (#8442) Jee Jee Li 2024-09-13 15:58:28 +08:00
• cab69a15e4 [doc] recommend pip instead of conda (#8446) youkaichao 2024-09-12 23:52:41 -07:00
• 9b4a3b235e [CI/Build] Enable InternVL2 PP test only on single node (#8437) Isotr0py 2024-09-13 14:35:20 +08:00
• acda0b35d0 bump version to v0.6.1.post1 (#8440) Simon Mo 2024-09-12 21:39:49 -07:00
• ba77527955 [bugfix] torch profiler bug for single gpu with GPUExecutor (#8354) William Lin 2024-09-12 21:30:00 -07:00
• 6821020109 [Bugfix] Fix async log stats (#8417) Alexander Matveev 2024-09-12 23:48:59 -04:00
• 8427550488 [CI/Build] Update pixtral tests to use JSON (#8436) Cyrus Leung 2024-09-13 11:47:52 +08:00
• 3f79bc3d1a [Bugfix] Bump fastapi and pydantic version (#8435) Cyrus Leung 2024-09-13 11:21:42 +08:00
• 40c396533d [Bugfix] Mapping physical device indices for e2e test utils (#8290) shangmingc 2024-09-13 11:06:28 +08:00
• 5ec9c0fb3c [Core] Factor out input preprocessing to a separate class (#7329) Cyrus Leung 2024-09-13 10:56:13 +08:00
• 8f44a92d85 [BugFix] fix group_topk (#8430) Dipika Sikka 2024-09-12 21:23:42 -04:00
• 360ddbd37e [Misc] Update Pixtral example (#8431) Roger Wang 2024-09-12 17:31:18 -07:00
• a480939e8e [Bugfix] Fix weight loading issue by rename variable. (#8293) Wenxiang 2024-09-13 07:25:00 +08:00
• d31174a4e1 [Hotfix][Pixtral] Fix multiple images bugs (#8415) Patrick von Platen 2024-09-13 00:21:51 +02:00
• b61bd98f90 [CI/Build] Disable multi-node test for InternVL2 (#8428) Roger Wang 2024-09-12 15:05:35 -07:00
• c16369455f [Hotfix][Core][VLM] Disable chunked prefill by default and prefix caching for multimodal models (#8425) Roger Wang 2024-09-12 14:06:51 -07:00
• 019877253b [Bugfix] multi-step + flashinfer: ensure cuda graph compatible (#8427) Alexander Matveev 2024-09-12 17:01:50 -04:00
• 551ce01078 [Core] Add engine option to return only deltas or final output (#7381) Nick Hill 2024-09-12 20:02:00 +01:00
• a6c0f3658d [multi-step] add flashinfer backend (#7928) William Lin 2024-09-12 11:16:22 -07:00
• f2e263b801 [Bugfix] Offline mode fix (#8376) Joe Runde 2024-09-12 12:11:57 -06:00
• 1f0c75afa9 [BugFix] Fix Duplicate Assignment in Hermes2ProToolParser (#8423) Luis Vega 2024-09-12 11:10:11 -07:00
• 8a23e93302 [BugFix] lazy init _copy_stream to avoid torch init wrong gpu instance (#8403) WANGWEI 2024-09-13 01:47:42 +08:00
• c6202daeed [Model] Support multiple images for qwen-vl (#8247) 3 people 2024-09-12 11:10:54 -06:00
• e56bf27741 [Bugfix] Fix InternVL2 inference with various num_patches (#8375) Isotr0pyandDarkLight1337 2024-09-13 01:10:35 +08:00
• 520ca380ae [Hotfix][VLM] Fixing max position embeddings for Pixtral (#8399) Roger Wang 2024-09-12 09:28:37 -07:00
• 7de49aa86c [torch.compile] hide slicing under custom op for inductor (#8384) youkaichao 2024-09-12 00:11:55 -07:00
• 42ffba11ad [Misc] Use RoPE cache for MRoPE (#8396) Woosuk Kwon 2024-09-11 23:13:14 -07:00
• 295c4730a8 [Misc] Raise error when using encoder/decoder model with cpu backend (#8355) Kevin Lin 2024-09-12 00:45:24 -05:00
• 1bf2dd9df0 [Gemma2] add bitsandbytes support for Gemma2 (#8338) Blueyo0 2024-09-12 12:53:12 +08:00
• 5a60699c45 [Bugfix]: Fix the logic for deciding if tool parsing is used (#8366) tomeras91 2024-09-12 06:55:30 +03:00
• b6c75e1cf2 Fix the AMD weight loading tests (#8390) Michael Goin 2024-09-11 23:35:33 -04:00
• b71c956deb [TPU] Use Ray for default distributed backend (#8389) Woosuk Kwon 2024-09-11 20:31:51 -07:00
• f842a7aff1 [misc] remove engine_use_ray (#8126) youkaichao 2024-09-11 18:23:36 -07:00
• a65cb16067 [MISC] Dump model runner inputs when crashing (#8305) Cody Yu 2024-09-11 18:12:25 -07:00
• 3fd2b0d21c Bump version to v0.6.1 (#8379) Simon Mo 2024-09-11 14:42:11 -07:00
• d394787e52 Pixtral (#8377) Patrick von PlatenandRoger Wang 2024-09-11 23:41:55 +02:00
• 775f00f81e [Speculative Decoding] Test refactor (#8317) Lily Liuandyoukaichao 2024-09-11 14:07:34 -07:00
• 8baa454937 [Misc] Move device options to a single place (#8322) Aarni Koskela 2024-09-11 23:25:58 +03:00
• 73202dbe77 [Kernel][Misc] register ops to prevent graph breaks (#6917) bnellnmandSage Moore 2024-09-11 15:52:19 -04:00
• 7015417fd4 [Bugfix] Add missing attributes in mistral tokenizer (#8364) Cyrus Leung 2024-09-12 02:36:54 +08:00
• aea02f30de [CI/Build] Excluding test_moe.py from AMD Kernels tests for investigation (#8373) Alexey Kondratiev(AMD) 2024-09-11 14:31:41 -04:00
• 0b952af458 [Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257) Li, Jiang 2024-09-12 00:46:46 +08:00
• 3b7fea770f [Model][VLM] Add Qwen2-VL model support (#7905) 3 people 2024-09-12 00:31:19 +08:00
• cea95dfb94 [Frontend] Create ErrorResponse instead of raising exceptions in run_batch (#8347) Pooya Davoodi 2024-09-10 22:30:11 -07:00
• 6a512a00df [model] Support for Llava-Next-Video model (#7559) 4 people 2024-09-11 13:21:36 +08:00
• efcf946a15 [Hardware][NV] Add support for ModelOpt static scaling checkpoints. (#6112) Pavani Majety 2024-09-10 21:38:40 -07:00
• 1230263e16 [Bugfix] Fix InternVL2 vision embeddings process with pipeline parallel (#8299) Isotr0py 2024-09-11 10:11:01 +08:00
• e497b8aeff [Misc] Skip loading extra bias for Qwen2-MOE GPTQ models (#8329) Jee Jee Li 2024-09-11 08:59:19 +08:00
• 94144e726c [CI/Build][Kernel] Update CUTLASS to 3.5.1 tag (#8043) Tyler Michael Smith 2024-09-10 19:51:58 -04:00
• 1d5e397aa4 [Core/Bugfix] pass VLLM_ATTENTION_BACKEND to ray workers (#8172) William Lin 2024-09-10 16:46:08 -07:00
• 22f3a4bc6c [Bugfix] lookahead block table with cuda graph max capture (#8340) Alexander Matveev 2024-09-10 19:00:35 -04:00
• b1f3e18958 [MISC] Keep chunked prefill enabled by default with long context when prefix caching is enabled (#8342) Cody Yu 2024-09-10 15:28:28 -07:00
• 04e7c4e771 [Misc] remove peft as dependency for prompt models (#8162) Prashant Gupta 2024-09-10 14:21:56 -07:00
• 5faedf1b62 [Spec Decode] Move ops.advance_step to flash attn advance_step (#8224) Kevin Lin 2024-09-10 15:18:14 -05:00
• 02751a7a42 Fix ppc64le buildkite job (#8309) sumitd2 2024-09-11 01:28:34 +05:30
• f421f3cefb [CI/Build] Enabling kernels tests for AMD, ignoring some of then that fail (#8130) Alexey Kondratiev(AMD) 2024-09-10 14:51:15 -04:00
• 8c054b7a62 [Frontend] Clean up type annotations for mistral tokenizer (#8314) Cyrus Leung 2024-09-11 00:49:11 +08:00
• 6234385f4a [CI/Build] enable ccache/scccache for HIP builds (#8327) Daniele 2024-09-10 17:55:08 +02:00
• da1a844e61 [Bugfix] Fix missing post_layernorm in CLIP (#8155) Cyrus Leung 2024-09-10 16:22:50 +08:00
• a1d874224d Add NVIDIA Meetup slides, announce AMD meetup, and add contact info (#8319) Simon Mo 2024-09-09 23:21:00 -07:00
• 6cd5e5b07e [Misc] Fused MoE Marlin support for GPTQ (#8217) Dipika Sikka 2024-09-09 23:02:52 -04:00
• c7cb5c3335 [Misc] GPTQ Activation Ordering (#8135) Kyle Sayers 2024-09-09 16:27:26 -04:00
• f9b4a2d415 [Bugfix] Correct adapter usage for cohere and jamba (#8292) Vladislav Kruglikov 2024-09-09 21:20:46 +03:00
• 58fcc8545a [Frontend] Add progress reporting to run_batch.py (#8060) Adam LugowskiandAdam Lugowski 2024-09-09 11:16:37 -07:00
• 08287ef675 [Bugfix] Streamed tool calls now more strictly follow OpenAI's format; ensures Vercel AI SDK compatibility (#8272) Kyle Mistele 2024-09-09 09:45:11 -05:00
• 4ef41b8476 [Bugfix] Fix async postprocessor in case of preemption (#8267) Alexander Matveev 2024-09-08 00:01:51 -04:00
• cfe712bf1a [CI/Build] Use python 3.12 in cuda image (#8133) Joe Runde 2024-09-07 14:03:16 -06:00
• b962ee1470 ppc64le: Dockerfile fixed, and a script for buildkite (#8026) sumitd2 2024-09-07 23:48:40 +05:30
• 36bf8150cc [Model][VLM] Decouple weight loading logic for Paligemma (#8269) Isotr0py 2024-09-08 01:45:44 +08:00
• e807125936 [Model][VLM] Support multi-images inputs for InternVL2 models (#8201) Isotr0py 2024-09-07 16:38:23 +08:00
• 9f68e00d27 [Bugfix] Fix broken OpenAI tensorizer test (#8258) Cyrus Leung 2024-09-07 16:02:39 +08:00
• ce2702a923 [tpu][misc] fix typo (#8260) youkaichao 2024-09-06 22:40:46 -07:00
• 795b662cff Enable Random Prefix Caching in Serving Profiling Tool (benchmark_serving.py) (#8241) Wei-Sheng Chin 2024-09-06 20:18:16 -07:00
• 2f707fcb35 [Model] Multi-input support for LLaVA (#8238) Cyrus Leung 2024-09-07 10:57:24 +08:00
• 41e95c5247 [Bugfix] Fix Hermes tool call chat template bug (#8256) Kyle MisteleandKyle Mistele 2024-09-06 21:49:01 -05:00
• 12dd715807 [misc] [doc] [frontend] LLM torch profiler support (#7943) William Lin 2024-09-06 17:48:48 -07:00
• 29f49cd6e3 [Model] Allow loading from original Mistral format (#8168) Patrick von PlatenandMichael Goin 2024-09-07 01:02:05 +02:00
• 23f322297f [Misc] Remove SqueezeLLM (#8220) Dipika Sikka 2024-09-06 18:29:03 -04:00
• 9db52eab3d [Kernel] [Triton] Memory optimization for awq_gemm and awq_dequantize, 2x throughput (#8248) rasmith 2024-09-06 17:26:09 -05:00
• 1447c97e75 [CI/Build] Increasing timeout for multiproc worker tests (#8203) Alexey Kondratiev(AMD) 2024-09-06 14:51:03 -04:00
• de80783b69 [Misc] Use ray[adag] dependency instead of cuda (#7938) Rui Qiao 2024-09-06 09:18:35 -07:00
• e5cab71531 [Frontend] Add --logprobs argument to benchmark_serving.py (#8191) afeldman-nm 2024-09-06 12:01:14 -04:00
• baa5467547 [BugFix] Fix Granite model configuration (#8216) Nick Hill 2024-09-05 20:39:29 -07:00
• db3bf7c991 [Core] Support load and unload LoRA in api server (#6566) Jiaxin ShanandJee Jee Li 2024-09-05 18:10:33 -07:00
• 2febcf2777 [Documentation][Spec Decode] Add documentation about lossless guarantees in Speculative Decoding in vLLM (#7962) sroy745 2024-09-05 13:25:29 -07:00
• 2ee45281a5 Move verify_marlin_supported to GPTQMarlinLinearMethod (#8165) Michael Goin 2024-09-05 11:09:46 -04:00
• 9da25a88aa [MODEL] Qwen Multimodal Support (Qwen-VL / Qwen-VL-Chat) (#8029) Alex BrooksandDarkLight1337 2024-09-05 06:48:10 -06:00
• 8685ba1a1e Inclusion of InternVLChatModel In PP_SUPPORTED_MODELS(Pipeline Parallelism) (#7860) manikandan.tm@zucisystems.com 2024-09-05 17:03:37 +05:30
• 288a938872 [Doc] Indicate more information about supported modalities (#8181) Cyrus Leung 2024-09-05 18:51:53 +08:00
• e39ebf5cf5 [Core/Bugfix] Add query dtype as per FlashInfer API requirements. (#8173) Elfie Guo 2024-09-04 22:12:26 -07:00
• ba262c4e5a [ci] Mark LoRA test as soft-fail (#8160) Kevin H. Luu 2024-09-04 20:33:12 -07:00
• 4624d98dbd [Misc] Clean up RoPE forward_native (#8076) Woosuk Kwon 2024-09-04 20:31:48 -07:00
• 1afc931987 [bugfix] >1.43 constraint for openai (#8169) William LinandMichael Goin 2024-09-04 17:35:36 -07:00
• e01c2beb7d [Doc] [Misc] Create CODE_OF_CONDUCT.md (#8161) Maureen McElaney 2024-09-04 19:50:13 -04:00