Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• b05f5c9238 [Core] Allow IPv6 in VLLM_HOST_IP with zmq (#8575) Russell Bryant 2024-09-23 15:15:41 -04:00
• 9b0e3ec970 [Kernel][LoRA] Add assertion for punica sgmv kernels (#7585) Jee Jee Li 2024-09-24 02:57:42 +08:00
• 86e9c8df29 [Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GPTQMarlin (#7701) 4 people 2024-09-23 13:46:26 -04:00
• ee5f34b1c2 [CI/Build] use setuptools-scm to set __version__ (#4738) Danieleandyoukaichao 2024-09-23 18:44:26 +02:00
• f2bd246c17 [VLM] Fix paligemma, fuyu and persimmon with transformers 4.45 : use config.text_config.vocab_size (#8707) Jani Monoses 2024-09-23 17:43:09 +03:00
• a79e522984 [Model] Support pp for qwen2-vl (#8696) Yanyi Liu 2024-09-23 21:46:59 +08:00
• 3e83c12b5c [Bugfix][CPU] fix missing input intermediate_tensors in the cpu_model_runner (#8733) Li, Jiang 2024-09-23 21:15:16 +08:00
• e551ca1555 [Hardware][CPU] Refactor CPU model runner (#8729) Isotr0py 2024-09-23 20:12:20 +08:00
• 9b8c8ba119 [Core][Frontend] Support Passing Multimodal Processor Kwargs (#8657) Alex Brooks 2024-09-23 01:44:48 -06:00
• d23679eb99 [Bugfix] fix docker build for xpu (#8652) Yan Ma 2024-09-23 13:54:18 +08:00
• 57a0702e63 [Bugfix] Fix CPU CMake build (#8723) Luka GovedičandYuan 2024-09-22 23:40:46 -04:00
• 3dda7c2250 [Bugfix] Avoid some bogus messages RE CUTLASS's revision when building (#8702) Tyler Michael Smith 2024-09-22 22:24:59 -04:00
• 92ba7e7477 [misc] upgrade mistral-common (#8715) youkaichao 2024-09-22 15:41:59 -07:00
• d4a2ac8302 [build] enable existing pytorch (for GH200, aarch64, nightly) (#8713) youkaichao 2024-09-22 12:47:54 -07:00
• c6bd70d772 [SpecDec][Misc] Cleanup, remove bonus token logic. (#8701) Lily Liu 2024-09-22 12:34:14 -07:00
• 5b59532760 [Model][VLM] Add LLaVA-Onevision model support (#8486) 5 people 2024-09-23 01:51:44 +08:00
• ca2b628b3c [MISC] rename CudaMemoryProfiler to DeviceMemoryProfiler (#8703) Huazhong Ji 2024-09-23 01:44:09 +08:00
• 8ca5051b9a [Misc] Use NamedTuple in Multi-image example (#8705) Alex Brooks 2024-09-22 06:56:20 -06:00
• 06ed2815e2 [Model] Refactor BLIP/BLIP-2 to support composite model loading (#8407) Cyrus Leung 2024-09-22 20:24:21 +08:00
• 0e40ac9b7b [ci][build] fix vllm-flash-attn (#8699) youkaichao 2024-09-21 23:24:58 -07:00
• 13d88d4137 [Bugfix] Refactor composite weight loading logic (#8656) Isotr0py 2024-09-22 12:33:27 +08:00
• d66ac62854 [Kernel][Bugfix] Delete some more useless code in marlin_moe_ops.cu (#8643) Tyler Michael Smith 2024-09-21 19:45:02 -04:00
• 9dc7c6c7f3 [dbrx] refactor dbrx experts to extend FusedMoe class (#8518) Divakar Verma 2024-09-21 16:09:39 -05:00
• ec4aaad812 [Kernel][Triton][AMD] Remove tl.atomic_add from awq_gemm_kernel, 2-5x speedup MI300, minor improvement for MI250 (#8646) rasmith 2024-09-21 04:20:54 -05:00
• 4dfdf43196 [Doc] Fix typo in AMD installation guide (#8689) Andy Dai 2024-09-21 00:24:12 -07:00
• 5e85f4f82a [VLM] Use SequenceData.from_token_counts to create dummy data (#8687) Cyrus Leung 2024-09-21 14:28:56 +08:00
• 71c60491f2 [Kernel] Build flash-attn from source (#8245) Luka Govedič 2024-09-21 02:27:10 -04:00
• 0faab90eb0 [beam search] add output for manually checking the correctness (#8684) youkaichao 2024-09-20 19:55:33 -07:00
• 0455c46ed4 [Core] Factor out common code in SequenceData and Sequence (#8675) Cyrus Leung 2024-09-21 10:30:39 +08:00
• d4bf085ad0 [MISC] add support custom_op check (#8557) Kunshang Jiandyoukaichao 2024-09-21 10:03:55 +08:00
• 0057894ef7 [Core] Rename PromptInputs and inputs(#8673) Cyrus Leung 2024-09-21 10:00:54 +08:00
• 0f961b3ce9 [Bugfix] Fix incorrect llava next feature size calculation (#8496) zyddnys 2024-09-20 18:48:32 -04:00
• 7f9c8902e3 [Hardware][AWS] update neuron to 2.20 (#8676) omrishiv 2024-09-20 15:19:44 -07:00
• 7c8566aa4f [Doc] neuron documentation update (#8671) omrishiv 2024-09-20 15:04:37 -07:00
• b4e4eda92e [Bugfix][Core] Fix tekken edge case for mistral tokenizer (#8640) Patrick von Platen 2024-09-20 23:33:03 +02:00
• 2874bac618 [Bugfix] Config got an unexpected keyword argument 'engine' (#8556) Pastel！ 2024-09-21 05:00:45 +08:00
• 035fa895ec [Misc] Show AMD GPU topology in collect_env.py (#8649) Cyrus Leung 2024-09-21 04:52:19 +08:00
• b28298f2f4 [Bugfix] Validate SamplingParam n is an int (#8548) saumya-saran 2024-09-20 12:46:02 -07:00
• 2940afa04e [CI/Build] Removing entrypoints/openai/test_embedding.py test from ROCm build (#8670) Alexey Kondratiev(AMD) 2024-09-20 13:27:44 -04:00
• 3b63de9353 [Model] Add OLMoE (#7922) Niklas Muennighoff 2024-09-20 09:31:41 -07:00
• 260d40b5ea [Core] Support Lora lineage and base model metadata management (#6315) Jiaxin Shan 2024-09-19 23:20:56 -07:00
• 9e5ec35b1f [bugfix] [AMD] add multi-step advance_step to ROCmFlashAttentionMetadata (#8474) William Lin 2024-09-19 20:49:54 -07:00
• 18ae428a0d [Bugfix] Fix Phi3.5 mini and MoE LoRA inference (#8571) Amit Garg 2024-09-19 17:54:02 -07:00
• de6f90a13d [Misc] guard against change in cuda library name (#8609) bnellnm 2024-09-19 18:36:30 -04:00
• 6cb748e190 [CI/Build] Re-enabling Entrypoints tests on ROCm, excluding ones that fail (#8551) Alexey Kondratiev(AMD) 2024-09-19 16:06:32 -04:00
• 9e99407e3c Create SECURITY.md (#8642) Simon Mo 2024-09-19 12:16:28 -07:00
• ea4647b7d7 [Doc] Add documentation for GGUF quantization (#8618) Isotr0py 2024-09-20 03:15:55 +08:00
• e42c634acb [Core] simplify logits resort in _apply_top_k_top_p (#8619) 盏一 2024-09-20 02:28:25 +08:00
• 9cc373f390 [Kernel][Amd] Add fp8 kv cache support for rocm custom paged attention (#8577) Charlie Fu 2024-09-19 12:37:57 -05:00
• 76515f303b [Frontend] Use MQLLMEngine for embeddings models too (#8584) Nick Hill 2024-09-19 17:51:06 +01:00
• 855c8ae2c9 [MISC] remove engine_use_ray in benchmark_throughput.py (#8615) Kunshang Ji 2024-09-19 13:33:20 +08:00
• c52ec5f034 [Bugfix] fixing sonnet benchmark bug in benchmark_serving.py (#8616) Kuntai Du 2024-09-18 22:24:24 -07:00
• 02c9afa2d0 Revert "[Misc][Bugfix] Disable guided decoding for mistral tokenizer" (#8593) Roger Wang 2024-09-18 21:14:28 -07:00
• 3118f63385 [Bugfix] [Encoder-Decoder] Bugfix for encoder specific metadata construction during decode of encoder-decoder models. (#8545) sroy745 2024-09-18 19:24:15 -07:00
• 4c34ce8916 [Kernel] Remove marlin moe templating on thread_m_blocks (#8573) Tyler Michael Smithand 2024-09-18 21:42:49 -04:00
• 0d47bf3bf4 [Bugfix] add dead_error property to engine client (#8574) Joe Runde 2024-09-18 16:10:01 -06:00
• d9cd78eb71 [BugFix] Nonzero exit code if MQLLMEngine startup fails (#8572) Nick Hill 2024-09-18 21:17:55 +01:00
• db9120cded [Kernel] Change interface to Mamba selective_state_update for continuous batching (#8039) Tyler Michael Smith 2024-09-18 16:05:06 -04:00
• b3195bc9e4 [AMD][ROCm]Quantization methods on ROCm; Fix _scaled_mm call (#8380) 3 people 2024-09-18 13:41:08 -04:00
• e18749ff09 [Model] Support Solar Model (#8386) Geun, LimandMichael Goin 2024-09-19 02:04:00 +09:00
• d65798f78c [Core] zmq: bind only to 127.0.0.1 for local-only usage (#8543) Russell Bryant 2024-09-18 12:10:27 -04:00
• a8c1d161a7 [Core] *Prompt* logprobs support in Multi-step (#8199) afeldman-nm 2024-09-18 11:38:43 -04:00
• 7c7714d856 [Core][Bugfix][Perf] Introduce MQLLMEngine to avoid asyncio OH (#8157) 5 people 2024-09-18 09:56:58 -04:00
• 9d104b5beb [CI/Build] Update Ruff version (#8469) Aaron PhamandCyrus Leung 2024-09-18 07:00:56 -04:00
• 6ffa3f314c [CI/Build] Avoid CUDA initialization (#8534) Cyrus Leung 2024-09-18 18:38:11 +08:00
• e351572900 [Misc] Add argument to disable FastAPI docs (#8554) Jiaxin Shan 2024-09-18 02:51:59 -07:00
• 95965d31b6 [CI/Build] fix Dockerfile.cpu on podman (#8540) Daniele 2024-09-18 04:49:53 +02:00
• 8110e44529 [Kernel] Change interface to Mamba causal_conv1d_update for continuous batching (#8012) Tyler Michael Smith 2024-09-17 19:44:27 -04:00
• 09deb4721f [CI/Build] Excluding kernels/test_gguf.py from ROCm (#8520) Alexey Kondratiev(AMD) 2024-09-17 19:40:29 -04:00
• fa0c114fad [doc] improve installation doc (#8550) youkaichaoandAndy Dai 2024-09-17 16:24:06 -07:00
• 98f9713399 [Bugfix] Fix TP > 1 for new granite (#8544) Joe Runde 2024-09-17 17:17:08 -06:00
• 56c3de018c [Misc] Don't dump contents of kvcache tensors on errors (#8527) Nick Hill 2024-09-17 20:24:29 +01:00
• a54ed80249 [Model] Add mistral function calling format to all models loaded with "mistral" format (#8515) Patrick von PlatenandCyrus Leung 2024-09-17 19:50:37 +02:00
• 9855b99502 [Feature][kernel] tensor parallelism with bitsandbytes quantization (#8434) chenqianfzh 2024-09-17 08:09:12 -07:00
• 1009e93c5d [Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631) sroy745 2024-09-17 07:35:01 -07:00
• 1b6de8352b [Benchmark] Support sample from HF datasets and image input for benchmark_serving (#8495) Isotr0py 2024-09-17 15:34:27 +08:00
• cbdb252259 [Misc] Limit to ray[adag] 2.35 to avoid backward incompatible change (#8509) Rui Qiao 2024-09-17 00:06:26 -07:00
• 99aa4eddaf [torch.compile] register allreduce operations as custom ops (#8526) youkaichao 2024-09-16 22:57:57 -07:00
• ee2bceaaa6 [Misc][Bugfix] Disable guided decoding for mistral tokenizer (#8521) Roger Wang 2024-09-16 22:22:45 -07:00
• 1c1bb388e0 [Frontend] Improve Nullable kv Arg Parsing (#8525) Alex Brooks 2024-09-16 22:17:32 -06:00
• 546034b466 [refactor] remove triton based sampler (#8524) Simon Mo 2024-09-16 20:04:48 -07:00
• cca61642e0 [Bugfix] Fix 3.12 builds on main (#8510) Joe Runde 2024-09-16 18:01:45 -06:00
• 5ce45eb54d [misc] small qol fixes for release process (#8517) Simon Mo 2024-09-16 15:11:27 -07:00
• 5478c4b41f [perf bench] set timeout to debug hanging (#8516) Simon Mo 2024-09-16 14:30:02 -07:00
• 47f5e03b5b [Bugfix] Bind api server port before starting engine (#8491) Kevin Lin 2024-09-16 15:56:28 -05:00
• 2759a43a26 [doc] update doc on testing and debugging (#8514) youkaichao 2024-09-16 12:10:23 -07:00
• 5d73ae49d6 [Kernel] AQ AZP 3/4: Asymmetric quantization kernels (#7270) Luka Govedič 2024-09-16 14:52:40 -04:00
• 781e3b9a42 [Bugfix][Kernel] Fix build for sm_60 in GGUF kernel (#8506) sasha0552 2024-09-16 18:15:57 +00:00
• acd5511b6d [BugFix] Fix clean shutdown issues (#8492) Nick Hill 2024-09-16 17:33:46 +01:00
• 837c1968f9 [Frontend] Expose revision arg in OpenAI server (#8501) lewtun 2024-09-16 17:55:26 +02:00
• a091e2da3e [Kernel] Enable 8-bit weights in Fused Marlin MoE (#8032) ElizaWszolaandDipika 2024-09-16 17:47:19 +02:00
• fc990f9795 [Bugfix][Kernel] Add IQ1_M quantization implementation to GGUF kernel (#8357) Isotr0py 2024-09-16 06:51:44 +08:00
• 3724d5f6b5 [Bugfix][Model] Fix Python 3.8 compatibility in Pixtral model by updating type annotations (#8490) Chris 2024-09-15 06:20:05 +02:00
• 50e9ec41fc [TPU] Implement multi-step scheduling (#8489) Woosuk Kwon 2024-09-14 16:58:31 -07:00
• 47790f3e32 [torch.compile] add a flag to disable custom op (#8488) youkaichao 2024-09-14 13:07:16 -07:00
• a36e070dad [torch.compile] fix functionalization (#8480) youkaichao 2024-09-14 09:46:04 -07:00
• 8a0cf1ddc3 [Model] support minicpm3 (#8297) ywfangandDarkLight1337 2024-09-14 22:50:26 +08:00
• 1ef0d2efd0 [Kernel][Hardware][Amd]Custom paged attention kernel for rocm (#8310) Charlie Fu 2024-09-13 19:01:11 -05:00
• 851725202a [Hardware][intel GPU] bump up ipex version to 2.3 (#8365) Kunshang JiandYan Ma 2024-09-14 07:54:34 +08:00
• 9ba0817ff1 bump version to v0.6.1.post2 (#8473) Simon Mo 2024-09-13 11:35:00 -07:00