Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 2689d5c027 [Model] Use autoweightloader for mamba (#16950) Flora Feng 2025-04-22 00:48:15 -07:00
• acba33a0f1 [Bugfix] Fix the issue where llm.generate cannot be called repeatedly after setting GuidedDecodingParams (#16767) ChaunceyandRussell Bryant 2025-04-22 14:02:20 +08:00
• a114bf20a3 [Perf] Optimize _update_states for GPU model runner (#16910) SnowCharm 2025-04-22 14:01:54 +08:00
• 3097ce3a32 [Doc] Update ai_accelerator/hpu-gaudi.inc.md (#16956) Michael Yao 2025-04-22 13:33:27 +08:00
• d6da9322c8 [Bugfix] Fix f-string for Python 3.9-3.11 (#16962) Cyrus Leung 2025-04-22 12:45:55 +08:00
• 71ce44047f Support S3 Sharded loading with RunAI Model Streamer (#16317) omer-dayanandCyrus Leung 2025-04-22 07:21:49 +03:00
• 188b7f9b8c [Performance][ROCm] Add skinny gemms for unquantized linear on ROCm (#15830) Charlie FuandTyler Michael Smith 2025-04-21 22:46:22 -05:00
• b9b4746950 [V1] Remove additional_config check (#16710) wangxiyuan 2025-04-22 11:45:27 +08:00
• 7b8a2ab76f [Kernel] Add expert_map support to Cutlass FP8 MOE (#16861) Varun Sundar Rabindranathandvarun sundar rabindranath 2025-04-21 23:44:32 -04:00
• c9acbf1141 [Misc] Remove the chunked prefill warning for LoRA (#16925) Jee Jee Li 2025-04-22 11:44:24 +08:00
• 5b794cae8d [ROCm] Add aiter tkw1 kernel for Llama4 fp8 (#16727) 3 people 2025-04-22 11:42:34 +08:00
• 0e4254492f [Bugfix]: fix issue with n>1 sampling on v1 requests overriding each other (#16863) Jeffrey Li 2025-04-21 23:40:19 -04:00
• 1311913f55 [BugFix][Spec Decode] No in-place update to draft probs (#16952) Woosuk Kwon 2025-04-21 19:54:19 -07:00
• 29f395c97c [Doc] Remove unnecessary V1 flag (#16924) Cyrus Leung 2025-04-22 09:04:38 +08:00
• fa3bba2a53 [TPU][V1] Enable Top-P (#16843) Nicolò Lucchesiandmgoin 2025-04-22 02:46:07 +02:00
• 986537f1c3 [V1] V1 FlashInfer Attention (#16684) Michael GoinandAurick Qiao 2025-04-21 18:38:41 -06:00
• 210207525e [TPU][V1] Capture multimodal encoder during model compilation (#15051) 3 people 2025-04-22 02:36:59 +02:00
• 71eda0bb76 Update Qwen1.5-MoE-W4A16-compressed-tensors.yaml (#16946) Michael Goin 2025-04-21 18:35:32 -06:00
• 471fe65630 [TPU][V1] Implicitly adjust page size when there's SMEM OOM (#16871) Chengji Yao 2025-04-21 14:43:13 -07:00
• 3a0fba5cf4 [V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087) Woosuk Kwon 2025-04-21 12:38:50 -07:00
• 299ebb62b2 [Core] Speed up decode by remove synchronizing operation in sampler (#16436) Chanh NguyenandChanh Nguyen 2025-04-21 11:18:22 -07:00
• f728ab8e35 [Doc] mention how to install in CPU editable mode (#16923) David Xia 2025-04-21 13:45:51 -04:00
• 63e26fff78 [doc] install required python3-dev apt package (#16888) David Xia 2025-04-21 12:15:18 -04:00
• fe3462c774 [XPU][Bugfix] minor fix for XPU (#15591) Yan Ma 2025-04-22 00:02:57 +08:00
• 3b34fd5273 Raise error for data-parallel with benchmark_throughput (#16737) Kartik RameshandSimon Mo 2025-04-21 10:51:43 -05:00
• 55d6d3fdb8 [Bugfix] Fix GLM rotary_dim issue and support v1 (#16912) Isotr0py 2025-04-21 22:26:34 +08:00
• 7272bfae77 [Misc] Refactor platform to get device specific stream and event (#14411) Shanshan Shen 2025-04-21 21:25:49 +08:00
• d9ac9e3dc5 [Misc] fix collect_env version parse (#15267) wangxiyuan 2025-04-21 20:29:40 +08:00
• d41faaf9df Restore buffers when wake up from level 2 sleep (#16564) (#16889) Han Zhang 2025-04-21 20:18:28 +08:00
• b34f33438a [Doc] Split dummy_processor_inputs() in Multimodal Docs (#16915) Alex Brooks 2025-04-21 05:10:01 -06:00
• 26c0406555 [Bugfix] Fix distributed bug in Qwen2.5-VL & Qwen2.5-Omni (#16907) Yang Fan 2025-04-21 18:25:21 +08:00
• 4c41278b77 [CI/CD][V1] Add spec decode tests to CI (#16900) Woosuk Kwon 2025-04-20 22:37:16 -07:00
• bb3605db85 [Bugfix] Fix v1/spec_decode/test_ngram.py (#16895) qizixi 2025-04-20 20:54:29 -07:00
• fe742aef5a [easy] Pass compile_fx only the config patches (#16845) Richard Zou 2025-04-20 00:25:19 -04:00
• 4b07d36891 Improve configs - CacheConfig (#16835) Harry Mellor 2025-04-20 05:25:04 +01:00
• 87aaadef73 Serialize tensors using int8 views (#16866) Staszek PaśkoandNick Hill 2025-04-19 19:28:34 +02:00
• 682e0b6d2f Log how much time loading a compiled artifact takes (#16848) Richard Zou 2025-04-19 12:50:46 -04:00
• d6195a748b [doc] update hyperlink (#16877) Reidandreidliu41 2025-04-20 00:40:38 +08:00
• 205d84aaa9 [VLM] Clean up models (#16873) Cyrus Leung 2025-04-19 20:13:06 +08:00
• 5124f5bf51 [Model] Qwen2.5-Omni Cleanup (#16872) Roger Wang 2025-04-19 02:37:02 -07:00
• 83f3c3bd91 [Model] Refactor Phi-4-multimodal to use merged processor and support V1 (#15477) Isotr0pyandDarkLight1337 2025-04-19 17:26:11 +08:00
• d9737ca1c6 [V1][Misc] stop update prefix cache stats when logs_stats is disabled (#16460) vie-serendipity 2025-04-19 17:25:19 +08:00
• 9d4ca19d50 [Misc] Benchmarks for audio models (#16505) Nicolò Lucchesi 2025-04-19 11:24:14 +02:00
• 2ef0dc53b8 [Frontend] Add sampling params to v1/audio/transcriptions endpoint (#16591) Nicolò LucchesiandJannis Schönleber 2025-04-19 09:03:54 +02:00
• 1d4680fad2 [rocm][MI300] llama4 maverick fp8 moe config tp8 (#16847) Divakar Verma 2025-04-19 01:21:43 -05:00
• 2c1bd848a6 [Model][VLM] Add Qwen2.5-Omni model support (thinker only) (#15130) 4 people 2025-04-19 14:14:36 +08:00
• 5c9121203c [release] Publish neuron docker image (#16733) omrishiv 2025-04-19 01:11:25 +01:00
• 490b1698a5 [Doc] Updated Llama section in tool calling docs to have llama 3.2 config info (#16857) Justin Ho 2025-04-18 19:28:53 -04:00
• 5a5e29de88 [Misc] refactor examples series - Chat Completion Client With Tools (#16829) Reidandreidliu41 2025-04-19 07:24:42 +08:00
• 3d3ab3689f [New Model]: Snowflake Arctic Embed (Family) (#16649) wang.yuqi 2025-04-18 23:11:57 +08:00
• 686623c5e7 Fix nullable_kvs fallback (#16837) Harry Mellor 2025-04-18 13:58:39 +01:00
• aadb656562 [Misc] Clean up Kimi-VL (#16833) Cyrus Leung 2025-04-18 20:15:09 +08:00
• 87e067de41 [Model] use AutoWeightsLoader for BigCode, GPT-J (#16823) Jonghyun Choe 2025-04-18 19:42:41 +09:00
• 26507f8973 [Docs] Fix a link and grammar issue in production-stack.md (#16809) Michael Yao 2025-04-18 14:42:58 +08:00
• 9c1d5b456d [Doc] add podman setup instructions for official image (#16796) Nathan Weinberg 2025-04-18 02:10:49 -04:00
• e31045f95c [Bugfix] fix pp for llama4 (#16746) Lucia Fang 2025-04-17 22:51:30 -07:00
• aaec845f8e [ROCm] [Attention] Cleanup ROCm output passing (#16431) Luka Govedič 2025-04-18 01:46:45 -04:00
• 7bdfd29a35 [Misc] add collect_env to cli and docker image (#16759) rongfu.leng 2025-04-18 13:13:35 +08:00
• e78587a64c Improve-mm-and-pooler-and-decoding-configs (#16789) Harry Mellor 2025-04-18 06:13:32 +01:00
• 7eb4255628 [BugFix] Accuracy fix for llama4 int4 - improperly casted scales (#16801) Lucas Wilkinson 2025-04-18 01:13:29 -04:00
• 6a0f547561 Add hardware print to TPU V1 test (#16792) Michael Goin 2025-04-17 23:13:26 -06:00
• 30ed81b7ca [V1][Structured Output] Minor modification to _validate_structured_output() (#16748) Shanshan Shen 2025-04-18 13:12:54 +08:00
• 7a4a5de729 [Misc] Update outdated note: LMCache now supports chunked prefill (#16697) Chauncey 2025-04-18 13:12:42 +08:00
• c16fb5dae8 [Doc] Improve help examples for --compilation-config (#16729) Cyrus Leung 2025-04-18 12:22:34 +08:00
• e37073efd7 Add property-based testing for vLLM endpoints using an API defined by an OpenAPI 3.1 schema (#16721) Tarun KumarandNick Hill 2025-04-18 09:38:27 +05:30
• 183dad7a85 [Attention] Update to lastest FA3 code (#13111) Lucas Wilkinson 2025-04-17 18:14:07 -04:00
• 3408e47159 [P/D][V1] KV Connector API V1 (#15960) 5 people 2025-04-17 15:22:40 -05:00
• 0377b8310b [MLA] Simplification to batch P/D reordering (#16673) Nick Hill 2025-04-17 13:12:09 -07:00
• e4755f7fac [V1][Metrics] Fix http metrics middleware (#15894) Mark McLoughlin 2025-04-17 20:52:18 +01:00
• 92edf35826 [ROCM] enable aiter fused moe kernel for llama4 bf16 checkpoints (#16674) Sijia(Jackson) Chen 2025-04-17 11:44:34 -07:00
• eb5819b2d9 [V1][TPU] Enable Top K (#15489) Nicolò LucchesiandHyesoo Yang 2025-04-17 20:18:11 +02:00
• 5989f4684d [TPU][V1] Fix padding recompilation when max-num-batched-tokens is not even (#16726) Nicolò Lucchesi 2025-04-17 20:09:57 +02:00
• 5125d72f02 [Model] use AutoWeightsLoader for olmoe,opt,orion,persimmon,phi3_small (#16548) rongfu.leng 2025-04-18 01:48:31 +08:00
• a018e555fd [Kernel] Add fp8_w8a8 fused MoE kernel tuning configs for DeepSeek V3/R1 on NVIDIA H20 (#16753) Ximingwang-09andximing.wxm 2025-04-18 00:01:30 +08:00
• 6211b92273 [Bugfix]Fix index out of range error in api server log (#16787) Robin 2025-04-18 00:01:07 +08:00
• 05fcd1b430 [V1][Perf] Faster incremental detokenization (#15137) Nick Hill 2025-04-17 07:45:24 -07:00
• 7c02d6a137 [Doc] Changed explanation of generation_tokens_total and prompt_tokens_total counter type metrics to avoid confusion (#16784) Insu Kim 2025-04-17 23:10:08 +09:00
• 11c3b98491 [Doc] Document Matryoshka Representation Learning support (#16770) wang.yuqi 2025-04-17 21:37:37 +08:00
• dbe7f07001 [Doc] Make sure to update vLLM when installing latest code (#16781) Cyrus Leung 2025-04-17 20:53:31 +08:00
• c69bf4ee06 fix: hyperlink (#16778) Reidandreidliu41 2025-04-17 19:34:20 +08:00
• d27ea94034 Improve configs - TokenizerPoolConfig + DeviceConfig (#16603) Harry Mellor 2025-04-17 12:19:42 +01:00
• 99ed526101 [Misc] refactor examples series - lmcache (#16758) Reidandreidliu41 2025-04-17 19:02:35 +08:00
• 207da28186 [Doc] Fix a 404 link in installation/cpu.md (#16773) Michael Yao 2025-04-17 18:46:21 +08:00
• 5b1aca2ae3 [Bugfix] Fix GLM4 model (#16618) intervitens 2025-04-17 13:35:07 +03:00
• d8e557b5e5 [doc] add open-webui example (#16747) Reidandreidliu41 2025-04-17 18:27:32 +08:00
• 61a44a0b22 [Doc] Add more tips to avoid OOM (#16765) Cyrus Leung 2025-04-17 17:54:34 +08:00
• a6481525b8 [misc] ignore marlin_moe_wna16 local gen codes (#16760) DefTruth 2025-04-17 17:15:14 +08:00
• 8cac35ba43 [Ray] Improve documentation on batch inference (#16609) Richard Liaw 2025-04-16 22:19:26 -07:00
• 9dbf7a2dc1 [V1] Remove log noise when idle (#16735) Russell Bryant 2025-04-17 00:34:08 -04:00
• 607029e515 [Bugfix] Revert max_prompt_len validation for decoder-only models. (#16741) David Heineman 2025-04-16 21:33:15 -07:00
• cb072ce93b [Bugfix] Update Florence-2 tokenizer to make grounding tasks work (#16734) Isotr0py 2025-04-17 12:17:39 +08:00
• 95aca283b4 [rocm][V0] fix selection logic for custom PA in V0 (#16426) Divakar Verma 2025-04-16 21:52:11 -05:00
• 2b05b8ce69 [V1][Frontend] Improve Shutdown And Logs (#11737) 7 people 2025-04-16 22:48:34 -04:00
• 3c776dcefb Adding vllm buildkite job for IBM Power (#16679) Aaruni Aggarwal 2025-04-17 08:17:47 +05:30
• 2cbd4d2999 [V1][Spec Dec Bug Fix] Respect Spec Dec Method Specification (#16636) Bryan Lu 2025-04-16 19:47:26 -07:00
• 3092375e27 [V1][Performance] Implement custom serializaton for MultiModalKwargs [Rebased] (#16432) 3 people 2025-04-17 04:28:32 +02:00
• 3cd91dc955 Help user create custom model for Transformers backend remote code models (#16719) Harry Mellor 2025-04-17 02:05:59 +01:00
• 8a7368e069 [Misc] Remove redundant comment (#16703) Jade Zheng 2025-04-17 08:44:52 +08:00
• 93e561ec4d Improve error for structured output backend selection (#16717) Harry Mellor 2025-04-17 01:35:35 +01:00
• e1b004839a [Hardware] Add processor inputs to platform validation (#16680) Joe Runde 2025-04-16 18:28:42 +02:00