Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 61de3ef74b [Model] Remove image mm limit for LLaMa4 (#16365) Ye (Charlotte) Qi 2025-04-10 02:36:27 -07:00
• ec1f9c8c91 Update Numba to 0.61.2 (#16376) cyyever 2025-04-10 15:59:37 +08:00
• 65e09094c4 [doc] add download model tips (#16389) Reidandreidliu41 2025-04-10 15:45:26 +08:00
• c70cf0fe06 [Kernel] Use moe_wna16 kernel for compressed tensors wna16 moe models (#16038) Michael Goin 2025-04-10 01:08:47 -06:00
• a5d11a54dc [Bugfix] Fix validation error for text-only Mllama 3.2 (#16377) Cyrus Leung 2025-04-10 14:19:42 +08:00
• 3d4c87758e [Misc] Update transformers version limits of multi-modal tests (#16381) Cyrus Leung 2025-04-10 14:03:33 +08:00
• a9bd832fc5 [Model] use AutoWeightsLoader for deepseek_v2, internlm2 (#16383) Aaron Ang 2025-04-10 02:01:00 -04:00
• 417bcefbae fix sonnet dataset sample when prefix len is very small (#16379) Chenyaaang 2025-04-09 22:35:07 -07:00
• baada0e737 [Bugfix][TPU] Fix TPU validate_request (#16369) Michael Goin 2025-04-09 22:55:12 -06:00
• 82eb61dd4c [misc] use tqdm.auto where appropriate (#16290) Benjamin Kitor 2025-04-09 21:54:54 -07:00
• 0d4d06fe2f [CI][Bugfix] Pin triton version for CPU (#16384) Roger Wang 2025-04-09 21:35:00 -07:00
• 4aed0ca6a2 [bugfix] Avoid the time consumption caused by creating dummy videos. (#16371) Jintao 2025-04-10 12:30:05 +08:00
• 1621b25288 [TPU] Fix dummy loading OOM (#16372) Chengji Yao 2025-04-09 21:06:16 -07:00
• a564797151 [Model] use AutoWeightsLoader for granite, granitemoe, granitemoeshared, grok1, mixtral (#16325) Aaron Ang 2025-04-09 23:07:40 -04:00
• 1da6a09274 [Bugfix]: do not shutdown server if skip_special_use=False for MistralTokenizer (#14094) Guillaume Calmettes 2025-04-10 04:43:09 +02:00
• 1e44ffc3ff Add GLM-4-0414 support (#16338) 9 people 2025-04-10 09:19:42 +08:00
• a454748544 [TPU][V1] Refine tpu_model_runner to mitigate future recompilation issues (#16275) Chengji Yao 2025-04-09 17:51:51 -07:00
• 1bff42c4b7 [Misc] refactor Structured Outputs example (#16322) Reidandreidliu41 2025-04-10 07:32:42 +08:00
• cb391d85dc [Hardware] add platform-specific request validation api (#16291) Joe Runde 2025-04-09 21:50:01 +02:00
• fee5b8d37f [Build/CI] Add tracing deps to vllm container image (#15224) Russell Bryant 2025-04-09 15:14:06 -04:00
• b2ce859bd2 Fix benchmark_throughput.py --backend=hf (#16352) Michael Goin 2025-04-09 13:09:28 -06:00
• 566f10a929 [CI]Fix hpu docker and numpy version for CI (#16355) Chendi.Xue 2025-04-09 12:52:26 -05:00
• c3b5189137 [Bugfix] catch AssertionError in MistralTokenizer as ValueError (#16344) Guillaume Calmettes 2025-04-09 19:33:24 +02:00
• a25866ac8d [Bugfix] Fix profiling.py (#16202) zh Wang 2025-04-10 01:03:34 +08:00
• 098900d7c2 Revert "Update label-tpu mergify and remove removal bot" (#16350) Michael Goin 2025-04-09 08:59:36 -06:00
• 98d01d3ce2 [Bugfix][Frontend] respect provided default guided decoding backend (#15476) Guillaume Calmettes 2025-04-09 14:11:10 +02:00
• d55244df31 [Model] Add SupportsMultiModal.get_language_model interface (#16007) Nicolò Lucchesi 2025-04-09 13:12:54 +02:00
• 04149cce27 [BugFix] fix some typos found by typos. (#16314) yihong 2025-04-09 18:43:59 +08:00
• 24834f4894 update neuron config (#16289) ajayvohra2005 2025-04-09 06:43:22 -04:00
• ec7da6fcf3 [BugFix] llama4 qknorm should be not shared across head (#16311) Lucia Fang 2025-04-09 00:59:14 -07:00
• 819d548e8a [BugFix] logger is not callable (#16312) yihong 2025-04-09 15:59:02 +08:00
• 477d2a8aa2 Update label-tpu mergify and remove removal bot (#16298) Michael Goin 2025-04-09 01:56:25 -06:00
• e484e02857 [Bugfix] Avoid transferring cached multi-modal items from P0 to P1 (#16273) Cyrus Leung 2025-04-09 15:51:27 +08:00
• 24f6b9a713 [Misc] Fix test_sharded_state_loader.py(#16004) (#16005) Accelerator1996 2025-04-09 14:47:30 +08:00
• 9cdde47289 [BugFix] Fix fusion test and add them to CI (#16287) Luka Govedič 2025-04-09 02:46:45 -04:00
• b1eb4ca152 [TPU] Update PyTorch/XLA (#16288) Chengji Yao 2025-04-08 23:46:32 -07:00
• 87b4ac56c2 [CI][Bugfix] Fix bad tolerance for test_batch_base64_embedding (#16221) Michael Goin 2025-04-08 22:14:46 -06:00
• cb84e45ac7 [Core] Upgrade to xgrammar 0.1.18, add cache size limit (#16283) Russell Bryant 2025-04-08 22:13:22 -04:00
• 4716377fbc [Feature] Estimate max-model-len use available KV cache memory (#16168) rongfu.leng 2025-04-09 10:12:51 +08:00
• 4e9cf8c1dd [Bugfix] fix gettid method is not define (#16084) rongfu.leng 2025-04-09 10:12:44 +08:00
• 2976dc27e9 [Bug] [ROCm] Fix Llama 4 Enablement Bug on ROCm: V0 ROCmFlashAttentionImpl and Triton Fused MoE bugs (#16198) 3 people 2025-04-09 10:12:34 +08:00
• 102bf967f0 [Model] Add smolvlm support (#16017) Chauncey 2025-04-09 10:12:17 +08:00
• 1f4b09b525 Add support to modelopt quantization of Mixtral model (#15961) yueshen2016 2025-04-08 18:53:31 -07:00
• 86c3369eb8 [CI/Build] Fix CI LoRA failure (#16270) Jee Jee Li 2025-04-09 09:13:56 +08:00
• 2755c34a8f [V1] Update structured output offline inference example (#15721) Russell Bryant 2025-04-08 18:34:09 -04:00
• db10422184 [Bugfix] fix deepseek fp16 scale bug (#14809) Jinzhen Linandmgoin 2025-04-09 04:56:09 +08:00
• e1a2c699dd [BugFix] Fix Llama4 - Index Error When Single Request Near Max Context (#16209) Lucas Wilkinson 2025-04-08 14:56:51 -04:00
• 0115ccd5c0 Add warning that content below line in template will be removed (#16276) Harry Mellor 2025-04-08 19:18:40 +01:00
• 40b4284fe3 [Bugfix] Handle process_weights_after_loading for QKVCrossParallelLinear (#15328) Isotr0py 2025-04-09 01:02:23 +08:00
• 4ebc0b9640 [Bugfix] Proper input validation for multi-modal encoder-decoder models (#16156) Cyrus Leung 2025-04-09 00:45:21 +08:00
• dc96fd54c6 [Misc] Avoid stripping meaningful whitespace from nvidia-smi topo -m output in collect_env.py (#16272) Kero Liang 2025-04-09 00:08:09 +08:00
• 1f5d13ab9f [New Model]: jinaai/jina-embeddings-v3 (#16120) wang.yuqi 2025-04-08 23:39:12 +08:00
• 90cb44eb02 Update to transformers==4.51.1 (#16257) Harry Mellor 2025-04-08 14:53:39 +01:00
• e11880deea [Bugfix] Remove triton do_bench fast_flush arg (#16256) Kebe 2025-04-08 21:51:06 +08:00
• 9351f91be9 [BugFix][ROCm] Fix GGUF MoE Dispatch Block_Dim for ROCm (#16247) TY-AMD 2025-04-08 20:10:26 +08:00
• 5a1e1c8353 [Model] use AutoWeightsLoader for phimoe,qwen2_moe,qwen3_moe (#16203) rongfu.leng 2025-04-08 19:05:47 +08:00
• 69ecaa7c79 [Misc] Add warning for multimodal data in LLM.beam_search (#16241) Alex Brooks 2025-04-08 05:05:27 -06:00
• 7f00899ff7 [Misc] format and refactor some examples (#16252) Reidandreidliu41 2025-04-08 18:42:32 +08:00
• 995e3d1f41 [Docs] Add Slides from Singapore Meetup (#16213) Simon Mo 2025-04-08 00:20:22 -07:00
• b4ac449a83 [Misc] Merge the logs of pp layers partitions (#16225) Kebe 2025-04-08 15:18:15 +08:00
• 8e5314a468 [V1] Add disable_chunked_mm_input arg to disable partial mm input prefill (#15837) Michael Goin 2025-04-08 00:24:07 -06:00
• 87918e40c4 [torch.compile][TPU] Make @support_torch_compile work for XLA backend (#15782) Siyuan Liuandmgoin 2025-04-07 23:23:53 -07:00
• f6b32efb7f [Bugfix] Fix and reorganize broken GGUF tests and bump gguf version (#16194) Isotr0py 2025-04-08 13:38:13 +08:00
• b99733d092 [Bugfix] Do not skip "empty" parts of chats that are parsable (#16219) Michael Goin 2025-04-07 23:14:15 -06:00
• 05a015d6a5 Add warning for Attention backends that do not support irope yet (#16212) Yong Hoon Shin 2025-04-07 20:59:26 -07:00
• ad971af8c7 [Bugfix] fix use-ep bug to enable ep by dp/tp size > 1 (#16161) zxfan-cpu 2025-04-08 11:48:47 +08:00
• f2ebb6f541 [V1] Scatter and gather placeholders in the model runner (#16076) 4 people 2025-04-07 19:43:41 -07:00
• 1d01211264 Update BASE_IMAGE to 2.22 release of Neuron (#16218) Satyajith Chilappagari 2025-04-07 19:11:18 -07:00
• f94ab12f79 [Misc] Update compressed-tensors to version 0.9.3 (#16196) Miles Williams 2025-04-08 03:09:06 +01:00
• a865bc1ca6 [core] do not send error across process (#16174) youkaichao 2025-04-08 10:09:03 +08:00
• 21802c4b6d [ROCm][Bugfix][FP8] Make fp8 quant respect fused modules mapping (#16031) Michael Goin 2025-04-07 19:28:14 -06:00
• 652907b354 Torchao (#14231) Driss Guessous 2025-04-07 16:39:28 -07:00
• 24f1c01e0f [Bugfix][V0] XGrammar structured output supports Enum (#15878) leon-seidel 2025-04-08 00:38:25 +02:00
• fad6e2538e [Misc] add description attribute in CLI (#15921) Reidandreidliu41 2025-04-08 06:30:35 +08:00
• 7f6d47c1a2 [V1][BugFix] Exit properly if engine core fails during startup (#16137) Nick Hill 2025-04-07 15:30:15 -07:00
• 3147586ebd [Bugfix] Fix guidance backend for Qwen models (#16210) Benjamin Chislett 2025-04-07 18:15:43 -04:00
• ed636d99ca [Misc] Move Llama 4 projector call into encoder execution (#16201) Roger Wang 2025-04-07 14:02:05 -07:00
• 090c856d76 [Misc] Human-readable max-model-len cli arg (#16181) Nicolò LucchesiandCyrus Leung 2025-04-07 20:40:58 +02:00
• ad434d4cfe Print the warning only once (#16193) Gregory Shtrasberg 2025-04-07 14:30:06 -04:00
• 66d433b94f [V1] Revert the default max_num_seqs to V0 values for most hardware (#16158) Cyrus Leung 2025-04-08 01:54:36 +08:00
• 027b204ff1 [Bugfix] Re-enable support for ChatGLMForConditionalGeneration (#16187) Cyrus Leung 2025-04-07 23:15:58 +08:00
• 55dcce91df Upstream Llama4 Support to Main (#16113) 4 people 2025-04-07 08:06:27 -07:00
• 8017c8db7f [Doc]Update image to latest version (#16186) Robin 2025-04-07 22:17:39 +08:00
• dc3529dbf6 [Misc] improve example mlpspeculator and llm_engine_example (#16175) Reidandreidliu41 2025-04-07 19:53:52 +08:00
• 7699258ef0 [Model] Add Qwen3 and Qwen3MoE (#15289) YamPengLiandCyrus Leung 2025-04-07 19:06:41 +08:00
• e9ba99f296 [V1][Structured Output] Add supports_structured_output() method to Platform (#16148) Shanshan Shen 2025-04-07 19:06:24 +08:00
• 7c80368710 [VLM] Florence-2 supports online serving (#16164) Isotr0py 2025-04-07 19:04:02 +08:00
• 95d63f38c0 doc: fix some typos in doc (#16154) yihong 2025-04-07 13:32:06 +08:00
• bb8dab821e [CI] Set max transformers version for Ultravox model test (#16149) Roger Wang 2025-04-06 21:37:58 -07:00
• fc0f87768a [Bugfix] Make dummy encoder prompt padding alternative and add missing warnings (#16129) Isotr0py 2025-04-07 12:07:15 +08:00
• 0a57386721 [Misc] Update Mistral-3.1 example (#16147) Cyrus Leung 2025-04-07 11:57:37 +08:00
• 3749e28774 [V1][Minor] Minor simplification for get_computed_blocks (#16139) Woosuk Kwon 2025-04-06 20:38:12 -07:00
• 86fc2321ff [Metrics] Add bucket for request_latency, time_to_first_token and time_per_output_token (#15202) Kay Yan 2025-04-07 11:34:51 +08:00
• 2549c0dfef Fix requires-python (#16132) Martin Hoyer 2025-04-07 04:22:25 +02:00
• b10e519895 [V1][Minor] Optimize get_cached_block (#16135) Woosuk Kwon 2025-04-06 13:48:14 -07:00
• 9bde5ba127 [TPU] Update PyTorch/XLA (#16130) Chengji Yao 2025-04-06 11:25:55 -07:00
• 72c8f1ad04 [Misc] update requires-python in pyproject.toml (#16116) Reidandreidliu41 2025-04-06 22:56:34 +08:00
• da224daaa9 [Bugfix] add hf_token to EngineArgs (#16093) paolovicandpaolovic 2025-04-06 16:47:33 +02:00
• 3a100b9278 [Bugfix] LoRA : Fix the order in which the kernels process LoRAs (#16040) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-04-06 10:04:50 -04:00
• 242a637aea [Model] use AutoWeightsLoader for stablelm,starcoder2,zamba2 (#16103) rongfu.leng 2025-04-06 20:52:01 +08:00