Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• ff7424f491 [Frontend] Support override generation config in args (#12409) Yanyi Liu 2025-01-29 17:41:01 +08:00
• d93bf4da85 [Model] Refactoring of MiniCPM-V and add MiniCPM-o-2.6 support for vLLM (#12069) +2939 people 2025-01-29 17:24:59 +08:00
• 036ca94c25 [Bugfix] handle alignment of arguments in convert_sparse_cross_attention_mask_to_dense (#12347) Travis JohnsonandWallas Santos 2025-01-29 01:54:35 -07:00
• ef001d98ef Fix the pydantic logging validator (#12420) Maximilien de Bayser 2025-01-29 04:53:13 -03:00
• 5f671cb4c3 [V1] Improve Error Message for Unsupported Config (#12535) Robert ShawandMichael Goin 2025-01-28 23:56:56 -05:00
• bd02164cf9 Bugfix for whisper quantization due to fake k_proj bias (#12524) Michael Goin 2025-01-28 23:49:03 -05:00
• 46fb056749 [V1][Metrics] Add TTFT and TPOT histograms (#12530) Mark McLoughlin 2025-01-29 04:11:16 +00:00
• dd6a3a02cb [Doc] Convert docs to use colon fences (#12471) Harry Mellor 2025-01-29 03:38:29 +00:00
• a7e3eba66f [Frontend] Support reasoning content for deepseek r1 (#12473) 4 people 2025-01-29 11:38:08 +08:00
• fbb5bd4cef [TPU] Add example for profiling TPU inference (#12531) Michael Goin 2025-01-28 22:16:47 -05:00
• 80fcc3ed1c [Kernel] Pipe attn_logits_soft_cap through paged attention TPU kernels (#12482) fenghuizhang 2025-01-28 14:36:44 -08:00
• c386c43ca3 [V1][Metrics] Add per-request prompt/generation_tokens histograms (#12516) Mark McLoughlin 2025-01-28 22:07:22 +00:00
• f26d790718 Do not run suggestion pre-commit hook multiple times (#12521) Harry Mellor 2025-01-28 20:05:27 +00:00
• 0f657bdc52 Replace missed warning_once for rerank API (#12472) Michael Goin 2025-01-28 14:06:32 -05:00
• 3fd1fb63ef [V1][Metrics] Hook up IterationStats for Prometheus metrics (#12478) Mark McLoughlin 2025-01-28 16:38:38 +00:00
• 925d2f1908 [Doc] Fix typo for x86 CPU installation (#12514) Jun Duan 2025-01-28 11:37:10 -05:00
• 8f58a51358 [VLM] Merged multi-modal processor and V1 support for Qwen-VL (#12504) Cyrus Leung 2025-01-29 00:25:05 +08:00
• 2079e43bee [Core] Make raw_request optional in ServingCompletion (#12503) Sebastian Schoennenbeck 2025-01-28 11:56:45 +01:00
• e29d4358ef [V1] Include Engine Version in Logs (#12496) Robert Shaw 2025-01-28 03:27:41 -05:00
• 8cbc424975 Update README.md with V1 alpha release (#12495) Roger Wang 2025-01-28 00:22:41 -08:00
• dd66fd2b01 [CI] fix pre-commit error (#12494) Mengqing Cao 2025-01-28 14:11:05 +08:00
• 0f465ab533 [FEATURE] Enables offline /score for embedding models (#12021) Gabriel Marinho 2025-01-28 00:30:13 -03:00
• 23a7cbc88b [CI/Build] Fixed the xla nightly issue report in #12451 (#12453) Hossein Sarshar 2025-01-27 22:18:07 -05:00
• 426a5c3625 Fix bad path in prometheus example (#12481) Michael Goin 2025-01-27 20:56:31 -05:00
• ddee88d0ff [Neuron][Kernel] NKI-based flash-attention kernel with paged KV cache (#11277) Liangfu ChenandJiangfei Duan 2025-01-27 17:31:16 -08:00
• 823ab79633 Update pre-commit hooks (#12475) Harry Mellor 2025-01-28 00:23:08 +00:00
• 6116ca8cd7 [Feature] [Spec decode]: Enable MLPSpeculator/Medusa and prompt_logprobs with ChunkedPrefill (#10132) Nicolò Lucchesiandwallashss 2025-01-27 22:38:35 +01:00
• 2bc3fbba0c [FlashInfer] Upgrade to 0.2.0 (#11194) Bowen Wangandyoukaichao 2025-01-28 02:19:24 +08:00
• 3f1fc7425a [V1][CI/Test] Do basic test for top-p & top-k sampling (#12469) Woosuk Kwon 2025-01-27 09:40:04 -08:00
• 01ba927040 [V1][Metrics] Add initial Prometheus logger (#12416) Mark McLoughlin 2025-01-27 17:26:28 +00:00
• 103bd17ac5 [Build] Only build 9.0a for scaled_mm and sparse kernels (#12339) Lucas Wilkinson 2025-01-27 10:40:00 -05:00
• ce69f7f754 [Bugfix] Fix gpt2 GGUF inference (#12467) Isotr0py 2025-01-27 18:31:49 +08:00
• 624a1e4711 [V1][Minor] Minor optimizations for update_from_output (#12454) Woosuk Kwon 2025-01-27 01:09:27 -08:00
• 372bf0890b [Bugfix] Fix missing seq_start_loc in xformers prefill metadata (#12464) Isotr0py 2025-01-27 15:25:30 +08:00
• 5204ff5c3f [Bugfix] Fix Granite 3.0 MoE model loading (#12446) Cyrus Leung 2025-01-27 13:26:44 +08:00
• 0cc6b383d7 [Frontend] Support scores endpoint in run_batch (#12430) Pooya Davoodi 2025-01-26 20:30:17 -08:00
• 28e0750847 [V1] Avoid list creation in input preparation (#12457) Woosuk Kwon 2025-01-26 19:57:56 -08:00
• 582cf78798 [DOC] Add link to vLLM blog (#12460) Yuan Tang 2025-01-26 21:46:19 -06:00
• 0034b09ceb [Frontend] Rerank API (Jina- and Cohere-compatible API) (#12376) Kyle Mistele 2025-01-26 20:58:45 -06:00
• 72bac73067 [Build/CI] Fix libcuda.so linkage (#12424) Tyler Michael Smith 2025-01-26 16:18:19 -05:00
• 68f11149d8 [Bugfix][Kernel] Fix perf regression caused by PR #12405 (#12434) Lucas Wilkinson 2025-01-26 14:09:34 -05:00
• 72f4880425 [Bugfix/CI] Fix broken kernels/test_mha.py (#12450) Tyler Michael Smith 2025-01-26 13:39:03 -05:00
• aa2cd2c43d [Bugfix] Disable w16a16 2of4 sparse CompressedTensors24 (#12417) Tyler Michael Smithandmgoin 2025-01-26 06:59:58 -05:00
• 9ddc35220b [Frontend] generation_config.json for maximum tokens(#12242) 9 people 2025-01-26 06:59:25 -05:00
• a5255270c3 [Misc] Revert FA on ViT #12355 and #12435 (#12445) Roger Wang 2025-01-26 03:56:34 -08:00
• 0ee349b553 [V1][Bugfix] Fix assertion when mm hashing is turned off (#12439) Roger Wang 2025-01-26 00:47:42 -08:00
• fa63e710c7 [V1][Perf] Reduce scheduling overhead in model runner after cuda sync (#12094) Keyun Tong 2025-01-26 00:42:37 -08:00
• 2a0309a646 [Misc][Bugfix] FA3 support to ViT MHA layer (#12435) Roger WangandIsotr0py 2025-01-25 21:00:31 -08:00
• 324960a95c [TPU][CI] Update torchxla version in requirement-tpu.txt (#12422) Siyuan Liu 2025-01-24 23:23:03 -08:00
• f1fc0510df [Misc] Add FA2 support to ViT MHA layer (#12355) Isotr0py 2025-01-25 15:07:35 +08:00
• bf21481dde [ROCm][MoE] MI300 tuned configs Mixtral-8x(7B,22B) | fp16, fp8 (#12408) Divakar Verma 2025-01-24 22:17:19 -06:00
• fb30ee92ee [Bugfix] Fix BLIP-2 processing (#12412) Cyrus Leung 2025-01-25 11:42:42 +08:00
• 221d388cc5 [Bugfix][Kernel] Fix moe align block issue for mixtral (#12413) ElizaWszola 2025-01-24 20:49:28 -05:00
• 3132a933b6 [Bugfix][Kernel] FA3 Fix - RuntimeError: This flash attention build only supports pack_gqa (for build size reasons). (#12405) Lucas Wilkinson 2025-01-24 15:20:59 -05:00
• df5dafaa5b [Misc] Remove deprecated code (#12383) Cyrus Leung 2025-01-25 03:45:20 +08:00
• ab5bbf5ae3 [Bugfix][Kernel] Fix CUDA 11.8 being broken by FA3 build (#12375) Lucas Wilkinson 2025-01-24 10:27:59 -05:00
• 3bb8e2c9a2 [Misc] Enable proxy support in benchmark script (#12356) Junichi Sato 2025-01-24 23:58:26 +09:00
• e784c6b998 [ci/build] sync default value for wheel size (#12398) youkaichao 2025-01-24 17:54:29 +08:00
• 9a0f3bdbe5 [Hardware][Gaudi][Doc] Add missing step in setup instructions (#12382) Mohit Deopujari 2025-01-24 01:43:49 -08:00
• c7c9851036 [ci/build] fix wheel size check (#12396) youkaichao 2025-01-24 17:31:25 +08:00
• 3c818bdb42 [Misc] Use VisionArena Dataset for VLM Benchmarking (#12389) Roger Wang 2025-01-24 00:22:04 -08:00
• 6dd94dbe94 [perf] fix perf regression from #12253 (#12380) youkaichao 2025-01-24 11:34:27 +08:00
• 0e74d797ce [V1] Increase default batch size for H100/H200 (#12369) Woosuk Kwon 2025-01-23 19:19:55 -08:00
• 55ef66edf4 Update compressed-tensors version (#12367) Dipika Sikka 2025-01-23 22:19:42 -05:00
• 5e5630a478 [Bugfix] Path join when building local path for S3 clone (#12353) omer-dayan 2025-01-24 05:06:07 +02:00
• d3d6bb13fb Set weights_only=True when using torch.load() (#12366) Russell Bryant 2025-01-23 21:17:30 -05:00
• 24b0205f58 [V1][Frontend] Coalesce bunched RequestOutputs (#12298) Nick HillandRobert Shaw 2025-01-23 17:17:41 -08:00
• c5cffcd0cd [Docs] Update spec decode + structured output in compat matrix (#12373) Russell Bryant 2025-01-23 20:15:52 -05:00
• 682b55bc07 [Docs] Add meetup slides (#12345) Woosuk Kwon 2025-01-23 14:10:03 -08:00
• 9726ad676d [Misc] Fix OpenAI API Compatibility Issues in Benchmark Script (#12357) Junichi Sato 2025-01-24 07:02:13 +09:00
• eb5cb5e528 [BugFix] Fix parameter names and process_after_weight_loading for W4A16 MoE Group Act Order (#11528) 3 people 2025-01-23 16:40:33 -05:00
• 2cbeedad09 [Docs] Document Phi-4 support (#12362) Isotr0py 2025-01-24 03:18:51 +08:00
• 2c85529bfc [TPU] Update TPU CI to use torchxla nightly on 20250122 (#12334) Siyuan Liu 2025-01-23 10:50:16 -08:00
• e97f802b2d [FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) Gregory ShtrasbergandMicah Williamson 2025-01-23 13:04:03 -05:00
• 6e650f56a1 [torch.compile] decouple compile sizes and cudagraph sizes (#12243) youkaichao 2025-01-24 02:01:30 +08:00
• 3f50c148fd [core] add wake_up doc and some sanity check (#12361) youkaichao 2025-01-24 02:00:50 +08:00
• 8c01b8022c [Bugfix] Fix broken internvl2 inference with v1 (#12360) Isotr0py 2025-01-24 01:20:33 +08:00
• 99d01a5e3d [V1] Simplify M-RoPE (#12352) Roger Wangandimkero 2025-01-23 07:13:23 -08:00
• d07efb31c5 [Doc] Troubleshooting errors during model inspection (#12351) Cyrus Leung 2025-01-23 22:46:58 +08:00
• 978b45f399 [Kernel] Flash Attention 3 Support (#12093) Lucas Wilkinson 2025-01-23 09:45:48 -05:00
• c5b4b11d7f [Bugfix] Fix k_proj's bias for whisper self attention (#12342) Isotr0py 2025-01-23 18:15:33 +08:00
• 8ae5ff2009 [Hardware][Gaudi][BugFix] Fix dataclass error due to triton package update (#12338) liuzhenwei 2025-01-23 16:35:46 +08:00
• 511627445e [doc] explain common errors around torch.compile (#12340) youkaichao 2025-01-23 14:56:02 +08:00
• f0ef37233e [V1] Add uncache_blocks (#12333) Cody Yu 2025-01-22 20:19:21 -08:00
• 7551a34032 [Docs] Document vulnerability disclosure process (#12326) Russell Bryant 2025-01-22 22:44:09 -05:00
• 01a55941f5 [Docs] Update FP8 KV Cache documentation (#12238) Michael GoinandCyrus Leung 2025-01-22 22:18:09 -05:00
• 8d7aa9de71 [Bugfix] Fixing AMD LoRA CI test. (#12329) Alexei-V-Ivanov-AMD 2025-01-22 20:53:02 -06:00
• 68c4421b6d [AMD][Quantization] Add TritonScaledMMLinearKernel since int8 is broken for AMD (#12282) rasmith 2025-01-22 18:10:37 -06:00
• aea94362c9 [Frontend][V1] Online serving performance improvements (#12287) Nick Hill 2025-01-22 14:22:12 -08:00
• 7206ce4ce1 [Core] Support reset_prefix_cache (#12284) Cody Yu 2025-01-22 10:52:27 -08:00
• 96f6a7596f [Bugfix] Fix HPU multiprocessing executor (#12167) Konrad Zawora 2025-01-22 19:07:07 +01:00
• 84bee4bd5c [Misc] Improve the readability of BNB error messages (#12320) Jee Jee Li 2025-01-23 00:56:54 +08:00
• fc66dee76d [Misc] Fix the error in the tip for the --lora-modules parameter (#12319) Robin 2025-01-23 00:48:41 +08:00
• 6609cdf019 [Doc] Add docs for prompt replacement (#12318) Cyrus Leung 2025-01-22 22:56:29 +08:00
• 16366ee8bb [Bugfix][VLM] Fix mixed-modality inference backward compatibility for V0 (#12313) Roger Wang 2025-01-22 05:06:36 -08:00
• 528dbcac7d [Model][Bugfix]: correct Aria model output (#12309) zhou fan 2025-01-22 19:39:19 +08:00
• cd7b6f0857 [VLM] Avoid unnecessary tokenization (#12310) Cyrus Leung 2025-01-22 19:08:31 +08:00
• 68ad4e3a8d [Core] Support fully transparent sleep mode (#11743) youkaichao 2025-01-22 14:39:32 +08:00
• 4004f144f3 [Build] update requirements of no-device (#12299) Mengqing Cao 2025-01-22 14:29:31 +08:00
• 66818e5b63 [core] separate builder init and builder prepare for each batch (#12253) youkaichao 2025-01-22 14:13:52 +08:00