Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 39be30351f Correct capitalisation: Github -> GitHub (#14561) Harry Mellor 2025-03-10 16:53:33 +01:00
• 001a9c7b0d [Doc] Update PaliGemma note to a warning (#14565) Cyrus Leung 2025-03-10 23:02:28 +08:00
• 89cdaa83e7 [Kernel] Add more dtype support for GGUF kernels (#14043) Szymon Ożóg 2025-03-10 15:30:04 +01:00
• b0746fae3d [Frontend] support image embeds (#13955) Chauncey 2025-03-10 20:36:03 +08:00
• 60a98b2de5 [Docs] Mention model_impl arg when explaining Transformers fallback (#14552) Harry Mellor 2025-03-10 13:13:10 +01:00
• 460f553a6d [Misc] Add log information for handle_process_request. (#14130) Chauncey 2025-03-10 16:40:50 +08:00
• 1253b15774 [Feature] Consolidate performance benchmark datasets (#14036) 3 people 2025-03-10 00:23:11 -07:00
• dc74613fa2 [Bugfix] Wrong requirements path - rocm (#14527) Martin Hoyer 2025-03-10 03:49:46 +01:00
• a21076ed3a [Misc] Ensure out-of-tree quantization method recognize by cli args (#14328) Yanyi Liu 2025-03-09 20:13:31 +08:00
• 212007b168 [Hardware][TPU] Fix the recompiling issue in logits processor after warmup (#14510) Chengji Yao 2025-03-09 01:44:39 -08:00
• fb16eea48b [Bugfix] Revert QKVCrossParallelLinear usage in Mllama to keep BNB quantization work (#14498) Isotr0py 2025-03-09 12:47:45 +08:00
• 73ae0b44e9 [Bugfix] Fix tqdm progress bar when SamplingParams.n > 1 (#12428) Yuchen Yan 2025-03-09 12:14:53 +08:00
• 6d7f037748 [Feat] Support chunked prefill for LMCache connector (#14505) Jiayi Yao 2025-03-08 21:30:06 -06:00
• 10f7552789 [V1][TPU] Remove unnecessary padding for running on TPU. (#14467) iefgnoix 2025-03-08 18:56:04 -08:00
• b0d541947a [Attention] Default to FlashMLA backend for MLA (#14451) Lucas WilkinsonandTyler Michael Smith 2025-03-08 21:18:39 -05:00
• 5f0b53c6ea Revert "[V1][Core] Fix memory issue with logits & sampling" (#14504) Robert ShawandRoger Wang 2025-03-08 20:43:37 -05:00
• eb8b5eb183 [V1] Support bad_words in sampler (#13376) 22quinnandNick Hill 2025-03-08 14:50:26 -08:00
• 9513290032 [Misc] Upgrade to Python 3.9 typing for additional directories (#14492) Cyrus Leung 2025-03-09 01:35:50 +08:00
• 0d5e73d30e Update CODEOWNERS for structured output (#14496) Russell Bryant 2025-03-08 12:19:51 -05:00
• 609ef61fea [Bugfix] Fix profiling OOM and decouple encoder multimodal profiling (#14361) Isotr0py 2025-03-09 00:52:34 +08:00
• db84f5eb3b [Bugfix] DeepSeek Accuracy (#14476) Lucas Wilkinson 2025-03-08 11:47:03 -05:00
• 206e2577fa Move requirements into their own directory (#12547) Harry Mellor 2025-03-08 17:44:35 +01:00
• e02883c400 [Misc] Don't run ruff at all on 3rd party libs (#14493) Cyrus Leung 2025-03-08 23:16:40 +08:00
• 9085aabd62 [benchmarks] Add option to use unique jsonschema for each request (#14457) Russell Bryant 2025-03-08 09:36:39 -05:00
• 8d5aa466fb [V1][Core] Fix memory issue with logits & sampling (#13776) Roger Wang 2025-03-08 06:11:04 -08:00
• 0b7f06b447 [Misc] add use_tqdm_on_load to reduce logs (#14407) Aaron Pham 2025-03-08 08:57:46 -05:00
• 03fe18ae0f [VLM] Add TP support for Phi-4-MM (#14453) Isotr0py 2025-03-08 21:57:14 +08:00
• cb8bdfade2 [V1] TPU - Add tensor parallel support via Ray (#13618) Alexander Matveev 2025-03-08 08:19:38 -05:00
• 33f227e16b [CI/Build] Use a fixed seed to avoid flaky tests (#14480) Cyrus Leung 2025-03-08 19:30:09 +08:00
• cfd0ae8234 Add RLHF document (#14482) Harry Mellor 2025-03-08 10:51:39 +01:00
• 7caff01a7b [Build/BugFix] Fix hopper 12.8 build (#14354) Lucas WilkinsonandTyler Michael Smith 2025-03-08 03:11:56 -05:00
• be0b399d74 Add training doc signposting to TRL (#14439) Harry Mellor 2025-03-08 08:35:07 +01:00
• b8b0ccbd2d [Bugfix] Make the deviceprofiler include LoRA memory. (#14469) Jee Jee Li 2025-03-08 15:12:22 +08:00
• c908a07f57 [Doc] Added QwQ-32B to the supported models list in the reasoning out… (#14479) Robin 2025-03-08 15:07:32 +08:00
• 7b6fd6e486 [Doc]add doc for Qwen models tool calling (#14478) Robin 2025-03-08 14:58:46 +08:00
• 47512b3200 Default to generation_config from model (#12622) Harry Mellor 2025-03-08 07:46:15 +01:00
• 3b9c6c6947 [CI/Build] refactor: set timezone of container to UTC (#12888) Roger Meier 2025-03-08 07:42:01 +01:00
• 4aae667668 [core] add extra_args to SamplingParams (#13300) Aviv Keshet 2025-03-07 22:41:18 -08:00
• 9f3bc0f58c [MISC][V1] Register process killing handler only in the main thread (#14380) Cody Yu 2025-03-07 22:40:06 -08:00
• 980385f8c1 [Bugfix][Disaggregated] Add a check in send_kv_caches_and_hidden_states and fix the reshape of the KVCache (#14369) Mathis Felardos 2025-03-08 07:39:31 +01:00
• ca7a2d5f28 Revert "[Perf] Reduce MLA CPU overheads in V1 (#14384)" (#14471) Tyler Michael Smith 2025-03-08 01:18:53 -05:00
• 333681408f [Bugfix][V1] Handle MLA in kv_cache_interface (#14462) Tyler Michael Smith 2025-03-08 01:18:25 -05:00
• ef64044079 [V1] Prompt logprobs + APC compatibility; prompt logprobs reqs cannot fill APC (#13949) afeldman-nm 2025-03-07 20:48:12 -05:00
• 66e16a038e [Bugfix] Fix torch_xla which can't handle None seed introduced in #14274 (#14459) yarongmu-google 2025-03-07 15:17:04 -08:00
• e1f0835ae0 [V1][Metrics] Fix traceback with preemptions+LoRA (#14220) Mark McLoughlin 2025-03-07 20:36:16 +00:00
• 8ed5421aaa [V1] Eagerly remove finished requests from the batch (#14388) Nick Hill 2025-03-07 10:56:00 -08:00
• c6359e8ca6 [v1] torch.compile integration explanation (#14437) youkaichao 2025-03-08 01:55:50 +08:00
• 952a074980 [Misc] Add Phi4-MM example (#14343) Jee Jee Li 2025-03-08 01:28:52 +08:00
• d0feea31c7 [Kernel] optimize performance of gptq marlin kernel when n is small (#14138) Jinzhen Lin 2025-03-08 00:53:38 +08:00
• 58abe35455 [Benchmarks] Make detokenization optional in benchmark scripts (#11697) Jeremy Arnold 2025-03-07 10:09:00 -06:00
• f7ebad2307 [Doc] Update prefix_caching.md to match the example image (#14420) York-RDWang 2025-03-07 23:29:00 +08:00
• 80e9afb5bc [V1][Core] Support for Structured Outputs (#12388) 4 people 2025-03-07 10:19:11 -05:00
• 1e3598edeb Use the optimized block sizes after tuning the kernel. (#14329) iefgnoix 2025-03-07 05:25:13 -08:00
• f7a6bd0fa1 Fix missing kv_caches and attn_metadata in OpenVINOCausalLM (#14271) Harry Mellor 2025-03-07 13:30:42 +01:00
• 0ca3b8e01c [BUGFIX] Skip tokenization support for throughput benchmark (#12712) 3 people 2025-03-07 02:51:47 -08:00
• cc10281498 [Misc] Set default value of seed to None (#14274) மனோஜ்குமார் பழனிச்சாமி 2025-03-07 16:10:01 +05:30
• 05fb6718f0 [Bugfix] Clean up multi-modal processors (#14417) Cyrus Leung 2025-03-07 18:33:38 +08:00
• 12c29a881f [Bugfix] Further clean up LoRA test (#14422) Jee Jee Li 2025-03-07 18:30:55 +08:00
• 70da0c0748 correct wrong markdown syntax (#14414) Peng Li 2025-03-07 16:01:18 +08:00
• c1588a2c94 [GH] Auto-apply multi-modality label to relevant PRs (#14402) Cyrus Leung 2025-03-07 15:26:32 +08:00
• 8ca7a71df7 OpenVINO: added CPU-like conditions (#14338) Ilya Lavrenov 2025-03-07 10:24:49 +04:00
• 63137cd922 [Build] Add nightly wheel fallback when latest commit wheel unavailable (#14358) Isotr0py 2025-03-07 14:10:57 +08:00
• ddd1ef66ec [Bugfix] Fix JambaForCausalLM LoRA (#14370) Jee Jee Li 2025-03-07 14:05:47 +08:00
• e5e03c2c1b [BugFix] Illegal Memory Access in the blockwise cutlass fp8 GEMMs (#14396) Lucas Wilkinson 2025-03-07 00:56:06 -05:00
• e1744502c2 [FP8] Refactor apply_fp8_linear and apply_fp8_linear_generic into an object (#14390) Luka Govedič 2025-03-07 00:20:16 -05:00
• dae6896977 [Perf] Reduce MLA CPU overheads in V1 (#14384) Lucas Wilkinson 2025-03-06 22:59:14 -05:00
• c34eeec58d [Bugfix] Correctly call cudaProfilerStop in benchmarks script (#14183) Brayden Zhong 2025-03-06 19:42:49 -05:00
• ad60bbb2b2 [Doc] Fix a typo (#14385) Daniel Li 2025-03-06 16:31:52 -08:00
• 0578e5a462 [Hardware][TPU]Enable ragged paged attention kernel and resolve recompilation issue (#14310) Chengji Yao 2025-03-06 15:31:05 -08:00
• 04222984f8 [Docs] Add nsight guide to profiling docs (#14298) Michael Goin 2025-03-06 17:19:58 -05:00
• 6832707e90 [V1][Bugfix] Standardize quantized kv cache rejection for attention backends (#14221) Michael Goin 2025-03-06 17:18:29 -05:00
• 6b2ef5cd17 [Bug] Fix Attention when ignored in by quant_method (#14313) Michael Goin 2025-03-06 17:18:06 -05:00
• 958adce478 [Bugfix] Fix use_direct_call condition in FusedMoE layer for (#14382) Tyler Michael Smith 2025-03-06 17:17:21 -05:00
• 99b0915d3b [Kernel] Add needs_fixed_stride_order tag to most GEMMs (#14306) Tyler Michael Smith 2025-03-06 17:17:09 -05:00
• 8ca2b21c98 [CI] Disable spawn when running V1 Test (#14345) Thomas Parnell 2025-03-06 22:52:46 +01:00
• d9292786e1 [CI/Build] Use uv python for docker rather than ppa:deadsnakes/ppa (#13569) Michael Goin 2025-03-06 16:08:36 -05:00
• cc2f9b32c8 [Distributed] Add enable_expert_parallel arg (#14305) Tyler Michael Smith 2025-03-06 13:54:45 -05:00
• cd579352bf [V1] Do not detokenize if sampling param detokenize is False (#14224) Himanshu JajuandNick Hill 2025-03-06 19:40:24 +01:00
• 9f1710f1ac Fix mla prefill context performance (#13897) Ying Zhong 2025-03-07 01:35:49 +08:00
• e642ec962c Add authors to license header. (#14371) 3 people 2025-03-06 17:43:09 +01:00
• ada19210a3 Adding cpu inference with VXE ISA for s390x architecture (#12613) Dilip Gowda BhagavanandRishika Kedia 2025-03-06 22:10:53 +05:30
• bf0560bda9 Reinstate best_of for V0 (#14356) Harry Mellor 2025-03-06 17:34:22 +01:00
• 151b08e0fe [RLHF] use worker_extension_cls for compatibility with V0 and V1 (#14185) youkaichao 2025-03-07 00:32:46 +08:00
• 81b2f4a45f [Doc] Fix date typo in README.md (#14366) Jitse Klomp 2025-03-06 17:29:57 +01:00
• 82551ad616 [Core] Don't use cache during multi-modal profiling (#14336) Cyrus Leung 2025-03-07 00:03:31 +08:00
• caac5c2e59 [Bugfix][Core] fix abort_seq_group and memory leak when n>1 (#14326) courage17340 2025-03-06 23:59:32 +08:00
• 6bd1dd9d26 [Kernel] [V1] Improved performance for V1 Triton (ROCm) backend (#14152) Thomas Parnell 2025-03-06 16:39:16 +01:00
• 4f27044aab [Doc] Correct beam_search using in generative_models.md (#14363) Irina Yuryeva 2025-03-06 18:37:10 +03:00
• 0ddc991f5c [Doc] Update reasoning with stream example to use OpenAI library (#14077) Yanyi Liu 2025-03-06 21:20:37 +08:00
• fa82b93853 [Frontend][Docs] Transcription API streaming (#13301) Nicolò Lucchesi 2025-03-06 11:39:35 +01:00
• 69ff99fdcd [Core] Optimizing cross-attention QKVParallelLinear computation (#12325) Nicolò LucchesiandNickLucche 2025-03-06 10:37:26 +01:00
• 5d802522a7 [V1][VLM][Pixtral-HF] Support Pixtral-HF on V1 (#14275) lkchen 2025-03-06 00:58:41 -08:00
• 1769928079 [Model] Update Paligemma multimodal processing with PromptUpdate (#14015) kYLeandCyrus Leung 2025-03-06 02:31:38 -06:00
• ed6ea06577 [Hardware] Update the flash attn tag to support Blackwell (#14244) Pavani Majety 2025-03-05 22:01:37 -08:00
• 5ee10e990d [Bugfix][CI] ALiBi test case in xformers multi_query_kv_attention (#11301) Nicolò Lucchesi 2025-03-06 05:00:53 +01:00
• 3dbd2d813a [V1] LoRA - Enable more V1 tests (#14315) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-03-05 22:55:42 -05:00
• f5f7f00cd9 [Bugfix][Structured Output] Support outlines engine with reasoning outputs for DeepSeek R1 (#14114) Ce Gao 2025-03-06 11:49:20 +08:00
• abcc61e0af [misc] Mention ray list nodes command to troubleshoot ray issues (#14318) Rui Qiao 2025-03-05 18:00:36 -08:00
• f6bb18fd9a [BugFix] MLA + V1, illegal memory access and accuracy issues (#14253) Lucas Wilkinson 2025-03-05 20:10:13 -05:00
• 71eaf8969b [Build] Add UV_HTTP_TIMEOUT to avoid timeout during installation (#13850) Yuan Tang 2025-03-05 20:09:29 -05:00