Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 726efc6a32 [Quantization][V1] BitsAndBytes support V1 (#15611) Jee Jee Li 2025-03-28 10:12:47 +08:00
• bd45912b99 [TPU] Lazy Import (#15656) Robert Shaw 2025-03-27 21:57:01 -04:00
• 15dac210f0 [V1] AsyncLLM data parallel (#13923) Nick Hill 2025-03-27 16:14:41 -07:00
• 112b3e5b3b [CI] Update rules for applying tpu label. (#15634) Russell Bryant 2025-03-27 18:15:26 -04:00
• 32d669275b Correct PowerPC to modern IBM Power (#15635) cnorman 2025-03-27 17:04:32 -05:00
• 4098b72210 [Bugfix][TPU][V1] Fix recompilation (#15553) Nicolò Lucchesi 2025-03-27 20:15:06 +01:00
• 46450b8d33 Use absolute placement for Ask AI button (#15628) Harry Mellor 2025-03-27 18:52:18 +00:00
• 13ac9cab21 [Misc] Avoid direct access of global mm_registry in compute_encoder_budget (#15621) Cyrus Leung 2025-03-28 01:52:00 +08:00
• 66aa4c0bf4 [Feature] Add middleware to log API Server responses (#15593) Yuan Tang 2025-03-27 13:49:38 -04:00
• 247181536f [Misc] Replace is_encoder_decoder_inputs with split_enc_dec_inputs (#15620) Cyrus Leung 2025-03-28 01:36:32 +08:00
• 07bf813fb5 [Doc] Link to onboarding tasks (#15629) Cyrus Leung 2025-03-28 00:30:53 +08:00
• 8958217ad5 [Bugfix] Fix use_cascade_attention handling for Alibi-based models on vllm/v1 (#15211) Hiroaki SugiyamaandWoosuk Kwon 2025-03-27 23:29:29 +09:00
• ac5bc615b0 [Model] MiniCPM-V/O supports V1 (#15487) Cyrus Leung 2025-03-27 21:07:29 +08:00
• 8063dfc61a [Doc] update --system for transformers installation in docker doc (#15616) Reidandreidliu41 2025-03-27 20:38:46 +08:00
• 6278bc829e Fix incorrect filenames in vllm_compile_cache.py (#15494) Richard Zouandyoukaichao 2025-03-27 06:33:41 -04:00
• 3f532cb6a6 [Misc] Use model_redirect to redirect the model name to a local folder. (#14116) wang.yuqi 2025-03-27 17:21:23 +08:00
• e6c9053f9e [Misc] Clean up scatter_patch_features (#15559) Cyrus Leung 2025-03-27 15:45:00 +08:00
• 43ed4143c4 [Quantization] Fp8 Channelwise Dynamic Per Token GroupedGEMM (#15587) 4 people 2025-03-27 02:47:25 -04:00
• f4c98b4d4c [Misc] Consolidate LRUCache implementations (#15481) Bella kira 2025-03-27 14:43:43 +08:00
• e1e0fd7543 [TPU] Avoid Triton Import (#15589) Robert Shaw 2025-03-27 02:43:02 -04:00
• df8d3d1287 [Misc] Restrict ray version dependency and update PP feature warning in V1 (#15556) Rui Qiao 2025-03-26 23:21:07 -07:00
• 619d3de8bd [TPU] [V1] fix cases when max_num_reqs is set smaller than MIN_NUM_SEQS (#15583) Chengji Yao 2025-03-26 22:46:26 -07:00
• ecff8309a3 [ROCm] Env variable to trigger custom PA (#15557) Gregory Shtrasberg 2025-03-27 01:46:12 -04:00
• dcf2a590f5 Allow torchao quantization in SiglipMLP (#15575) Jerry Zhang 2025-03-26 22:45:51 -07:00
• 54aa619459 [V1] Refactor num_computed_tokens logic (#15307) Cody YuandWoosuk Kwon 2025-03-26 21:54:36 -07:00
• fb22be5817 [moe][quant] add weight name case for offset (#15515) Mengqing Cao 2025-03-27 12:50:29 +08:00
• 7f301dd8ef [Doc] Update V1 user guide for fp8 kv cache support (#15585) Wei Zeng 2025-03-26 19:39:03 -07:00
• 8095341a01 [misc] LoRA: Remove unused long context test data (#15558) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-03-26 19:04:51 -07:00
• 69db16a46a add platform check back (#15578) Chenyaaang 2025-03-26 18:50:27 -07:00
• ce78f9af4e Add automatic tpu label to mergify.yml (#15560) Michael Goin 2025-03-26 19:39:58 -06:00
• 9239bf718e [Kernel] CUTLASS grouped gemm fp8 MoE kernel (#13972) ElizaWszolaandLucas Wilkinson 2025-03-27 01:54:44 +01:00
• 7a6d45bc8a Support FIPS enabled machines with MD5 hashing (#15299) Matthew Vine 2025-03-26 20:19:46 -04:00
• e74ff409e0 [TPU] support disabling xla compilation cache (#15567) Chengji Yao 2025-03-26 17:09:28 -07:00
• 7a888271f5 Use Cache Hinting for fused_moe kernel (#15511) Wes 2025-03-26 17:21:34 -06:00
• 9d119a86ae [V1] TPU CI - Fix test_compilation.py (#15570) Alexander Matveev 2025-03-26 17:51:54 -04:00
• b2e85e26f4 [V1] TPU - Revert to exponential padding by default (#15565) Alexander Matveev 2025-03-26 17:35:05 -04:00
• dd8a29da99 Applying some fixes for K8s agents in CI (#15493) Alexei-V-Ivanov-AMD 2025-03-26 15:35:11 -05:00
• 27df5199d9 Support SHA256 as hash function in prefix caching (#15297) marko 2025-03-26 19:11:28 +01:00
• 35fad35a48 [V1][Sampler] Faster top-k only implementation (#15478) Nick Hill 2025-03-26 10:56:47 -07:00
• 733e7c9e95 [Refactor] Remove unnecessary backend parameter in structured output interface (#15317) Aaron Pham 2025-03-26 13:51:56 -04:00
• 0af4d764d6 Fix weight loading for some models in Transformers backend (#15544) Harry Mellor 2025-03-26 17:17:53 +00:00
• e64afa455c multi-node offline DP+EP example (#15484) youkaichao 2025-03-26 23:54:24 +08:00
• 1711b929b6 [Model] Add Reasoning Parser for Granite Models (#14202) Alex BrooksandJoe Runde 2025-03-26 08:28:07 -06:00
• c091c0a588 Improve validation of TP in Transformers backend (#15540) Harry Mellor 2025-03-26 14:26:48 +00:00
• 1aa162e030 Apply torchfix (#15532) cyyever 2025-03-26 20:09:06 +08:00
• cf5c8f1686 Separate base model from TransformersModel (#15467) 3 people 2025-03-26 10:13:38 +00:00
• 4ec2cee000 [Misc] improve example script output (#15528) Reidandreidliu41 2025-03-26 18:12:47 +08:00
• 99f536f830 [Misc] Enhance warning information to user-defined chat template (#15408) wwl2755 2025-03-26 04:21:15 -05:00
• 5ebf66748b [FEAT][ROCm] Integrate Fused MoE Kernels from AITER (#14967) vllmellmandtjtanaa 2025-03-26 16:30:30 +08:00
• 781d056280 [Feature] Enhance EAGLE Architecture with Proper RMS Norms (#14990) Bryan LuandCyrus Leung 2025-03-26 01:24:07 -07:00
• 5aefd6ac31 Fix raw_request extraction in load_aware_call decorator (#15382) daniel-salib 2025-03-25 22:29:54 -07:00
• 6c663dfd5e [misc] LoRA - Skip LoRA kernels when not required (#15152) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-03-25 20:33:45 -07:00
• 33437bc6e7 [BugFix] Fix nightly MLA failure (FA2 + MLA chunked prefill, i.e. V1, producing bad results) (#15492) Lucas Wilkinson 2025-03-25 23:33:22 -04:00
• 23114d3364 [Misc] Warn about v0 in benchmark_paged_attn.py (#15495) Tyler Michael Smith 2025-03-25 23:31:04 -04:00
• 997c8811d6 [Model] Support multi-image for Molmo (#15438) Cyrus Leung 2025-03-26 11:26:33 +08:00
• e42389f9d7 Transformers backend already supports V1 (#15463) Harry Mellor 2025-03-26 03:26:16 +00:00
• ff38f0a32c [CI/Build] LoRA: Delete long context tests (#15503) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-03-25 17:18:34 -07:00
• a5cfbab3c8 [Core] LoRA: V1 Scheduler optimization (#15422) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-03-25 15:50:09 -07:00
• ac3cd6e83c [core] add bucket padding to tpu_model_runner (#14995) Chenyaaangandrshaw@neuralmagic.com <robertgshaw2@gmail.com> 2025-03-25 14:27:22 -07:00
• 082ab86f5f [V1] Support long_prefill_token_threshold in v1 scheduler (#15419) Lu Fang 2025-03-25 14:22:26 -07:00
• 6aa196c8dc [V1][Minor] Use SchedulerInterface type for engine scheduler field (#15499) Nick Hill 2025-03-25 14:21:36 -07:00
• a0dd7dcd49 [TPU][V1] Fix Sampler recompilation (#15309) Nicolò Lucchesi 2025-03-25 21:43:54 +01:00
• e977c11111 Add workaround for shared field_names in pydantic model class (#13925) Maximilien de Bayser 2025-03-25 17:31:08 -03:00
• 5f063a80bd [bugfix] add supports_v1 platform interface (#15417) Joe Runde 2025-03-25 15:00:32 -04:00
• 5d8e1c9279 [Bugfix] Support triton==3.3.0+git95326d9f for RTX 5090 (Unsloth + vLLM compatibility) (#15471) Antonio GómezandServerAI 2025-03-25 18:59:25 +01:00
• 0a049c7d86 [CI/Build] Add tests for the V1 tpu_model_runner. (#14843) yarongmu-google 2025-03-25 09:27:16 -07:00
• d0cfec7ab9 [bugfix] fix inductor cache on max_position_embeddings (#15436) youkaichao 2025-03-25 22:05:39 +08:00
• a608160027 [Kernel] Fix conflicting macro names for gguf kernels (#15456) Szymon Ożóg 2025-03-25 14:50:49 +01:00
• 3f04a7fbf2 [Doc] Update V1 user guide for multi-modality (#15460) Cyrus Leung 2025-03-25 19:01:58 +08:00
• 5994430b84 [Misc] Remove redundant num_embeds (#15443) Cyrus Leung 2025-03-25 18:27:57 +08:00
• a9e879b316 [Misc] Clean up MiniCPM-V/O code (#15337) Cyrus Leung 2025-03-25 18:22:52 +08:00
• 3e2f37a69a Dockerfile.ppc64le changes to move to UBI (#15402) Md. Shafi Hussain 2025-03-25 15:45:14 +05:30
• 4f044b1d67 [Kernel][CPU] CPU MLA (#14744) Thien Tran 2025-03-25 17:34:59 +08:00
• 4157f563b4 [Hardware][TPU][Bugfix] Fix v1 mp profiler (#15409) Siyuan Liu 2025-03-25 01:43:00 -07:00
• 051da7efe3 Fix CUDA kernel index data type in vllm/csrc/quantization/gptq_marlin/awq_marlin_repack.cu +10 (#15160) Lu FangandRichard Barnes 2025-03-25 00:36:45 -07:00
• 25f560a62c [V1][Spec Decode] Update target_logits in place for rejection sampling (#15427) Woosuk Kwon 2025-03-24 21:04:41 -07:00
• a09ad90a72 [V1] guidance backend for structured output + auto fallback mode (#14779) 3 people 2025-03-25 00:02:33 -04:00
• 10b34e36b9 [Bugfix] Fixed the issue of not being able to input video and image simultaneously (#15387) Chauncey 2025-03-25 11:48:08 +08:00
• b5269db959 Revert "Fix non-contiguous input passed to Marlin kernel (#15319)" (#15398) Tyler Michael Smith 2025-03-24 23:43:51 -04:00
• 6db94571d7 [Misc] Remove LoRA log (#15388) Jee Jee Li 2025-03-25 11:43:48 +08:00
• 97cfa65df7 Add pipeline parallel support to TransformersModel (#12832) Harry MellorandIsotr0py 2025-03-25 02:41:45 +00:00
• 911c8eb000 [Minor][Spec Decode] Remove compiled_softmax (#15416) Woosuk Kwon 2025-03-24 19:09:04 -07:00
• ebcebeeb6b [V1][Spec Decode] Enable spec decode for top-p & top-k sampling (#15063) Woosuk Kwon 2025-03-24 17:16:46 -07:00
• f533b5837f [ROCm][Kernel] MoE weights padding (#14454) Gregory Shtrasbergandcharlifu 2025-03-24 19:45:30 -04:00
• 8279201ce6 [Build] Cython compilation support fix (#14296) Gregory Shtrasberg 2025-03-24 19:37:54 -04:00
• 23fdab00a8 [Hardware][TPU] Skip failed compilation test (#15421) Siyuan Liu 2025-03-24 16:28:57 -07:00
• 623e2ed29f [BugFix][V1] Quick fix for min_tokens with multiple EOS (#15407) Nick Hill 2025-03-24 15:58:59 -07:00
• 9d72daf4ce [V1][Perf] Simpler request output queues (#15156) Nick Hillandrshaw@neuralmagic.com <robertgshaw2@gmail.com> 2025-03-24 15:44:08 -07:00
• 6dd55af6c9 [Doc] Update docs on handling OOM (#15357) Cyrus LeungandRoger Wang 2025-03-25 05:29:34 +08:00
• 3eb08ed9b1 [DOC] Add Kubernetes deployment guide with CPUs (#14865) Yuan Tang 2025-03-24 13:48:43 -04:00
• 5eeadc2642 [Hardware][Gaudi][Feature] Enable Dynamic MoE for Mixtral (#12303) liuzhenwei 2025-03-25 00:48:40 +08:00
• 3aee6573dc [V1] Aggregate chunked prompt logprobs in model runner (#14875) Nick Hill 2025-03-24 09:27:57 -07:00
• 9cc645141d [MISC] Refine no available block debug msg (#15076) Yi LiuandYi Liu 2025-03-25 00:01:10 +08:00
• 0893567db9 [V1][Minor] fix comments (#15392) Chen1022andchenjincong 2025-03-24 23:45:32 +08:00
• 8abe69b499 [Core] Don't force uppercase for VLLM_LOGGING_LEVEL (#15306) Russell Bryant 2025-03-24 11:27:30 -04:00
• 761702fd19 [Core] Integrate fastsafetensors loader for loading model weights (#10647) Manish Sethi 2025-03-24 11:08:02 -04:00
• 9606d572ed [distributed] fix dp group (#15355) youkaichao 2025-03-24 22:54:27 +08:00
• cbcdf2c609 [Bugfix] Fix chat template loading (#15143) 3 people 2025-03-24 21:50:09 +08:00
• 038de04d7b Fix zmq IPv6 URL format error (#15341) Russell Bryant 2025-03-24 09:30:41 -04:00
• 6b3cc75be0 [Kernel] allow non-contiguous input for marlin kernel (#14658) Jinzhen Lin 2025-03-24 21:21:33 +08:00