Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 5e70dcd6e6 [Doc] Fix CPU doc format (#21316) Li, Jiang 2025-07-22 12:47:49 +08:00
• 25d585ab7b [XPU] Enable external_launcher to serve as an executor via torchrun (#21021) Chaojun Zhang 2025-07-22 12:47:35 +08:00
• 8d0a01a5f2 [v1][sampler] Inplace logprobs comparison to get the token rank (#21283) Lu Fang 2025-07-21 13:47:47 -07:00
• 0ec82edda5 [perf] Speed up align sum kernels (#21079) Himanshu Jaju 2025-07-21 19:19:23 +01:00
• 005ae9be6c Fix bad lm-eval fork (#21318) Michael Goin 2025-07-21 13:47:51 -04:00
• 29d1ffc5b4 [DP] Fix Prometheus Logging (#21257) Robert ShawandRobert Shaw 2025-07-21 12:11:35 -04:00
• 304dce7ec0 [Attention] Clean up iRoPE in V1 (#21188) Lucas WilkinsonandMichael Goin 2025-07-21 12:10:30 -04:00
• 6ece16c4fe [Misc] Add dummy maverick test (#21199) Ming Yangandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-07-21 09:08:09 -07:00
• a0e827e07c [BugFix] make utils.current_stream thread-safety (#21252) (#21253) simpx 2025-07-22 00:07:36 +08:00
• a15a50fc17 [CPU] Enable shared-memory based pipeline parallel for CPU backend (#21289) Li, Jiang 2025-07-22 00:07:08 +08:00
• 6dda13c86b [Misc] Add sliding window to flashinfer test (#21282) Woosuk Kwon 2025-07-21 08:37:49 -07:00
• 6b46c4b653 Add Nvidia ModelOpt config adaptation (#19815) Zhiyu 2025-07-21 07:02:58 -07:00
• d97841078b [Misc] unify variable for LLM instance (#20996) Ning Xie 2025-07-21 19:18:33 +08:00
• e6b90a2805 [Docs] Make tables more space efficient in supported_models.md (#21291) Harry Mellor 2025-07-21 10:25:02 +01:00
• be54a951a3 [Docs] Fix hardcoded links in docs (#21287) Harry Mellor 2025-07-21 10:23:57 +01:00
• 042af0c8d3 [Model][1/N] Support multiple poolers at model level (#21227) Cyrus Leung 2025-07-21 17:22:21 +08:00
• 378d33c392 [Bugfix] Fix missing placeholder in logger debug (#21280) Cyrus Leung 2025-07-21 13:50:06 +08:00
• 940af1f03a Add the instruction to run e2e validation manually before release (#21023) Huy Do 2025-07-20 22:29:18 -07:00
• 92615d7fe8 [Docs] Add RFC Meeting to Issue Template (#21279) Simon Mo 2025-07-20 21:58:07 -07:00
• 8188196a1c [CI] Cleanup modelscope version constraint in Dockerfile (#21243) Kay Yan 2025-07-21 11:13:02 +08:00
• 7ba34b1241 [bugfix] fix syntax warning caused by backslash (#21251) Jiayi Yan 2025-07-21 01:12:10 +08:00
• 9499e26e2a [Model] Support VLMs with transformers backend (#20543) 4 people 2025-07-20 15:25:50 +02:00
• 51ba839555 [Model] use AutoWeightsLoader for bart (#18299) Calvin Chen 2025-07-20 16:15:50 +08:00
• d1fb65bde3 Enable v1 metrics tests (#20953) Seiji Eicher 2025-07-19 20:22:02 -07:00
• 3a1d8940ae [TPU] support fp8 kv cache quantization (#19292) Chengji Yao 2025-07-19 20:01:00 -07:00
• 2b504eb770 [Docs] [V1] Update docs to remove enforce_eager limitation for hybrid models. (#21233) Thomas Parnell 2025-07-20 01:09:58 +02:00
• 10eb24cc91 GLM-4 Update (#20736) 3 people 2025-07-20 06:40:31 +08:00
• 2e8cbb58f3 [BugFix] Fix full cuda graph slot_mapping (#21228) fhl2000 2025-07-20 05:13:18 +08:00
• 752c6ade2e [V0 Deprecation] Deprecate BlockSparse Attention & Phi3-Small (#21217) Woosuk Kwon 2025-07-19 13:53:17 -07:00
• 881e3cbe3b [V1] [Hybrid] Enable piecewise CUDA Graph for mamba layers (#21194) Thomas Parnell 2025-07-19 21:27:21 +02:00
• 9f414a12ad [BugFix] Make PD work with Ray (#21072) kourosh hakhamaneshi 2025-07-19 08:46:50 -07:00
• 6a971ed692 [Docs] Update the link to the 'Prometheus/Grafana' example (#21225) Jiayi Yan 2025-07-19 21:58:07 +08:00
• da6579bf41 [CI/CD][bugfix]fix: error argument to loads has incompatible type (#21223) Sungjae Lee 2025-07-19 21:16:48 +09:00
• c81259d33a Fix/remove some broken model executor tests (#21224) Rabi Mishra 2025-07-19 17:45:07 +05:30
• e3a0e43d7f [bugfix] Fix auto thread-binding when world_size > 1 in CPU backend and refactor code (#21032) Li, Jiang 2025-07-19 20:13:55 +08:00
• b3d82108e7 [Bugfix][Frontend] Fix openai CLI arg middleware (#21220) 22quinn 2025-07-19 02:40:38 -07:00
• 6d0734c562 [NVIDIA] Add SM100 Flashinfer MoE blockscale fp8 backend for low latency (#20645) Kaixi Houandmgoin 2025-07-19 02:33:01 -07:00
• 7d94577138 Add torch golden impl for moe_align_block_size kernel test (#20653) shixiancandShixian Cui 2025-07-19 02:32:36 -07:00
• 59f935300c [BugFix] Fix potential cuda-graph IMA (#21196) Lucas Wilkinson 2025-07-19 05:18:47 -04:00
• 18e519ec86 [Bugfix] Fix ndarray video color from VideoAsset (#21064) Isotr0py 2025-07-19 17:17:16 +08:00
• 1eaff27815 [V0 deprecation] Remove long context LoRA (#21169) Jee Jee Li 2025-07-19 17:15:41 +08:00
• cf8cc32674 Fix a couple of Voxtral tests (#21218) Huy Do 2025-07-19 02:13:41 -07:00
• 3a2cb2649d [Misc][Tools][Benchmark] Add readme file for auto_tune script (#20779) Chenyaaang 2025-07-19 02:06:59 -07:00
• 3e04107d97 [Model] EXAONE 4.0 model support (#21060) 김종곤 2025-07-19 15:25:44 +09:00
• 37bd8d6e4c [Bug] DeepGemm: Fix TypeError: per_block_cast_to_fp8() missing 1 required positional argument: 'use_ue8m0' for SM100 (#21187) Wentao Ye 2025-07-19 02:25:22 -04:00
• 468e2400fe [BugFix][CPU] Fix TorchSDPABackendImpl doesn't have use_irope (#21200) Lucas Wilkinson 2025-07-19 02:18:48 -04:00
• dcc6cfb991 [Kernel][Performance] Tweak MoE Batched silu_mul_fp8_quant_deep_gemm kernel (#21193) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-07-19 11:39:51 +05:30
• dd572c0ab3 [V0 Deprecation] Remove V0 Spec Decode workers (#21152) Woosuk Kwon 2025-07-18 21:47:50 -07:00
• 9ffe905a41 [Bugfix][Model] Fix LoRA for Mistral-Small-3.1-24B-Instruct-2503 (#21183) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-07-19 09:45:03 +05:30
• 9a9fda1423 [Core] Support Local Chunked Attention for Hybrid KV Cache (#19351) Lucia FangandLu Fang 2025-07-19 11:48:38 +08:00
• 466e878f2a [Quantization] Enable BNB support for more MoE models (#21100) Jee Jee Li 2025-07-19 08:52:02 +08:00
• 217937221b Elastic Expert Parallel Initial Support (#20775) Rui Qiao 2025-07-18 17:46:09 -07:00
• 5782581acf [Bugfix] Voxtral on Blackwell GPUs (RTX 50 series) (#21077) hax0r31337 2025-07-19 00:40:18 +02:00
• 0f199f197b [Core] Avoid KVCacheBlock.__eq__ invocations in FreeKVCacheBlockQueue (#21005) JialinOuyang-Meta 2025-07-18 12:34:40 -07:00
• b2eb2b5ad7 [Kernel] Apply torch.Tag.needs_fixed_stride_order only for torch==2.6.0 (#19346) Richard Zou 2025-07-18 14:10:21 -04:00
• 21274ab476 [CI] Update CODEOWNERS for vllm/compilation (#21185) Richard Zou 2025-07-18 09:51:12 -04:00
• ed8cbfedf8 Let GraniteMoeAttention use YaRN (#21174) Thomas Parnell 2025-07-18 14:52:52 +02:00
• 45badd05d0 [Core] Set pooling params based on task and model (#21128) Cyrus Leung 2025-07-18 20:41:17 +08:00
• 4adc66f64d [Bugfix] Allocate less memory in non-batched CUTLASS MoE (#21121) ElizaWszola 2025-07-18 12:55:52 +02:00
• 55ad648715 [Doc] Fix typo in model name (#21178) Cyrus Leung 2025-07-18 18:55:10 +08:00
• 5895afd780 [Bugfix] The special_tokens in tokenizer should also be controlled by do_lower_case in encoder_config. (#20750) wang.yuqi 2025-07-18 17:10:47 +08:00
• ca4eb82bcb [Model] Re-add the implicit conversion feature for as_seq_cls_model (#21103) wang.yuqi 2025-07-18 15:15:07 +08:00
• ba2dfbb0c2 [Misc] Make MM embedding merge interface explicit in model runner (#21147) Roger Wangandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-07-18 00:13:57 -07:00
• 1bf65138f6 [benchmark] Sending request strictly follows the random intervals (#21108) Jialin Ouyang 2025-07-17 23:22:08 -07:00
• 54cf1cae62 [Misc] Do not print async output warning for v1 (#21151) Woosuk Kwon 2025-07-17 21:57:02 -07:00
• 5780121c95 [Perf] Add swap_ab to SM90 FP8 non-block CUTLASS moe grouped gemm (#20911) shixiancandShixian Cui 2025-07-17 21:34:43 -07:00
• c7d8724e78 [Core] FlashInfer CUTLASS fused MoE backend (NVFP4) (#20037) Shu Wangandmgoin 2025-07-17 23:32:45 -05:00
• b38baabcf9 [Doc] Add inplace weights loading example (#19640) 22quinn 2025-07-17 21:12:23 -07:00
• 89cab4d01f [Attention] Make local attention backend agnostic (#21093) Lucas Wilkinson 2025-07-18 00:10:42 -04:00
• b9a21e9173 [Docs] Update supported models documentation with missing models (#20844) Lucia Fang 2025-07-18 11:12:13 +08:00
• c4e3b12524 [Docs] Add minimal demo of Ray Data API usage (#21080) Ricardo Decal 2025-07-17 20:09:19 -07:00
• 8dfb45ca33 [Bugfix] Fix the tensor non-contiguous issue for Flashinfer TRT-LLM backend attention kernel (#21133) elvischenv 2025-07-18 08:35:58 +08:00
• 8a8fc94639 [Log] Debugging Log with more Information (#20770) Wentao Ye 2025-07-17 20:19:46 -04:00
• 4de7146351 [V0 deprecation] Remove V0 HPU backend (#21131) Woosuk Kwon 2025-07-17 16:37:36 -07:00
• ac9fb732a5 On environments where numa cannot be detected we get 0 (#21115) Eric Curtin 2025-07-17 19:52:17 +01:00
• a3a6c695f4 [Misc] Qwen MoE model supports LoRA (#20932) Jee Jee Li 2025-07-18 02:32:52 +08:00
• 90bd2ab6e3 [Model] Update pooling model interface (#21058) Cyrus Leung 2025-07-18 00:05:40 +08:00
• 9fb2d22032 [Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) ElizaWszola 2025-07-17 15:56:44 +02:00
• 2d6a38209b [Docs] Move code block out of admonition now that it's short (#21118) Harry Mellor 2025-07-17 14:12:29 +01:00
• 89e3c4e9b4 [Misc] Avoid unnecessary import (#21106) wangxiyuan 2025-07-17 20:57:41 +08:00
• fe8a2c544a [Docs] Improve docstring formatting for FusedMoEParallelConfig.make (#21117) Harry Mellor 2025-07-17 12:13:00 +01:00
• 4ef00b5cac [VLM] Add Nemotron-Nano-VL-8B-V1 support (#20349) kYLeandCyrus Leung 2025-07-17 05:07:55 -05:00
• 5a7fb3ab9e [Model] Add ToolParser and MoE Config for Hunyuan A13B (#20820) Asher 2025-07-17 17:10:09 +08:00
• 11dfdf21bf [Kernel] DeepGemm MoE : Integrate triton permute / unpermute kernels (#20903) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-07-17 13:40:37 +05:30
• fdc5b43d20 [Bugfix]: Fix final_res_batch list index out of range error (#21055) Chauncey 2025-07-17 15:29:09 +08:00
• c5b8b5953a [Misc] Fix PhiMoE expert mapping (#21085) Jee Jee Li 2025-07-17 13:47:49 +08:00
• 4fcef49ec4 [V1] [KVConnector] Fix MultiprocExecutor worker output aggregation (#21048) David Ben-DavidandDavid Ben-David 2025-07-17 08:29:45 +03:00
• 8a4e5c5f3c [V1][P/D]Enhance Performance and code readability for P2pNcclConnector (#20906) Zhonghua Deng 2025-07-17 13:13:00 +08:00
• 76b494444f [Attention] Refactor attention metadata builder interface (#20466) Lucas Wilkinson 2025-07-17 00:44:25 -04:00
• 28a6d5423d [Bugfix] Fix Machete zero point issue for GPTQ models on SM90 (#21066) Michael Goin 2025-07-16 22:54:45 -04:00
• 58760e12b1 [TPU] Start using python 3.12 (#21000) XiongfeiWei 2025-07-16 19:37:44 -07:00
• a50d918225 [Docker] Allow FlashInfer to be built in the ARM CUDA Dockerfile (#21013) Michael Goin 2025-07-16 22:37:13 -04:00
• c9ba8104ed [Bugfix] weight loading use correct tp_group with patch_tensor_parallel_group (#21024) Kevin_Xiong 2025-07-17 10:36:36 +08:00
• 4e7dfbe7b4 Update PyTorch to torch==2.7.1 for CUDA (#21011) Michael Goin 2025-07-16 22:30:44 -04:00
• 72ad273582 Remove torch_xla.tpu.version() from pallas.py. (#21065) QiliangCui 2025-07-16 17:25:26 -07:00
• 01513a334a Support FP8 Quantization and Inference Run on Intel Gaudi (HPU) using INC (Intel Neural Compressor) (#12010) Nir DavidandUri Livne 2025-07-16 22:33:41 +03:00
• ac2bf41e53 [Model] Remove model sampler (#21059) Cyrus Leung 2025-07-17 03:03:37 +08:00
• a931b4cdcf Remove Qwen Omni workaround that's no longer necessary (#21057) Harry Mellor 2025-07-16 17:25:23 +01:00
• a0f8a79646 [fix] fix qwen image_embeds input (#21049) Avshalom Manevich 2025-07-16 17:17:20 +02:00
• 18bdcf4113 feat - add a new endpoint get_tokenizer_info to provide tokenizer/chat-template information (#20575) Mac Misiura 2025-07-16 14:52:14 +01:00