Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• f719772281 [Bugfix] Properly reject requests with empty list guided_choice (#20195) Michael Goin 2025-06-28 14:50:52 +09:00
• d45417b804 fix ci issue distributed 4 gpu test (#20204) Wentao Ye 2025-06-28 01:50:00 -04:00
• a29e62ea34 Fix num_token_padding support for static per-tensor scaled_fp8_quant (#20188) Michael Goin 2025-06-28 14:48:13 +09:00
• e53be6f00a [Misc] Add type assertion of request_id for LLMEngine.add_request (#19700) Chales Xu 2025-06-28 13:47:36 +08:00
• c329ceca6d [CI Fix] Pin tests/models/registry.py MiniMaxText01ForCausalLM to revision due to model changes (#20199) Michael Goin 2025-06-28 14:43:06 +09:00
• 3c545c0c3b [CI/Build] Allow hermetic builds (#18064) 4 people 2025-06-27 18:04:39 +02:00
• e8c3bd2cd1 [Bugfix] Fix some narrowing conversion warnings (#20141) Tyler Michael Smith 2025-06-27 12:01:28 -04:00
• c6c983053d [Bugfix] Mark 'hidden_states' as mutable in moe_forward registration. (#20152) bnellnm 2025-06-27 11:42:22 -04:00
• aafabaa0d5 [Fix][torch.compile] Enable custom ops by default when Inductor off (#20102) Luka Govedič 2025-06-27 11:00:42 -04:00
• 94a55c7681 [Fix][ROCm] Remove unused variables to fix build error on GFX11/12 (#19891) Hosang 2025-06-27 10:14:44 -04:00
• aa0dc77ef5 [Perf] Improved perf for resolve_chat_template_content_format (#20065) Ilya Lavrenov 2025-06-27 13:16:41 +04:00
• 4ab3ac285e [Bugfix] Fix flaky failure when getting DP ports (#20151) Michael Goin 2025-06-27 16:30:53 +09:00
• d1c956dc0f Gemma3n (Text-only) (#20134) Robert ShawandRoger Wang 2025-06-27 03:16:26 -04:00
• dec197e3e5 Quick Fix by adding conditional import for flash_attn_varlen_func in flash_attn (#20143) Chendi.Xue 2025-06-27 00:48:13 -05:00
• 6e244ae091 [Perf][Frontend] eliminate api_key and x_request_id headers middleware overhead (#19946) Yazan Sharaya 2025-06-27 07:44:14 +03:00
• cd4cfee689 [Model][1/N] Automatic conversion of CrossEncoding model (#20012) wang.yuqi 2025-06-27 12:10:04 +08:00
• e110930680 [Fix] Fix gemma CI test failing on main (#20124) Thomas Parnell 2025-06-27 06:06:59 +02:00
• 8b64c895c0 [CI] Sync test dependency with test.in for torch nightly (#19632) 5 people 2025-06-26 20:55:25 -07:00
• 0740e29b66 [Feature] add quick all reduce (#19744) li haoyangandilmarkov 2025-06-27 11:54:24 +08:00
• 44d2e6af63 [Bugfix] Build moe_data for both sm100 and sm90 (#20086) Michael Goin 2025-06-27 12:50:12 +09:00
• 2d7779f888 [Perf] SM100 FP8 GEMM Optimizations after cutlass_profiler (#20071) Ilya Markovandilmarkov 2025-06-27 05:50:09 +02:00
• a57d57fa72 [Quantization] Bump to use latest compressed-tensors (#20033) Dipika SikkaandKyle Sayers 2025-06-26 23:50:06 -04:00
• 71799fd005 [CI Failure] Fix OOM with test_oot_registration_embedding (#20144) Michael Goin 2025-06-27 12:21:04 +09:00
• e9fd658a73 [Feature] Expert Parallelism Load Balancer (EPLB) (#18343) Bowen Wang 2025-06-26 15:30:21 -07:00
• 07b8fae219 [Doc] correct LoRA capitalization (#20135) Kyle Yu 2025-06-26 18:22:12 -04:00
• 562308816c [Refactor] Rename commnication utils (#20091) Wentao Ye 2025-06-26 18:19:32 -04:00
• 04e1642e32 [TPU] add kv cache update kernel (#19928) Chengji Yao 2025-06-26 10:01:37 -07:00
• b69781f107 [Hardware][Intel GPU] Add v1 Intel GPU support with Flash attention backend. (#19560) Kunshang Ji 2025-06-27 00:27:18 +08:00
• 0bceac9810 Spam folks if config.py changes (#20131) Tyler Michael Smith 2025-06-26 11:19:46 -04:00
• 34878a0b48 [Doc] Rename page titles (#20130) Cyrus Leung 2025-06-26 23:18:49 +08:00
• 6393b03986 [Doc] Auto sign-off for VSCode (#20132) Cyrus Leung 2025-06-26 23:18:36 +08:00
• 0907d507bf [Doc] Automatically signed-off by PyCharm (#20120) wang.yuqi 2025-06-26 22:34:17 +08:00
• c894c5dc1f [Bug Fix] Fix address/port already in use error for deep_ep test (#20094) Wentao Ye 2025-06-26 10:33:13 -04:00
• 1f5d178e9c Revert "[Bugfix] default set cuda_graph_sizes to max_num_seqs for v1 engine" (#20128) Michael Goin 2025-06-26 23:32:22 +09:00
• 27c065df50 [Bugfix][V1][ROCm] Fix AITER Flash Attention Backend (Fix API Break and Local Attention Logic: affecting Llama4) (#19904) TJian 2025-06-26 05:42:31 -07:00
• 84c260caeb [Docs] Improve frameworks/helm.md (#20113) Michael Yao 2025-06-26 18:41:51 +08:00
• 167aca45cb [Misc] Use collapsible blocks for benchmark examples. (#20017) Reidandreidliu41 2025-06-26 18:35:16 +08:00
• 0567c8249f [CPU] Fix torch version in x86 CPU backend (#19258) Li, Jiang 2025-06-26 18:34:47 +08:00
• d188913d99 [Refactor] Remove unused library (#20099) Wentao Ye 2025-06-26 05:16:10 -04:00
• 1d7c29f5fe [Doc] Update docs for New Model Implementation (#20115) Cyrus Leung 2025-06-26 15:47:06 +08:00
• 65397e40f5 [Bugfix] Allow CUDA_VISIBLE_DEVICES='' in Platform.device_id_to_physical_device_id (#18979) Seiji Eicher 2025-06-26 00:01:57 -07:00
• 9502c38138 [Benchmark][Bug] Fix multiple bugs in bench and add args to spec_decode offline (#20083) Ekagra Ranjan 2025-06-26 01:06:27 -04:00
• 2582683566 [PD] Skip tp_size exchange with rank0 (#19413) Nicolò Lucchesi 2025-06-26 05:04:39 +02:00
• 754b00edb3 [Bugfix] Fix Mistral tool-parser regex for nested JSON (#20093) Michael Goin 2025-06-26 10:01:17 +09:00
• 296ce95d8e [CI] Add SM120 to the Dockerfile (#19794) Michael Goin 2025-06-26 08:23:56 +09:00
• 2d7620c3eb [TPU] Add TPU specific var VLLM_TPU_MOST_MODEL_LEN (#19919) Chenyaaang 2025-06-25 15:51:02 -07:00
• 55c65ab495 [P/D] Avoid stranding blocks in P when aborted in D's waiting queue (#19223) Nick Hill 2025-06-25 15:19:44 -07:00
• 2cc2069970 [TPU][Bugfix] fix kv cache padding (#20048) Chengji Yao 2025-06-25 14:24:10 -07:00
• 9f0608fc16 [Bugfix] default set cuda_graph_sizes to max_num_seqs for v1 engine (#20062) zhrrr 2025-06-26 05:03:17 +08:00
• 4e0db57fff Fix the path to the testing script. (#20082) QiliangCui 2025-06-25 13:48:17 -07:00
• c40692bf9a [Misc] Add parallel state node_count function (#20045) Nick Hill 2025-06-25 13:38:53 -07:00
• 4734704b30 [PD] let toy proxy handle /chat/completions (#19730) lkchen 2025-06-25 12:17:45 -07:00
• 8b8c209e35 static_scaled_fp8_quant should not run when scale.numel is not 1 (#20076) Eldar Kurtić 2025-06-25 21:08:03 +02:00
• 23a04e0895 [Fix] Support cls pooling in ModernBertPooler (#20067) lsz05 2025-06-26 04:07:45 +09:00
• 02c97d9a92 [Quantization] Add compressed-tensors emulations support for NVFP4 (#19879) Dipika Sikka 2025-06-25 14:28:19 -04:00
• e795d723ed [Frontend] Add /v1/audio/translations OpenAI API endpoint (#19615) Nicolò LucchesiandRoger Wang 2025-06-25 19:54:14 +02:00
• 8359f4c8d8 [V1][Speculative Decoding] Fix DeepSeek MTP (#20022) cjackal 2025-06-26 00:41:02 +09:00
• bf5181583f [Doc] Guide for Incremental Compilation Workflow (#19109) Michael Goin 2025-06-25 22:06:46 +09:00
• c53fec1fcb [doc] add reference link for Intel XPU (#20064) Reidandreidliu41 2025-06-25 20:24:07 +08:00
• 0f9e7354f5 [BugFix] Fix full-cuda-graph illegal memory access in FA3 (#20057) Lucas Wilkinson 2025-06-25 04:39:04 -04:00
• ba7ba35cda [Chore] debloat some initial logs (#19438) Aaron Pham 2025-06-25 02:36:22 -04:00
• 015fab8c2f [Kernels][Bugfix] Use torch op for all kernels in FusedMoE forward. Add additional testing for cudagraphs. (#19717) bnellnm 2025-06-25 02:22:58 -04:00
• f59fc60fb3 [Feat][CLI] enforce-include-usage (#19695) Max Wittig 2025-06-25 07:43:04 +02:00
• 879f69bed3 [Refactor] Remove duplicate ceil_div (#20023) Wentao Ye 2025-06-25 01:19:09 -04:00
• 7108934142 [Frontend] speed up import time of vllm.config (#18036) David Xia 2025-06-25 00:41:11 -04:00
• 3443aaf8dd Move to a faster base64 implementation (#19984) h-avsha 2025-06-25 06:33:51 +03:00
• 2273ec322c Revert "Fix(models/siglip): Add compatibility for Gemma models quantized by llm-compressor" (#20030) Isotr0py 2025-06-25 11:23:29 +08:00
• a6c4b87fbc Revert "[Feature] Integrate new deepgemm (#19820)" (#20049) Wentao Ye 2025-06-24 22:45:22 -04:00
• 1afa9948f5 [Llama4] Update attn_temperature_tuning (#19997) Brayden Zhong 2025-06-24 22:42:53 -04:00
• 0d06b533a0 cmake: Update vllm_flash_attn for vllm_kernels (#20032) Eli Uriegas 2025-06-24 15:44:10 -07:00
• c01d1c5aba use .dev for version comparison with pytorch nightly release (#20031) Boyuan Feng 2025-06-24 14:52:16 -07:00
• ead369845d [Easy] Remove submodule added in #19463 (#20039) Brayden Zhong 2025-06-24 16:23:15 -04:00
• c6e3bba8e6 [Feature] Integrate new deepgemm (#19820) Wentao Ye 2025-06-24 15:51:56 -04:00
• 91f7d9d0b6 [P/D] Asynchronously do _nixl_handshake (#19836) lkchenandNick Hill 2025-06-24 12:46:10 -07:00
• 8619e7158c [BugFix] Fix multi-node offline data parallel (#19937) Nick Hill 2025-06-24 12:45:20 -07:00
• c635c5f744 [Misc][Benchmarking] Add variable request-rate ("ramp-up") to the benchmarking client. (#19423) 3 people 2025-06-24 20:41:49 +02:00
• a045b7e89a [Perf] Improve/Fix-regression for FA3 in High QPS regimes (#19463) Lucas Wilkinson 2025-06-24 13:09:01 -04:00
• 981eeca41a [Fix][V1] Remove --scheduling-policy oracle (#20010) amit 2025-06-24 19:52:15 +03:00
• 26d34eb67e refactor example - qwen3_reranker (#19847) Reidandreidliu41 2025-06-24 22:03:20 +08:00
• 53da4cd397 [Bugfix][CPU] Fix InputBatch for pooling models in the CPU v1 (#20014) Li, Jiang 2025-06-24 21:20:04 +08:00
• 9a3b88328f [PERF] Speedup of MRoPE prepare inputs (#19939) Vadim Gimpelson 2025-06-24 10:01:26 +04:00
• 3014c920da add some examples for other benchmark scripts (#19893) Reidandreidliu41 2025-06-24 13:57:46 +08:00
• 0eed516951 [doc] Fix broken link in the installation for CPU (#19980) Kay Yan 2025-06-24 12:04:11 +08:00
• ee5ad8d2c5 [Misc][Tools][Benchmark] Add profile to autotune script (#19711) Chenyaaang 2025-06-23 17:59:41 -07:00
• a738dbb2a1 Update test case parameter to have the throughput above 8.0 (#19994) QiliangCui 2025-06-23 17:18:10 -07:00
• 33d5e29be9 [TPU] Fix tpu model runner test (#19995) Chenyaaang 2025-06-23 16:04:28 -07:00
• 4671ac6e2a [Bugfix][Benchmark] Fix Marlin benchmark (#19929) 22quinn 2025-06-23 15:25:12 -07:00
• dd2ccf8dde Feat Dynamic Quantization for MoE Layers in GPTQ Marlin Backend (#19395) Jun-Howie 2025-06-24 06:23:28 +08:00
• a3bc76e4b5 [CI/Build] Push latest tag for cpu and neuron docker image (#19897) 22quinn 2025-06-23 14:15:37 -07:00
• e6327c9b3e [Feature] Support sequence parallelism for static fp8 quantization (#19181) cascade 2025-06-23 13:09:02 -07:00
• d0132f025d [Misc] Add type alias ReqId and EngineId for better readability (#19880) lkchen 2025-06-23 12:57:57 -07:00
• 61f4fc5dc6 [Bugfix][v1] Fix step pooler implementation and step pooling usage in v1 (#19956) Isotr0py 2025-06-24 02:38:06 +08:00
• 68aaeb3749 [EP+DP] Optimize the little operations in the DeepGEMM + DeepEP low latency case (#19885) Tyler Michael SmithandVarun Sundar Rabindranath 2025-06-23 14:07:47 -04:00
• c3649e4fee [Docs] Fix syntax highlighting of shell commands (#19870) Lukas Geiger 2025-06-23 18:59:09 +01:00
• 53243e5c42 [doc] improve readability for long commands (#19920) Reidandreidliu41 2025-06-23 22:27:07 +08:00
• a6e6604d32 [Bugfix] Fix CI bitsandbytes failure (#19969) Jee Jee Li 2025-06-23 21:30:55 +08:00
• b82e0f82cb [doc] use MkDocs collapsible blocks - supplement (#19973) Reidandreidliu41 2025-06-23 18:54:16 +08:00
• 5111642a6f [Doc] Update V1 status for decoder-only embedding models (#19952) Isotr0py 2025-06-23 17:31:06 +08:00
• 1bcd15edc7 [BugFix][P/D] Fix for cases where _recving_transfers can be cleaned up when *all* transfer done (#19874) lkchen 2025-06-22 22:41:53 -07:00
• 2ebff5b77c [P/D][NixlConnector] Support tp_size > num_kv_heads deployments (#19691) Nicolò LucchesiandNick Hill 2025-06-23 07:41:50 +02:00