Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

2dbe8c0774 [Perf] API-server scaleout with many-to-many server-engine comms (#17546) Nick Hill 2025-05-30 08:17:00 -07:00
84ec470fca Improve "failed to get the hash of the compiled graph" error (#18956) Richard Zou 2025-05-30 11:00:54 -04:00
b29ca5c4d5 [Docs] Update SECURITY.md with link to our security guide (#18961) Russell Bryant 2025-05-30 10:37:27 -04:00
ec6833c5e9 [doc] show the count for fork and watch (#18950) Reidandreidliu41 2025-05-30 21:45:59 +08:00
e1fadf1197 [Feature] minicpm eagle support (#18943) Shawn Huangandhuangyuxiang03 2025-05-30 21:45:56 +08:00
43ff405b90 [CI/Build] remove regex from build dependencies (#18945) DanieleandCyrus Leung 2025-05-30 13:02:50 +02:00
fba02e3bd1 [Bugfix][TPU] Fix tpu model runner testcase failure (#18810) Carol Zheng 2025-05-30 10:04:03 +00:00
4577fc9abb [Misc]Fix typo (#18947) Always-Naive 2025-05-30 17:21:35 +08:00
5f1d0c8118 [Bugfix][Failing Test] Fix test_vllm_port.py (#18618) Rabi Mishra 2025-05-30 14:43:47 +05:30
c3bb9f2331 [Model] Use in-place adds in SigLIP (#18922) Lukas Geiger 2025-05-30 10:12:59 +01:00
8f8900cee9 [doc] add mkdocs doc (#18930) Reidandreidliu41 2025-05-30 15:58:44 +08:00
6acb7a6285 [Misc]Fix benchmarks/README.md for speculative decoding (#18897) Rabi Mishra 2025-05-30 13:28:04 +05:30
4f4a6b844a [Deprecation] Remove mean pooling default for Qwen2EmbeddingModel (#18913) Cyrus Leung 2025-05-30 14:53:37 +08:00
4d0a1541be [Bugfix] Remove NVFP4 scales assertions to fix load_format=dummy (#18861) Michael Goin 2025-05-30 01:37:36 -04:00
77b6e74fe2 [ROCm] Remove unnecessary assertion of max_model_len in ROCM_AITER_MLA attention backend. (#18938) vllmellm 2025-05-30 13:33:17 +08:00
5acf828d99 [docs] fix: fix markdown syntax (#18927) H 2025-05-29 22:20:48 -07:00
3987e2ae96 [Model] Use AutoWeightsLoader for mamba2 (#18918) iLeGend 2025-05-30 12:50:10 +08:00
77164dad5e [Bugfix] Consistent ascii handling in tool parsers (#18883) Chauncey 2025-05-30 12:44:43 +08:00
3de3eadf5b improve the robustness of parsing vlms config in AutoRound (#18894) Wenhua Cheng 2025-05-30 10:24:47 +08:00
3132290a14 [TPU][CI/CD] Clean up docker for TPU tests. (#18926) Carol Zheng 2025-05-30 02:24:19 +00:00
1aa2f81b43 [Misc] Update type annotation for rotary embedding base (#18914) Cyrus Leung 2025-05-30 10:17:01 +08:00
d54af615d5 [Bugfix] Fix PP default fallback behavior for V1 (#18915) Michael Goin 2025-05-29 22:13:17 -04:00
a1cc9f33a3 [TPU] remove transpose ops in moe kernel (#18923) Chengji Yao 2025-05-29 16:00:11 -07:00
a521ef06e5 Use standalone_compile by default in torch >= 2.8.0 (#18846) Richard Zou 2025-05-29 18:41:58 -04:00
64eaf5fe05 [P/D] NixlConnector DP fixes (#18903) Will Eaton 2025-05-29 14:08:40 -04:00
d1d61f3351 [BugFix] Make DP work with connector-delayed new requests (#18559) Nick HillandWill Eaton 2025-05-29 11:04:18 -07:00
32ce3cf7c9 [V1] Allocate kv_cache with stride order for V1 (#18775) Nicolò Lucchesi 2025-05-29 19:54:16 +02:00
d58f9c7f7a [Misc] Remove duplicate init for self.vllm_config (#18896) CYJiang 2025-05-30 01:26:07 +08:00
c29034037d [Deprecation] Disallow pos-args other than model when initializing LLM (#18802) Cyrus Leung 2025-05-30 00:36:58 +08:00
1b7cfd5a36 [ROCm][V0][Attention] Revert to the previous FA triton kernel (#18226) Gregory Shtrasberg 2025-05-29 12:13:18 -04:00
da4b69d0b4 [Attention][V1] Toggle for v1 attention backend (#18275) Gregory Shtrasberg 2025-05-29 10:48:24 -04:00
c9479b2920 [Bugfix] Fix the failing gte embedding test (#18720) Isotr0py 2025-05-29 22:39:25 +08:00
6f2909405e [Doc] Fix codeblocks formatting in LoRA adapters documentation (#18907) Hyogeun Oh (오효근) 2025-05-29 23:38:55 +09:00
b169d5f7b6 [Misc][Tools][Benchmark] Add benchmark_serving supports for llama.cpp. (#18692) Duyi-Wang 2025-05-29 20:02:08 +08:00
f8977c233f Fix an error in dummy weight loading for quantization models (#18855) Chenyaaang 2025-05-29 03:07:20 -07:00
f274581f44 [BugFix] Update pydantic to fix error on python 3.10 (#18852) Luka Govedič 2025-05-29 06:05:46 -04:00
0b1447f890 [Bugfix] Ensure tensors are contiguous during serialisation (#18860) Lukas Geiger 2025-05-29 11:05:20 +01:00
24d0ef8970 [Misc] Replace TODO in serving transcription (#18895) Nicolò Lucchesi 2025-05-29 11:58:14 +02:00
7fcfd954ff [Bugfix] Fix misleading information in the documentation (#18845) Jee Jee Li 2025-05-29 17:54:14 +08:00
e740d07f07 [doc] add CLI doc (#18871) Reidandreidliu41 2025-05-29 17:51:36 +08:00
a652e71dd0 [Doc] Remove redundant spaces from compatibility_matrix.md (#18891) Michael Yao 2025-05-29 17:51:20 +08:00
34d6c447c4 [LoRA] Add LoRA support for InternVL (#18842) Jee Jee Li 2025-05-29 16:46:24 +08:00
972eddf7c9 [Neuron] Add multi-LoRA support for Neuron. (#18284) Satyajith Chilappagari 2025-05-29 01:41:22 -07:00
fd7bb88d72 Fixes a dead link in nightly benchmark readme (#18856) Brent Salisbury 2025-05-29 00:41:39 -04:00
3c49dbdd03 Skip device and quant Pydantic validation to make plugin device work (#18843) Yikun Jiang 2025-05-29 11:12:30 +08:00
1661a9c28f [Doc][Neuron] Update documentation for Neuron (#18868) aws-elaineyz 2025-05-28 19:44:01 -07:00
8e882ffdc0 [Bugfix][TPU] fix moe custom kernel import (#18853) Chengji Yao 2025-05-28 19:34:19 -07:00
26b4fa45be Add ability to use CUDAGraphs with use_inductor=False (#17345) Richard Zou 2025-05-28 22:16:52 -04:00
515b413ebf Prevent the cross-encoder logic from being applied to classification tasks (#18838) Maximilien de BayserandCyrus Leung 2025-05-28 23:16:17 -03:00
269d901734 [Bugfix][ROCm] fix the power of 2 exception from triton_unified_attention.py when running llama4 models and unit test fix (#18100) Hongxia Yangandtjtanaa 2025-05-28 19:21:46 -04:00
7951d78738 [Core] Enable CUDA graphs for DP + All2All kernels (#18724) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-05-28 18:55:30 -04:00
6dbe5b5c93 Remove checks for None for fields which should never be None (#17985) Harry Mellor 2025-05-28 22:32:19 +01:00
643622ba46 [Hardware][TPU][V1] Multi-LoRA Optimisations for the V1 TPU backend (#15655)
Akshat Tripathi Chengji Yao xihajun Jorge de Freitas Jorge de Freitas
2025-05-28 20:59:09 +01:00
a09c7ca9f2 [Chore][Spec Decode] Update check NoneType instead of assigning variables (#18836) Aaron Pham 2025-05-28 14:57:19 -04:00
0e98964e94 [V1][Metrics] Remove metrics that were deprecated in 0.8 (#18837) Mark McLoughlin 2025-05-28 19:54:12 +01:00
c68b5c63eb [Misc] fix olmoe model layer can't laod in tp gt 1 (#18828) rongfu.leng 2025-05-29 01:36:21 +08:00
fced756923 [Chore] update ty configuration (#18839) Aaron Pham 2025-05-28 11:59:11 -04:00
321331b8ae [Core] Add Lora Support to Beam Search (#18346) Alex Brooks 2025-05-28 09:58:24 -06:00
6e4cea1cc5 decrement server_load on listen for disconnect (#18784) daniel-salib 2025-05-28 07:15:12 -07:00
435fa95444 [Frontend] add run batch to CLI (#18804) Reidandreidliu41 2025-05-28 22:08:57 +08:00
4c2b38ce9e Enable Pydantic mypy checks and convert configs to Pydantic dataclasses (#17599) Harry Mellor 2025-05-28 13:46:04 +01:00
d781930f90 [Platform][Dist] Make torch distributed process group extendable (#18763) Mengqing Cao 2025-05-28 18:52:34 +08:00
ce75efeecb [BugFix] FA2 MLA Accuracy Issue (#18807) Lucas Wilkinson 2025-05-28 04:59:39 -04:00
aa42561e40 Fix PiecewiseCompileInterpreter (#17338) Richard Zou 2025-05-28 04:40:53 -04:00
de65fc8e1e [CI] improve embed testing (#18747) wang.yuqi 2025-05-28 15:16:35 +08:00
0c492b7824 [Deprecation] Remove fallbacks for Embeddings API (#18795) Cyrus Leung 2025-05-28 15:09:04 +08:00
0f0926b43f [Deprecation] Remove unused sync methods in async_timeout (#18792) Cyrus Leung 2025-05-28 15:08:48 +08:00
7f2c1a87e9 [Deprecation] Require overriding get_dummy_text and get_dummy_mm_data (#18796) Cyrus Leung 2025-05-28 15:08:35 +08:00
b78f844a67 [Bugfix][FailingTest]Fix test_model_load_with_params.py (#18758) Rabi Mishra 2025-05-28 11:12:54 +05:30
5e13c07d00 [V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (2) (#18781) RonaldBXu 2025-05-27 22:09:14 -07:00
774c5fde30 [V1] fix torch profiling for V1 offline scenarios (#18445) Divakar Verma 2025-05-27 23:16:30 -05:00
9a21e331ff [Bugfix]: correctly propagate errors message caught at the chat_templating step to the client (#18769) Guillaume Calmettes 2025-05-28 05:35:43 +02:00
3e9ce609bd [Bugfix] Fix nomic max_model_len (#18755) wang.yuqi 2025-05-28 11:29:53 +08:00
794ae1f551 [rocm] Fix wrong attention log (#18764) fxmarty-amd 2025-05-28 04:45:41 +02:00
d73a9457a5 [Core] Improve Tensor serialisation (#18774) Lukas Geiger 2025-05-28 02:46:21 +01:00
a3896c7f02 [Build] Fixes for CMake install (#18570) Luka Govedič 2025-05-27 20:49:24 -04:00
51e98e4ffd [Bugfix] Disable prefix caching by default for benchmark (#18771) cascade 2025-05-27 17:18:09 -07:00
e56f44d9ec Support datasets in vllm bench serve and sync with benchmark_[serving,datasets].py (#18566) Michael Goin 2025-05-27 19:59:48 -04:00
e0cbad4e30 [Neuron] Support quantization on neuron (#18283) Satyajith Chilappagari 2025-05-27 15:10:33 -07:00
b48d5cca16 [CI/Build] [TPU] Fix TPU CI exit code (#18282) Carol Zheng 2025-05-27 14:54:59 -07:00
5873877241 [Bugfix] Mistral tool calling when content is list (#18729) Michael Goin 2025-05-27 12:05:37 -04:00
696259ca01 [Core] Automatically cast multi-modal input dtype (#18756) Cyrus Leung 2025-05-27 23:45:48 +08:00
6b6d496114 optimize get_kv_cache_torch_dtype (#18531) chunxiaozheng 2025-05-27 21:08:44 +08:00
aaa4ac1c95 Disable prefix cache by default for benchmark (#18639) cascade 2025-05-27 05:06:34 -07:00
06a0338015 [V1][Metrics] Add API for accessing in-memory Prometheus metrics (#17010) Mark McLoughlin 2025-05-27 10:37:06 +01:00
4318c0559d [CI/Build] Remove imports of built-in re (#18750) Cyrus Leung 2025-05-27 17:19:18 +08:00
a68e293cb9 [Doc] Convert Sphinx directives ( {class}, {meth}, {attr}, ...) to MkDocs format for better documentation linking (#18663) Hyogeun Oh (오효근) 2025-05-27 17:44:20 +09:00
6881107948 [BUG FIX] minicpm (#18739) Shawn Huangandhuangyuxiang03 2025-05-27 16:04:49 +08:00
e0f0ff87b8 [Build] fix cpu build missing libtbbmalloc.so (#18744) Kebe 2025-05-27 16:03:56 +08:00
c24b1572ac Minor fix about MooncakeStoreConnector (#18721) maobaolong 2025-05-27 16:02:28 +08:00
4693a3438c [Doc] cleanup deprecated flag for doc (#18715) Calvin Chen 2025-05-27 15:12:02 +08:00
bbd9a84dc5 [Hardware][Intel-Gaudi] [CI/Build] Fix multiple containers using the same name in run-hpu-test.sh (#18752) Łukasz Durejko 2025-05-27 09:10:26 +02:00
a547aeb828 feat(rocm-support): support mamba2 on rocm (#18565) almersawiandIslam Almersawi 2025-05-27 11:07:53 +04:00
fc6d0c290f [Misc] improve docs (#18734) Reidandreidliu41 2025-05-27 15:07:01 +08:00
753944fa9b [Doc] Update reproducibility doc and example (#18741) Cyrus Leung 2025-05-27 15:03:13 +08:00
25a817f202 [Doc] Update OOT model docs (#18742) Cyrus Leung 2025-05-27 14:30:31 +08:00
d260f799a9 [FEAT] [ROCm] Upgrade AITER Fused MoE kernels. (#18271) vllmellm 2025-05-27 14:14:07 +08:00
b50602d5f0 [Model][Gemma3] Cast image pixel values already on CPU (#18732) Lukas Geiger 2025-05-27 06:42:54 +01:00
1f1b1bc03b [V1][Quantization] Add CUDA graph compatible v1 GGUF support (#18646) Isotr0py 2025-05-27 12:40:28 +08:00
1f88dbd2bb [Misc] improve web section group title display (#18684) Reidandreidliu41 2025-05-27 12:35:16 +08:00

... 164 165 166 167 168 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86