Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

16dc4052b0 Fix pre-commit on main (#23747) Harry Mellor 2025-08-27 14:39:48 +01:00
8dd2baa597 Add vLLM Korea Meetup in the README.md and meetups.md (#23746) rebel-hongseokandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-08-27 22:25:49 +09:00
5eeef1b908 [Model] Explicit default_pooling_type interface (#23736) Cyrus Leung 2025-08-27 21:24:09 +08:00
704432af3c [V1] [Hybrid] Disable prefix caching by default for hybrid or mamba-based models (#23716) Thomas Parnell 2025-08-27 14:51:54 +02:00
a403d0fa41 [Misc] Remove unnecessary _send_reconfig_message() in core_client.py (#23127) Nick Hill 2025-08-27 05:50:47 -07:00
8c13820f0b [Bugfix] Fix task field initialization when PYTHONOPTIMIZE is enabled (#23718) cndoit18 2025-08-27 20:42:20 +08:00
9d30de4469 [model] Support MiniCPM-V 4.5 (#23586) +46
tc-mb Xin Yang Wentao Ye Zhonghua Deng Chaojun Zhang Pate Motter Terrence Zhao Woosuk Kwon Simon Mo Michael Goin weiliang Siyuan Fugemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung Copilot ProExpertProg Luka Govedič Zijing Liu Bin Jia Jiangyun Zhu Harry Mellor Raghavan Jee Jee Li Cyrus Leung Roger Wang Roger Wang knlnguyen1802 Huy Do Matúš Námešný Guillaume Calmettes Chen Zhang En OuyangLi, Jiang <jiang1.li@intel.com>nvjullin Didier Durand TianyuLi0 Hongxia Yang Yuekai Zhang vllmellm Hyogeun Oh Thomas Parnell Russell Bryant Lukas Geiger Isotr0py Huzaifa Sidhpurwala Federico zixuanzhang226 wuhang yzds hongchao czhu-cohere Wei Yiheng Xu Aaron Pham Chenheli Hua CSWYF3634076
2025-08-27 20:38:00 +08:00
1f7a9c95e4 [Docs] Fix a 1-2-3 list and style issues in tpu.md (#23729) Michael Yao 2025-08-27 20:37:52 +08:00
8f0d7eaea8 [XPU] Fix OOM issue for data parallel with Ray backend (#22500) Fanli LinandCyrus Leung 2025-08-27 19:57:38 +08:00
e03940762b [CI/Build] Reduce LoRA layer test cases (#23721) Jee Jee Li 2025-08-27 18:59:35 +08:00
11eddf02f0 [FlashInfer] Cache hyper params in metadata builder (#23732) Woosuk Kwon 2025-08-27 03:45:04 -07:00
04ff1e43fb [Misc] Move CpuGpuBuffer to vllm/v1/utils.py (#23728) Woosuk Kwon 2025-08-27 03:25:00 -07:00
6578e87365 Optimize input preparation for FlashInfer [2/N] (#23174) Woosuk Kwon 2025-08-27 02:52:45 -07:00
5bd9f84158 [Docs] Fix an admonition important (#23726) Michael Yao 2025-08-27 17:50:09 +08:00
91e382c935 [CI/Build] Remove redundant register in model init tests (#23715) Cyrus Leung 2025-08-27 16:11:15 +08:00
6446677839 [XPU]fix cuda event used in XPU model runner (#23708) Kunshang Ji 2025-08-27 15:27:14 +08:00
69244e67e6 [Core] Use key-only cache for BaseMultiModalProcessor (#23018) Cyrus Leung 2025-08-27 14:19:13 +08:00
8dbf6ed7be [Bugfix] fix when config.yaml config value is list parse error (#23528) rongfu.leng 2025-08-27 13:54:39 +08:00
9de25c294b [CI/Build] Remove redundant LoRA model tests (#23706) Jee Jee Li 2025-08-27 13:51:50 +08:00
fce10dbed5 [XPU] Add xpu torch.compile support (#22609) Kunshang Ji 2025-08-27 13:33:27 +08:00
d272415e57 [Quantization] Expand compressed-tensors MoE matching logic to support NFP4 + FP8 MoEs (#22674) Dipika Sikka 2025-08-27 01:00:21 -04:00
142ac08030 [Frontend] Optimize beam search performance by limiting concurrency (#23599) Chen Zhang 2025-08-26 21:59:14 -07:00
3210264421 [Frontend] Add --log-error-stack to print stack trace for error response (#22960) Chen Zhang 2025-08-26 21:58:59 -07:00
644d57d531 [Model] Add Ernie4.5 VL Model Support (#22514) CSWYF3634076 2025-08-27 12:02:55 +08:00
c905684cfe [Core] Asynchronous h2d in merge_multimodal_embeddings via pinned memory. (#23686) Chenheli HuaandRoger Wang 2025-08-26 20:05:34 -07:00
786835807b [Bugfix]: Qwen3 Coder Tool Parser (#23099) Yiheng XuandAaron Pham 2025-08-27 10:58:32 +08:00
fecbb7c782 [Bugfix][gpt-oss] passing the cache config in gpt-oss (#23613) Wei 2025-08-26 19:54:23 -07:00
6dab89b8ec [Docs] Fix math rendering in docs (#23676) Harry Mellor 2025-08-27 02:47:08 +01:00
de02b07db4 [Bugfix] Lazy import gpt_oss_triton_kernels_moe for mxfp4 (#23678) Michael Goin 2025-08-26 21:34:57 -04:00
eb1995167e [gpt-oss] Enable unit test for response API harmony integration (#23533) Chen Zhang 2025-08-26 18:23:26 -07:00
2c2b140ae8 [quantization] use channel scales for w4a8 + misc fixes (#23570) czhu-cohere 2025-08-26 21:23:23 -04:00
c7c80af084 fix pynccl reduce_scatter (#23648) yzdsandhongchao 2025-08-27 09:21:11 +08:00
6891205b16 [Feature][Responses API] Support MCP tool in background mode (#23494) wuhang 2025-08-27 09:06:58 +08:00
b1625dbe9c feat: add triton fused moe config for GLM-4.5-Air-FP8 on B200 (#23695) zixuanzhang226 2025-08-26 18:06:10 -07:00
585e0bde36 [Bugfix] UnboundLocalError when GptOss reasoning specified (#23054) Federico 2025-08-27 02:29:52 +02:00
714872f1a9 [Compile] Fix Cmake Warning (#23689) Wentao Ye 2025-08-26 19:48:32 -04:00
5f1af97f86 [V1] [Hybrid] Enable Full CUDA graph by default for hybrid models in V1 (#22594) Thomas Parnell 2025-08-27 01:28:55 +02:00
c3b0fd1ee6 [V1][P/D]P2pNcclConnector supports flashinfer (#23536) Zhonghua DengandSimon Mo 2025-08-27 06:56:16 +08:00
6421b66bf4 [Docs] Move quant supported hardware table to README (#23663) Harry Mellor 2025-08-26 23:26:46 +01:00
2f13319f47 Enhance the pre-notification policy (#23532) Huzaifa Sidhpurwala 2025-08-27 00:41:36 +04:00
d696f86e7b [doc] Hybrid KV Cache Manager design doc (#22688) Chen ZhangandHarry Mellor 2025-08-26 13:19:05 -07:00
9816b81f5f [Model] Enable video support for InternVL3.5 models (#23658) Isotr0py 2025-08-27 03:46:52 +08:00
c37c0af990 [Misc] Fix comments in tests/kernels/quantization (#23675) Jiangyun Zhu 2025-08-27 03:31:20 +08:00
9715f7bb0f [Bugfix] Fix incorrect original shape in hashing (#23672) Cyrus LeungandLukas Geiger 2025-08-27 03:01:25 +08:00
98aa16ff41 [v1] Add cross-attention KV cache support for encoder-decoder models (#23664) Russell Bryant 2025-08-26 14:49:06 -04:00
227e231b55 [Docs] [V1] [Hybrid] Update docs to remove FlashInfer constraint for hybrid models (#23665) Thomas Parnell 2025-08-26 20:33:16 +02:00
730d0ac8b9 [Docs] Fix warnings in mkdocs build (#23649) Hyogeun Oh (오효근)andHarry Mellor 2025-08-27 03:19:23 +09:00
9b0187003e [Bugfix] Fix cuda event usage with CPU model runner (#23643) Li, Jiang 2025-08-27 01:10:42 +08:00
44ac25eae2 [CI] [Doc]: Add GH Action for auto labeling issues with rocm tag (#20988) vllmellmandCyrus Leung 2025-08-27 00:20:13 +08:00
7ea22e42d5 [Misc] Add override for allreduce fusion thresholds (#23639) nvjullin 2025-08-26 23:53:04 +08:00
9d4183dd2e [model] support qwen2audio embedding input (#23625) Yuekai Zhangandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-08-26 23:48:08 +08:00
513298f1b4 [Bugfix] fix bf16 multimodal model hash (#23623)
Yuekai Zhanggemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Roger Wang Cyrus Leung
2025-08-26 23:47:50 +08:00
379f828fba [Docs] Reduce requirements for docs build (#23651) Harry Mellor 2025-08-26 16:43:28 +01:00
1fdc732419 [ROCm] Starting to add AMD code reviewers for ROCm components (#23496) Hongxia Yang 2025-08-26 10:32:37 -04:00
f58675bfb3 [CPU] add cpu fused moe pytorch native implementation (#23146) TianyuLi0andLi, Jiang <jiang1.li@intel.com> 2025-08-26 22:09:17 +08:00
7c04779afa [Doc]: fix various spelling issues in multiple files (#23636) Didier Durand 2025-08-26 16:05:29 +02:00
f66673a39d [Kernel] Added flashinfer fp8 per-tensor gemms (#22895) nvjullinandMichael Goin 2025-08-26 21:54:04 +08:00
b78bed1bc5 [Hardware][Mac] Fix the installation fail for Apple Silicon (CPU) (#23565) En OuyangandLi, Jiang <jiang1.li@intel.com> 2025-08-26 21:04:25 +08:00
164b2273c8 [Docs] Fix broken links to docs/api/summary.md (#23637) Harry Mellor 2025-08-26 14:00:18 +01:00
2b4fc9bd9b Support FlashAttention Backend for Hybrid SSM Models (#23299) Chen Zhang 2025-08-26 05:41:52 -07:00
ebd5a77bb5 feat: add usage to TranscriptionResponse (text and json response_format) (#23576) Guillaume Calmettes 2025-08-26 14:26:26 +02:00
384dd1b0a8 [Bugfix] Add missing enable_log_outputs parameter to init_app_state function (#23634) Matúš Námešný 2025-08-26 14:13:15 +02:00
fdeb3dac13 [Model] fix DeepSeek e_score_correction_bias dtype to fp32 (#23640) Jee Jee Li 2025-08-26 20:09:47 +08:00
d52358c1e0 [Perf] Remove duplicated NVFP4 blockscales to save memory (#23379) Michael Goin 2025-08-26 07:16:33 -04:00
6ace2f72b0 Fix writing benchmark results with tuple keys (#23633) Huy Do 2025-08-26 04:16:09 -07:00
b00e69f8ca Fix nits from #20059 (#23548) Harry Mellor 2025-08-26 11:27:20 +01:00
50fede6634 [V1] Enable V1 for compute capability < 8.0 + FP32 (#23614) Cyrus Leung 2025-08-26 18:00:18 +08:00
b5d34af328 [Bugfix] Fix scheduling when repeated images in one request (#23544)
Roger Wang Roger Wang knlnguyen1802
2025-08-26 02:46:28 -07:00
9b5f64238f [Bugfix] Fix Qwen25VL packed_modules_mapping (#23604) Jee Jee LiandMichael Goin 2025-08-26 16:09:14 +08:00
ff77764f86 Fix CLI parameter documentation inconsistency in pooling_models.md (#23630) Raghavan 2025-08-26 13:35:37 +05:30
bfc1edc9f5 [Docs] Fix titles for multi-file examples that are rendered in the docs (#23573) Harry Mellor 2025-08-26 08:16:44 +01:00
3ecbb14b81 [Benchmarks] add benchmark for embedding models (#23000) Jiangyun Zhu 2025-08-26 14:57:08 +08:00
7d67a9d9f9 [mypy] Fix incorrect type hint for EAGLE3 support (#23617) Cyrus Leung 2025-08-26 14:50:17 +08:00
959783fb99 [fix] fix seed-oss-parser (#23560) Bin Jia 2025-08-26 14:16:36 +08:00
ce0e9dbd43 [CI/Build] Fix typo in #23561 (#23616) Cyrus Leung 2025-08-26 14:13:03 +08:00
b395b3b0a3 [Disagg][Perf] Use CUDA event sync instead of blocking tolist to avoid unintentional copy ops blocking across different CUDA streams, improving disagg TTIT/TTFT (#22760) Zijing Liu 2025-08-25 21:06:00 -07:00
6fad29b11b Remove graph_pool as member of VllmBackend and argument to CUDAGraphWrapper (#23385)
Copilotcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>ProExpertProg Luka Govedič
2025-08-25 19:34:15 -07:00
6fd45e7b8a [CI/Build] Use vLLM client's user agent to fetch images (#23561) Cyrus Leung 2025-08-26 10:34:12 +08:00
56dcf4e7e9 [Bug] Fix DeepGEMM Env Control (#23591) Wentao Ye 2025-08-25 21:41:21 -04:00
ae067888d6 Update Flashinfer to 0.2.14.post1 (#23537)
weiliang Siyuan Fu Michael Goingemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-08-26 09:30:44 +08:00
906e461ed6 [CI Fix] Pin deepep and pplx tags in tools/ep_kernels/, gate multigpu tests (#23568) Michael Goin 2025-08-25 21:29:00 -04:00
2a97ffc33d [Misc] Add release note draft to PR template (#23598) Simon Mo 2025-08-25 16:44:51 -07:00
efc88cf64a [Misc] Simplify FlashInfer attention metadata (#23585) Woosuk Kwon 2025-08-25 15:42:29 -07:00
7b6a837275 [Docs] Update Documentation of Cohere Command-A Models (#23584) Terrence ZhaoandZhonghua Deng 2025-08-25 17:53:52 -04:00
c34c82b7fe [TPU][Bugfix] Fixes prompt_token_ids error in tpu tests. (#23574) Pate Motter 2025-08-25 14:29:16 -07:00
8a044754bd [XPU] Delay BF16 check to worker init for spawn compatibility (#22979) Chaojun Zhang 2025-08-26 04:09:26 +08:00
9188ae7cb5 [Bugfix][V1][P/D]Fix the issue where repeated requests for the same input produce abnormal outputs for P2pNcclConnector (#23403) Zhonghua Deng 2025-08-26 03:57:08 +08:00
8a3cd90af5 [Kernel] Add fused grouped_topk kernel for MoE (#23274) Xin YangandWentao Ye 2025-08-25 11:47:52 -07:00
2a167b2eeb [test][RL] Add sleep level 2 test and fix reload with sleep mode (#23521) 22quinn 2025-08-25 09:25:52 -07:00
0ff902f3b4 [Refactor] Refactor persistent buffers with CpuGpuBuffer (#23515) Woosuk Kwonandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-08-25 08:44:48 -07:00
a9082a4d14 [Bugfix] Fix Qwen3 MoE GPTQ inference (#23490) Isotr0py 2025-08-25 21:40:20 +08:00
e0329ed4b4 Updates to Flex + VLLm integration (#21416) Driss Guessous 2025-08-25 06:32:42 -07:00
6879cd80ae [Refactor] Pass tokenizer explicitly instead of binding to prompt update (#23542) Cyrus Leung 2025-08-25 21:31:57 +08:00
e269be2ba2 [Doc] Add caution for API server scale-out (#23550) Cyrus Leung 2025-08-25 21:14:15 +08:00
5c4b6e66fe [Attention] Unify mamba and attention backend selection (#23171) Ayush Satyam 2025-08-25 14:39:36 +05:30
d0a4a3f645 [misc] add shanghai meetup (#23535) youkaichao 2025-08-25 17:00:03 +08:00
ebafb0936d [Bugfix] Allow dynamic number of patches for llava_onevision (#23525) Cyrus Leung 2025-08-25 16:34:54 +08:00
0cb7b065c3 Feature/benchmark/random mm data/images (#23119) Breno Baldas Skuk 2025-08-25 10:28:35 +02:00
2da02dd0d8 [Fix] DeepSeek V3.1 tool parser error message (#23492) ZiTian Zhao 2025-08-25 15:56:39 +08:00
d765cf01fe [Core][Multimodal] Track encode cache entries by mm_hash and enable embedding sharing between requests (#22711)
Chenguang Zheng knlnguyen1802 Roger Wang
2025-08-25 15:41:17 +08:00

... 143 144 145 146 147 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86