Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

3bcbb30cbf add add_truncate_prompt_tokens in repr for PoolingParams (#29683) 果冻虾仁 2025-11-29 00:41:05 +08:00
9e6bcda3ac [mypy] Enable type checking for more directories (#29674) Cyrus Leung 2025-11-29 00:39:27 +08:00
9eec282cb5 Guard FlashInfer sampler using the same check as FlashInfer attention backend (#29415) Harry MellorandCyrus Leung 2025-11-28 16:34:48 +00:00
0808eb813b [Misc] Remove yapf directives (#29675) Cyrus Leung 2025-11-28 23:07:23 +08:00
460d8bbf2d Remove upstream fa checks (#29471) Mingyuan MaandRoger Wang 2025-11-28 05:52:42 -08:00
e2f56c309d [CPU] Update torch 2.9.1 for CPU backend (#29664) Li, Jiang 2025-11-28 21:37:54 +08:00
f8151b66fa Revert "Supress verbose logs from model_hosting_container_standards (… (#29335) HappyAmazonianandCyrus Leung 2025-11-28 05:29:05 -08:00
1168768a2d [Optimization] Early return for _apply_matches and _iter_placeholders (#29668) Cyrus Leung 2025-11-28 21:26:47 +08:00
8e7a891602 [BugFix] Fix spec decoding max_tokens scheduling perf issue (#29542) Nick Hill 2025-11-28 04:52:23 -08:00
953d9c820b [mypy] Pass type checking for vllm/utils and vllm/v1/pool (#29666) Cyrus Leung 2025-11-28 20:40:47 +08:00
33b06a6f24 [Misc] Remove redundant attention var constants (#29650) Cyrus Leung 2025-11-28 20:35:19 +08:00
5c2b5cb422 [Docs] Add SPLADE and Ultravox models to supported models documentation (#29659) Wilson WuandCyrus Leung 2025-11-28 18:29:28 +08:00
3cb32e5d6e [Rocm] Set VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS default is disabled (#28985)
杰兮 zhyajie TJian
2025-11-28 18:08:42 +08:00
ccbdf51bd5 [Doc] Reorganize benchmark docs (#29658) Cyrus Leung 2025-11-28 17:19:25 +08:00
5f5521bd5d Fix parameter order in GPT-OSS weight loading function for non-MXFP4 weights (#29506) Filipp FisinandCyrus Leung 2025-11-28 09:45:10 +01:00
b2c1d294fa [BUGFIX] MistralTokenizer._call__ adds an invalid EOS token (#29607)
Julien Denizegemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
2025-11-28 09:44:47 +01:00
cc0f2a0e19 [Doc] Improve abnormal information string (#29655) maang-h 2025-11-28 16:12:20 +08:00
480598958e [Feature][Bench] Add pareto visualization (#29477) rongfu.leng 2025-11-28 15:53:20 +08:00
b34e8775a3 Revert "[CPU]Update CPU PyTorch to 2.9.0 (#29589)" (#29647) Cyrus Leung 2025-11-28 14:43:18 +08:00
f4b76056ee Improve enable chunked_prefill & prefix_caching logic. (#26623) wang.yuqiandCyrus Leung 2025-11-28 14:05:48 +08:00
37b15e97e8 [Multimodal][Speculative Decoding]Eagle3 mm support, enablement on qwen3vl (#29594)
EanWang211123 Louie Tsai Cyrus Leung
2025-11-28 14:05:45 +08:00
c7ba1f6bc7 [BugFix] Fix ValueError in NewRequestData repr methods (#29392) maang-h 2025-11-28 13:42:30 +08:00
18523b87f6 [Docs] Update supported models for Olmo 3 in tool calling documentation (#29411) Wilson Wu 2025-11-28 10:53:55 +08:00
745a3bae1a [LoRA] Support FusedMoE LoRA Triton kernel for mxfp4 (#28971) Xin YangandJee Jee Li 2025-11-27 18:48:28 -08:00
35657bcd7a [CPU]Update CPU PyTorch to 2.9.0 (#29589) scydasandLi, Jiang <jiang1.li@intel.com> 2025-11-28 09:34:33 +08:00
be493e0b3c [BugFix] Fix new nightly failures (#29578) Lucas Wilkinson 2025-11-27 16:45:38 -05:00
ae0ce1be27 [Model Runner V2][BugFix] Keep reference to GPU tensors in AsyncOutput (#29623) Woosuk Kwon 2025-11-27 12:38:53 -08:00
a5345bf49d [BugFix] Fix plan API Mismatch when using latest FlashInfer (#29426) Andrii SkliarandAndrii Skliar 2025-11-27 20:34:59 +01:00
e5a621b724 [CI] Add batched audios Whisper test (#29308) Nicolò Lucchesi 2025-11-27 20:31:52 +01:00
38658ec6f3 [Bugfix][MM encoder] Fix ViT attention backend resolving for Turing GPU (#29614) Isotr0py 2025-11-28 03:17:37 +08:00
a24ea5414b [Deprecation] Advance deprecation status (#29617) Cyrus Leung 2025-11-28 03:04:58 +08:00
ea228b4491 [Misc] Remove unused code from protocol.py (#29616) Cyrus Leung 2025-11-28 02:39:59 +08:00
d45269b378 add skip_reading_prefix_cache in repr for PoolingParams (#29620) 果冻虾仁 2025-11-28 01:21:00 +08:00
ee9841daa9 [Bugfix] Fix doc build on main (#29619) Cyrus Leung 2025-11-28 01:08:08 +08:00
0840abdd24 [BugFix] Optional tokenizer argument when loading GGUF models (#29582)
Injae Ryougemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Isotr0py
2025-11-28 01:53:10 +09:00
e1f262337b Update Transformers pin in CI to 4.57.3 (#29418) Harry Mellor 2025-11-27 16:42:14 +00:00
fc1d8be3dc [Attention] Update attention imports (#29540) Matthew Bonanni 2025-11-27 11:19:09 -05:00
cd007a53b4 [bugfix] avoid NIXL_ERR_REMOTE_DISCONNECT in nixl_connector when Prefill dies (#28120) Mathis Felardos 2025-11-27 16:32:38 +01:00
66d3d5422c [Doc]: fixing typos in diverse files (#29492) Didier Durand 2025-11-27 16:15:50 +01:00
bab438ff3e [CI/Build] Skip ray tests on ROCm (#29556) Ryan Rock 2025-11-27 09:01:37 -06:00
882851dc81 [CI/Build][Bugfix] Fix auto label issues for CPU (#29610) Li, Jiang 2025-11-27 22:51:26 +08:00
2f5f9acd55 [LoRA] Continue optimizing MoE LoRA weight loading (#29322) Jee Jee Li 2025-11-27 21:56:28 +08:00
cf348c8d27 [Bugfix] Fix HunyuanVL XD-RoPE (#29593) Roger Wang 2025-11-27 04:36:24 -08:00
a5abd1d384 [CI] Auto label CPU related issues (#29602) Li, Jiang 2025-11-27 19:33:19 +08:00
e6d4f3c254 [Bugfix] Fix pre-commit (#29601) Cyrus Leung 2025-11-27 18:23:06 +08:00
51906c8c55 [Docs] Improve priority parameter documentation (#29572) maang-handCyrus Leung 2025-11-27 18:09:24 +08:00
0838b52e2e [Frontend][torch.compile] CompilationConfig Overhaul (#20283): Set up -O infrastructure (#26847)
Morrison Turnansky adabeyta Luka Govedič Harry Mellor
2025-11-27 04:55:58 -05:00
00d3310d2d [Bugfix] Update Ultravox compatibility (#29588) Cyrus Leung 2025-11-27 17:36:18 +08:00
da3222f371 [Model Runner V2] Implement multi-step Eagle with CUDA graph (#29559) Woosuk Kwon 2025-11-27 00:09:41 -08:00
43c5792592 [ROCm][CI] Fix test_cpu_offloading for ROCm (#29548) Micah Williamson 2025-11-27 01:54:44 -06:00
3ecabd06ee Fix tpu-inference platform path (#29554) Johnny Yang 2025-11-26 23:25:21 -08:00
c069086b9c [Bugfix] Fix getting device for MoE LoRA (#29475) Jee Jee Li 2025-11-27 15:16:07 +08:00
11ea5ec1ff [Model Runner V2] Refactor CudaGraphManager (#29583) Woosuk Kwon 2025-11-26 21:37:59 -08:00
ecb1952378 [cpu][fix] Fix Arm CI tests (#29552) Fadi Arafeh 2025-11-27 05:09:41 +00:00
da8e1a1bf9 [DOC] Add vLLM Bangkok Meetup info (#29561) TJian 2025-11-27 12:42:50 +08:00
ee80aee1ca [Model Runner V2] Minor cleanup for build_attn_metadata (#29576) Woosuk Kwon 2025-11-26 20:10:12 -08:00
0aeb698b77 [Model Runner V2] Minor code cleanup (#29570) Woosuk Kwon 2025-11-26 19:47:17 -08:00
9bb33c8919 add xpu supported model and model id for cpu (#29380) Louie Tsai 2025-11-26 19:30:50 -08:00
a67dec7cba [Bugfix] fix IMA issue in certain cases of the moe marlin kernel (#28619)
Jinzhen Lin youkaichao Michael Goin Wentao Ye
2025-11-27 11:02:21 +08:00
77740191de [Attention][Async] Eliminate seq_lens_cpu in FlashAttention metadata building with DCP > 1 (#29449) Matthew Bonanni 2025-11-26 21:48:43 -05:00
df01eda4dc [Bugfix] Make compressed-tensors MoEs respect ignored layers (#28878) HDCharles 2025-11-26 21:35:13 -05:00
ba1fcd84a7 [TPU] add tpu_inference (#27277) Johnny Yang 2025-11-26 14:46:36 -08:00
56539cddac [Core] Refactor padding logic and pad for CUDA graphs before attention metadata building (#28579) Lucas Wilkinson 2025-11-26 14:07:13 -05:00
430dd4d9eb [Attention] Remove imports from vllm/attention/__init__.py (#29342) Matthew Bonanni 2025-11-26 12:53:15 -05:00
c4c0354eec [CI/Build] allow user modify pplx and deepep ref by ENV or command line (#29131) Alec 2025-11-26 09:41:16 -08:00
e603129505 [refactor] CTConfig methods to static/class methods (#28870) HDCharlesandIsotr0py 2025-11-26 12:21:58 -05:00
0b0aa874e8 [Perf] Optimize batch invariant BMM, 18.1% Throughput improvement, 10.7% TTFT improvement (#29345) Wentao Yeandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-11-26 11:38:52 -05:00
70d5953f82 Revert "[Bugfix] Fix GPT-OSS AR+NORM fusion (#28841)" (#29483) Huamin Li 2025-11-26 06:27:26 -08:00
3650a74ed8 Optimize the wording of the document and unify the terminology and th… (#29491) yxt 2025-11-26 21:16:12 +08:00
bb706d6048 Fix TeleChatForCausalLM not register issue (#29473) Yejing Lai 2025-11-26 21:15:00 +08:00
e30859dff3 [Bugfix] Fix handling of image embeds in models (#29480) Cyrus Leung 2025-11-26 21:00:15 +08:00
452a7c9f7c [Misc] Allow LM only loading for Pixtral (#29451) Roger Wang 2025-11-26 05:00:00 -08:00
d9d342d214 [Performance][MLA][ROCm] Remove redundant D2D copy in deepseek (#27457) Pleaplusone 2025-11-26 12:45:28 +08:00
53d7f1f601 [Kernel] Use pre-allocated output buffer for triton kernel fused_experts (#29219) Xin Yang 2025-11-25 18:21:00 -08:00
c5ee430328 Bump actions/checkout from 4 to 6 (#29293) dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 2025-11-26 01:57:08 +00:00
8d6a89dffd [UX] Suppress gloo log spam (#29250) Michael Goin 2025-11-25 20:19:35 -05:00
56531b79cc [Misc] Add backup hash algorithm for FIPS constrained environments (#28795) George D. TorresandRussell Bryant 2025-11-25 18:50:22 -06:00
12866af748 dummy run corner case (#29433) Xieyang Xu 2025-11-25 16:20:35 -08:00
d8819c88eb fix assertion for single world use case (uni) (#29429) Lucia FangandLucia Fang 2025-11-25 16:14:23 -08:00
de75b0bb70 [BugFix] Fix initialization of draft model. (#29319) Andrey KhalyavinandTyler Michael Smith 2025-11-26 02:45:58 +03:00
7df0289782 Change warning logs to debug for unimplemented MXFP4 Linear/Attention (#29441) Michael GoinandWentao Ye 2025-11-25 17:52:31 -05:00
0abc79482a [caching] Add enable_prompt_embeds and cpu_offload_gb to compile hashes. (#29435) Zhengxu Chen 2025-11-25 16:46:41 -05:00
4e57c6587f [Core] Support logprobs with spec decode + async scheduling (#29223) Nick Hill 2025-11-25 12:55:24 -08:00
e7d776273d [Compile] Refactor. Move PostGradPassManager out of Compilation config (#29340) Ilya Markov 2025-11-25 20:58:56 +01:00
c32a18cbe7 Attempt to fix GPU OOM in a spec-decoding test (#29419) Eldar Kurtić 2025-11-25 20:23:36 +01:00
b07555d26f [responsesAPI][2] parse ResponseFunctionToolCallOutputItem (#29383) Andrew XiaandAndrew Xia 2025-11-25 10:27:26 -08:00
0353d2e162 Fix RoPE related failures in Transformers nightly tests (#29333) Harry Mellor 2025-11-25 16:23:45 +00:00
a1f2676879 Scheduled removal of override_pooler_config and disable_log_requests (#29402) Harry Mellorandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-11-25 16:08:57 +00:00
48ddb02b79 [Hybrid Allocator] Support KV cache groups with different block_size (#29143) Yifan QiaoandChen Zhang 2025-11-25 07:30:57 -08:00
e502098643 [Kernel] Add NVFP4 MoE CUTLASS support for SM120 (#29242) Michael Goin 2025-11-25 09:59:07 -05:00
dbc3d9991a [UX] Put CUDA attention backend selection log into one line (#29337) Michael Goin 2025-11-25 09:46:18 -05:00
794029f012 [Feature]: Improve GGUF loading from HuggingFace user experience like repo_id:quant_type (#29137) Injae RyouandIsotr0py 2025-11-25 23:28:53 +09:00
0231ce836a Revert back to torch.equal over torch.allclose from #28819 (#29086) Eldar Kurtić 2025-11-25 15:23:38 +01:00
516c3f7847 [Bugfix] Fix logic for choosing default prefix caching setting (#29393) Thomas Parnell 2025-11-25 15:05:10 +01:00
51fc9e017a Scheduled removal of CompilationConfig.use_inductor (#29323) Harry Mellor 2025-11-25 12:55:42 +00:00
bf0c75cd4f Make Transformers Nightly tests soft-fail and enable all tests (#29401) Harry Mellor 2025-11-25 12:41:15 +00:00
c2c661af9b [Bugfix] Fix overallocation in MM profiling (#29386) Roger Wang 2025-11-25 04:38:36 -08:00
798e87db5c [Core] Generalize Encoder-Decoder seq_lens computation to avoid Whisper hardcoded logic (#29268) Nicolò LucchesiandEkagra Ranjan 2025-11-25 12:32:11 +01:00
de6889946b [Misc] Suppress log outputs when constructing the default vllm config. (#29291)
wang.yuqigemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung Harry Mellor
2025-11-25 19:00:44 +08:00
7a80b01889 [CI] Resettle pooling entrypoints tests. (#29370) wang.yuqi 2025-11-25 18:39:10 +08:00

... 115 116 117 118 119 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86