Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

60d17251c9 [Disagg] Support large batch size in proxy server and update NixlConnector doc for DP (#28782) Ming Yang 2025-12-08 16:01:08 -08:00
1fb632fdb6 [Perf] Improve fp8 quant in mla; replace ReduceSum with ReduceScatterSum (#29795) Lain 2025-12-08 15:02:34 -08:00
6af70e11a0 [ROCm][CI] Fix test_max_len.py for Rocm (#29916) Charlie Fu 2025-12-08 15:58:30 -06:00
ae0f69b16a Add SpecDec support to selective_state_update (#29488) roikoren755 2025-12-08 23:45:18 +02:00
799804d140 Bump nvshmem to 3.3.24 and fix CUDA 13 installation (#30149) Dmitry Tokarevandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-12-08 15:24:34 -05:00
0d402d2600 online fp8 quant with streaming weight post-processing (#29196) Vasiliy Kuznetsov 2025-12-08 15:15:10 -05:00
d1b5e7afbf [TPU] Bump tpu-inference to 0.12.0 (#30221) Johnny Yang 2025-12-08 12:10:10 -08:00
fcd5306f65 Add latent MoE support (#30203) shaharmor98andTyler Michael Smith 2025-12-08 19:35:01 +02:00
398a596ed2 [MP executor] fix get device count for multi node of mp executor feature (#30042) weiguihua2 2025-12-09 01:33:48 +08:00
67312cad11 [Misc] Split the LoRA code (#30253) Jee Jee Li 2025-12-09 00:59:31 +08:00
87aee9ed2b Add evaluate_guards option to DynamicShapesConfig (#27432) Laith Sakka 2025-12-08 07:46:15 -08:00
184076c3fe [DeepSeek v3.2] Make top-k work for any logit values. (#27568)
Daniel Cámpora Michael Goin Cyrus Leung
2025-12-08 15:55:58 +01:00
eb1051fb95 [ROCm] Guard group quant RMS norm fusion patterns (#30239) Ye (Charlotte) Qi 2025-12-08 06:44:48 -08:00
80433e225e [LoRA] Reduce the loading time of MoE LoRA (#30243) Jee Jee Li 2025-12-08 21:29:47 +08:00
5c2433a6f3 Add tip for mypy and markdownlint to the pre-commit comment (#30259) Harry Mellor 2025-12-08 13:11:51 +00:00
77072e93b3 [docs] governance documents (#24801)
Simon Mo Mark McLoughlin Russell Bryant Michael Goin Wentao Ye Luka Govedič Cyrus Leung
2025-12-08 03:06:20 -09:00
2e660c2434 [Frontend] Binary embedding response does not return metadata by setting encoding_format to bytes_only. (#30249) wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-12-08 20:01:21 +08:00
408cf42f67 [CI] Prevents triggering of an inactive issue/PR check for forked repository. (#29654) Shiming Zhang 2025-12-08 18:29:14 +08:00
9e77ffca3f [Model][7/N] Improve all pooling task | Deprecation as_reward_model. Extract hidden states prefer using new multi-vector retrieval API (#26686) wang.yuqi 2025-12-08 16:10:09 +08:00
bcb6f5947f [Perf] Remove sync point in vit torch sdpa attn backend (#30232) Dazhi Jiang 2025-12-08 15:12:42 +08:00
cd00c443d2 [Misc] Rename TensorRT Model Optimizer to Model Optimizer (#30091) Zhiyu 2025-12-07 23:05:27 -08:00
d143271234 [Bugfix] fix fuse_allreduce_rms when tp =1 (#30178) Jiangyun Zhu 2025-12-08 14:43:47 +08:00
c6df05ebb4 [ROCm] [Fused Moe EP] Use binary expert mask for aiter fused moe kernel (#29773) Zhiwei 2025-12-08 13:23:46 +08:00
d726a7b0ed [BugFix] Unblock use of LoRA with data parallel mode (#30220) Nick Hill 2025-12-07 20:21:05 -08:00
344b50d525 Address comment to mergify.yml in #30117 (#30219) Zhijian Jiang 2025-12-07 19:26:25 -08:00
735284ed86 [responsesAPI][7] Browser, Container MCP tools for non harmony models (#29989)
Andrew Xia Andrew Xia Cyrus Leung
2025-12-07 18:04:03 -08:00
444f0e3f33 [Frontend] Add MCP type support infrastructure to Responses API (#30054) daniel-salib 2025-12-07 18:02:52 -08:00
af0444bf40 [Performance] Fused blockwise quant RMS norm (#27883) ElizaWszolaandyewentao256 2025-12-07 17:38:04 +01:00
0044c4038c [BugFix][DeepSeek-V3.2] Fix backend selection logic for Blackwell (#30195) Lucas Wilkinson 2025-12-07 10:53:51 -05:00
b952f4d3c3 [v1] Add PrefixLM support to FlexAttention backend (#27938) Isotr0py 2025-12-07 23:51:36 +08:00
541a2ef892 [Perf] Deepgemm fused layout kernel for activations, 4.3% throughput improvement, 10.7% TTFT improvement. (#29546) Wentao Ye 2025-12-07 07:31:14 -05:00
b0f4866a77 [CI/Build]Temporary workaround for test_default_mm_loras timeout (#30202) Jee Jee Li 2025-12-07 20:27:11 +08:00
879ddb09c3 [Kernel][MoE] optimize moe_align_block_size (#29642) Jinzhen LinandJee Jee Li 2025-12-07 17:58:47 +08:00
1b0482b9d1 [Misc][Core] Remove unused req_index increment in scheduler (#30176) Yifan Qiao 2025-12-07 00:39:21 -08:00
e83b7e379c Revert "[Renderer] Separate out RendererConfig from ModelConfig (#30145)" (#30199) Cyrus Leung 2025-12-07 16:00:22 +08:00
27f4c2fd46 [Renderer] Separate out RendererConfig from ModelConfig (#30145) Cyrus Leung 2025-12-07 15:15:42 +08:00
a49d813fa8 Lazy loading to avoid importing all files (#29716) Luke 2025-12-06 23:13:14 -08:00
17eb25e327 [Perf] Enable cuda graph for deepepHT, 5.3% throughput improvement, 4.4% TTFT improvement (#29558) Wentao Ye 2025-12-06 23:44:50 -05:00
dce6d229f7 Support multiple image/audio embeddings per requests (#29988) jeremyteboulandJeremy Teboul 2025-12-06 20:34:24 -08:00
cbedb703cc [Frontend] Remove confusing -O.xx flag error (#30169) Yanan Cao 2025-12-06 18:53:42 -08:00
8d3da4c79d [MISC]: change NIXL compatibility hash logging level to debug (#30182) AuruTus 2025-12-07 08:21:03 +08:00
421125d03a [ez] move harmony utils to parser folder (#30117) Andrew XiaandAndrew Xia 2025-12-06 14:34:34 -08:00
671427efbf [Model] Move multimodal_cpu_fields definition to field config (#30181) Cyrus Leung 2025-12-06 21:40:02 +08:00
21bb323542 Gigachat 3 tool parser and tests (#29905) Viacheslav 2025-12-06 15:04:14 +03:00
17a9abec2b simplify requires_files list creation (#29656) Chukwuma Nwaugha 2025-12-06 09:42:41 +00:00
92c35abb24 [Misc] Fix circular import in vllm.transformers_utils.config (#30179) Ye (Charlotte) Qi 2025-12-06 01:24:03 -08:00
43e7593031 Support tokenization_kwargs override (#29794) Yu Jiaqi 2025-12-06 17:12:53 +08:00
c46b932df2 [Chore] Deprecate SupportsMultiModal.merge_by_field_config (#30170) Cyrus Leung 2025-12-06 15:57:28 +08:00
6476382384 prefix caching design doc sha256 now default (#29261) redwrasse 2025-12-05 23:39:56 -08:00
d6aeaddf4a [bugfix] fix type[AttentionBackend] bug in kv_connector_base_v1 (#30051) kxand01267596 2025-12-06 15:11:31 +08:00
a238cbd89d [Model Runner V2] Support min-p sampling (#30171) Woosuk Kwon 2025-12-05 21:42:47 -08:00
4026ae31e9 [Misc] Move disable_nccl_for_dp_synchronization init logic into VllmConfig (#30161) Nick Hill 2025-12-05 20:59:04 -08:00
b12f4a9830 [CI/Build][AMD] Use ROCM_ATTN instead of FLASH_ATTN test for test_register_kv_caches for ROCm and update test for TRITON_ATTN (#29985)
rasmith Randall Smith TJian
2025-12-05 22:57:38 -06:00
40a046cd82 [Bugfix]: Fix TokenizerLike interface (#30009) Rohan Potdar 2025-12-05 22:56:40 -06:00
e858bc4d14 [Model] Add support for transformer-based Ultravox v0.7 projector (#30089) Peter Salas 2025-12-05 20:55:43 -08:00
e3fbb6f152 fix#30092 Kimi-Linear model loading failure with missing indexer_rotary_emb (#30093) Dongjie Zou 2025-12-05 23:55:09 -05:00
c4d62618ca Fix AWQ MoE marlin check issue in marlin_utils.py for AMD backend (#30102) yuttian1 2025-12-06 12:54:38 +08:00
62079d8600 [CI/Build][AMD] Skip marlin, machete, and hadacore tests since these require _C functions not defined for ROCm (#30109) rasmithandRandall Smith 2025-12-05 22:54:17 -06:00
bf4a901af9 Better error when world size is larger than node and distributed_executor_backend is not set (#30140) Harry Mellor 2025-12-06 04:53:52 +00:00
7e31c3a3f6 [CI]: Remove unnecessary imports from test_lmache_integration (#30157) Samuel ShenandSamuel Shen 2025-12-05 20:53:34 -08:00
dc839ad03d [CI/Build][AMD][Quantization] Fix test_int8_kernel.py by updating int8_utils to use hip.libdevice.round (#30151) rasmithandRandall Smith 2025-12-05 22:52:11 -06:00
02a4169193 [Tests] Tool call tests for openai/gpt-oss-20b (#26237) Deboleina 2025-12-05 22:03:29 -05:00
7b5575fa7d [Bug] Fix vLLM config is not set error (#29999) Wentao Ye 2025-12-05 16:42:12 -05:00
77e4472809 let draft model follow target model's config_format (#30152) Bangsheng Tang 2025-12-05 13:33:42 -08:00
962d703818 [Bugfix][llama4_eagle] Fix missing 'lm_head' attribute (#29926) Divakar Verma 2025-12-05 13:57:26 -06:00
e23ca3a0e8 [CI] Re-use whisper_client for all tests (#30148) Nicolò Lucchesi 2025-12-05 20:47:37 +01:00
3633035a3f [Misc] Rename CohereForAI references to CohereLabs (#30147) Russell Bryant 2025-12-05 14:41:40 -05:00
bff78310d9 [Enc-Dec] Fix OOT tokenizer issue (#30144) Nicolò Lucchesi 2025-12-05 20:23:33 +01:00
adb315060c [KVConnector][Feature] Support KV connector cache reset via /reset_prefix_cache (#27170) Tova Movshovitzandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-12-05 20:33:26 +02:00
4e26d3b09e [Compile] Conditional compilation. Introduce compile_ranges (#24252)
Ilya Markov Luka Govedič Luka Govedič Robert Shaw Luka Govedič
2025-12-05 19:17:32 +01:00
66e674cdd5 [Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments (#26315)
Matthew Bonannigemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Harry Mellor Lucas Wilkinson
2025-12-05 12:48:43 -05:00
dff0a2b394 [NIXL] Add remote_request_id to kv_transfer_params (#29665) Mark McLoughlin 2025-12-05 17:43:48 +00:00
dc264bcea1 [BugFix] Eagerly abort cancelled final-step requests (#29987) Nick Hill 2025-12-05 09:28:32 -08:00
78c44fd722 [NIXL] Small cleanup of unused variables (#29618) Nicolò LucchesiandCyrus Leung 2025-12-05 18:17:36 +01:00
e7296b08da [bugfix] Pass globals to aot_compiled function (#29428) Angela Yi 2025-12-05 08:54:26 -08:00
da7bc54ea8 [responsesAPI][5] ResponsesParser with tools for full MCP python loop (#29798) Andrew XiaandAndrew Xia 2025-12-05 08:11:50 -08:00
949a6a19d2 [NIXL] Add compatibility checking to NIXL KV connector handshake (#29503) Mark McLoughlin 2025-12-05 14:52:45 +00:00
2c174420f5 Reduce validation to a warning (#28749)
Alec S Alec Solder Harry Mellor
2025-12-05 09:02:49 -05:00
0d8a7d8a26 [Compressed Tensors] Add XPU wNa16 support (#29484) Yi Liu 2025-12-05 22:02:09 +08:00
9843e332da [CPU][Perf] Add fast vectorized exp impl from Arm Optimized Routines (#30068) ElhamandUbuntu 2025-12-05 08:09:20 -05:00
b7d85cf25c [CI] Have pre-commit comment on a PR if pre-commit was not used (#30077) Harry Mellor 2025-12-05 13:03:45 +00:00
c2894d3883 [Feature] Add Layer-wise NVTX Support (#29990) Max HuandMax Hu 2025-12-05 06:20:07 -05:00
3628bcaaf2 [ROCm][MXFP4] Infer w4a4 quant method in rocm aiter fused moe (#29775) Zhiwei 2025-12-05 19:01:16 +08:00
b73b158ab0 [Bugfix] Fix parse_output_message crash on commentary with no recipient (#29972) strinczer 2025-12-05 10:51:12 +00:00
7ae13c66ba [typing] fix type (#29964) Ning Xie 2025-12-05 18:46:08 +08:00
f16356fe36 [bench] Support common prefix len config (for decode-only bench) (#29934) Ming Yang 2025-12-05 02:26:52 -08:00
65ee97288a [BugFix] Adding env variable to disable async grammar compilation (#29996)
Alec S Alec Soldergemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>22quinn
2025-12-05 03:49:37 -05:00
62b3333448 [Frontend] Remove deprecated -O.xx flag (#29991) Yanan Cao 2025-12-05 00:47:22 -08:00
feecba09af [CI/Build][AMD] Use float16 in test_reset_prefix_cache_e2e to avoid accuracy issues (#29997) rasmithandRandall Smith 2025-12-05 02:42:25 -06:00
6038b1b04b [Frontend][Model] Add 'float16' to possible mamba cache dtype values, override mamba SSM cache dtype value for NemotronH (#29978) amitz-nv 2025-12-05 10:34:33 +02:00
60a66ea2dc [DOC]: Add kthena to integrations (#29931) Tiger Xu / Zhonghu Xu 2025-12-05 16:11:03 +08:00
06579f9a82 [AMD][CI] Add ray[default] Dependency On ROCm To Pass v1/metrics/test_engine_logger_apis.py (#30110) Micah Williamson 2025-12-05 00:48:23 -06:00
6e865b6a83 Refactor example prompts fixture (#29854) Chukwuma Nwaugha 2025-12-05 06:44:32 +00:00
d698bb382d [Bugfix] Correct num_q_heads on DCP for Flashinfer backends (#29487) Jingchun GaoandJingchun Gao 2025-12-05 13:54:31 +08:00
2c22c4ca2d [ROCm][CI] Increase the memory threshold for test_deep_sleep_fp8_kvcache (#30104) Charlie Fu 2025-12-04 22:51:44 -06:00
5867819eaf Do not guard during noop elimination pass (#30095) Laith Sakka 2025-12-04 20:10:12 -08:00
7c9b2c8f81 [ROCm][CI] Add jiwer dependency for testing (#30081) Charlie Fu 2025-12-04 21:34:51 -06:00
0098a6e3da [PCP&DCP] move CUDAGraph check for PCP&DCP to the check func of platforms (#29952) QiuandCyrus Leung 2025-12-05 10:40:51 +08:00
befb59e5b1 [Model] Add Holo2 reasoning parser (#30048) Hubert de La Jonquiere 2025-12-05 03:38:45 +01:00
aaddc9c82a [CI] fix silent error in nightly wheel index generation script, add generation time to HTML index (#30060) Shengqi Chen 2025-12-05 08:48:59 +08:00

... 112 113 114 115 116 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86