Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

48cb2109b6 [V1] Move usage stats to worker and start logging TPU hardware (#16211) Daniel Li 2025-04-25 13:06:01 -07:00
a5450f11c9 [Security] Use safe serialization and fix zmq setup for mooncake pipe (#17192) Russell BryantandShangming Cai 2025-04-25 12:53:23 -04:00
9d98ab5ec6 [Misc] Inline Molmo requirements (#17190) Cyrus Leung 2025-04-26 00:41:44 +08:00
df5c879527 [doc] update wrong hf model links (#17184) Reidandreidliu41 2025-04-26 00:40:54 +08:00
423e9f1cbe Use Transformers helper get_text_config() instead of checking for text_config (#17105) Harry Mellor 2025-04-25 16:47:35 +01:00
0bd7f8fca5 Bump Transformers to 4.51.3 (#17116) Harry Mellor 2025-04-25 16:34:34 +01:00
d5615af9ae [Bugfix] Fix Mistral ChatCompletionRequest Body Exception (#16769) Jasmond LandCyrus Leung 2025-04-25 22:26:30 +08:00
19dcc02a72 [Bugfix] Fix mistral model tests (#17181) Cyrus Leung 2025-04-25 21:03:34 +08:00
7feae92c1f [Doc] Move todo out of beam search docstring (#17183) Alex Brooks 2025-04-25 05:44:58 -06:00
f851b84266 [Doc] Add two links to disagg_prefill.md (#17168) Michael Yao 2025-04-25 18:23:57 +08:00
fc966e9cc6 Only turn on FastIncrementalDetokenizer when tokenizers >= 0.21.1 (#17158) Lu Fang 2025-04-25 02:10:32 -07:00
ef19e67d2c [Doc] Add headings to improve gptqmodel.md (#17164) Michael Yao 2025-04-25 16:13:13 +08:00
a41351f363 [Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734) rasmithandLuka Govedič 2025-04-25 02:45:02 -05:00
6aae216b4e [Bugfix] remove fallback in guided_json (int range, patterns) (#16725) Sangyeon Choand조상연[플레이스 AI] <sang-yeon.cho@navercorp.com> 2025-04-25 15:54:43 +09:00
b22980a1dc [Perf]Optimize rotary_emb implementation to use Triton operator for improved inference performance (#16457) yexin(叶鑫)andMagnetoWang 2025-04-25 14:52:28 +08:00
881f735827 [Misc] Benchmark Serving Script Support Appending Results (#17028) Lucas Wilkinson 2025-04-25 01:53:55 -04:00
2f54045508 [Bugfix][Misc] Use TritonPlaceholderModule to defensively import triton (#15099) Mengqing Cao 2025-04-25 13:51:02 +08:00
5aa6efb9a5 [Misc] Clean up redundant code in uniproc_executor.py (#16762) Lifu Huang 2025-04-24 22:49:30 -07:00
6ca0234478 Move missed SchedulerConfig args into scheduler config group in EngineArgs (#17131) Harry Mellor 2025-04-25 06:48:53 +01:00
649818995f [Docs] Fix True->true in supported_models.md (#17141) Michael Goin 2025-04-24 22:20:04 -06:00
7a0a9da72b [Doc] V1 : Update LoRA status (#17133) Varun Sundar Rabindranathandvarun sundar rabindranath 2025-04-24 23:17:22 -04:00
69bff9bc89 fix float16 support for kimi-vl (#17156) Zaida Zhouandzhouzaida 2025-04-25 11:16:32 +08:00
41ca7eb491 [Attention] FA3 decode perf improvement - single mma warp group support for head dim 128 (#16864) Lucas Wilkinson 2025-04-24 23:12:21 -04:00
eef364723c [FEAT] [ROCm]: AITER Fused MOE V1 Support (#16752) vllmellmandtjtanaa 2025-04-25 11:06:50 +08:00
0d6e187e88 Use custom address for listening socket (#15988) jglaser 2025-04-24 21:57:16 -04:00
9420a1fc30 Better error message for missing mistral params.json (#17132) Michael Goin 2025-04-24 17:43:08 -06:00
583e900996 [Misc] Add example to run DeepSeek with Ray Serve LLM (#17134) Rui Qiao 2025-04-24 15:25:21 -07:00
05e1fbfc52 Add chat template for Llama 4 models (#16428) Maximilien de Bayser 2025-04-24 17:19:36 -03:00
fe92176321 Add collective_rpc to llm engine (#16999) Yinghai Lu 2025-04-24 13:16:52 -07:00
6d0df0ebeb [Docs] Generate correct github links for decorated functions (#17125) Russell Bryant 2025-04-24 13:39:43 -04:00
0fa939e2d1 Improve configs - LoRAConfig + PromptAdapterConfig (#16980) Harry Mellor 2025-04-24 18:29:34 +01:00
0422ce109f Add :markdownhelp: to EngineArgs docs so markdown docstrings render properly (#17124) Harry Mellor 2025-04-24 18:28:45 +01:00
47bdee409c Molmo Requirements (#17026) Eyshika Agarwal 2025-04-24 12:08:37 -05:00
49f189439d existing torch installation pip command fix for docs (#17059) Atilla 2025-04-24 20:07:21 +03:00
5adf6f6b7f Updating builkite job for IBM Power (#17111) Aaruni Aggarwal 2025-04-24 22:36:17 +05:30
4115f19958 [CI] Add automation for the tool-calling github label (#17118) Russell Bryant 2025-04-24 12:22:00 -04:00
340d7b1b21 [V1][Spec Decoding] Add num_drafts and num_accepted_tokens_per_position metrics (#16665) Mark McLoughlin 2025-04-24 16:57:40 +01:00
1bcbcbf574 [Misc] refactor example series - structured outputs (#17040) Reidandreidliu41 2025-04-24 22:49:48 +08:00
82e43b2d7e Add missing rocm_skinny_gemms kernel test to CI (#17060) Michael Goin 2025-04-24 08:49:37 -06:00
67309a1cb5 [Frontend] Using matryoshka_dimensions control the allowed output dimensions. (#16970) wang.yuqi 2025-04-24 22:06:28 +08:00
b724afe343 [V1][Structured Output] Clear xgrammar compiler object when engine core shut down to avoid nanobind leaked warning (#16954) Shanshan Shen 2025-04-24 21:15:03 +08:00
21f4f1c9a4 Improve static type checking in LoRAModelRunnerMixin (#17104) Harry Mellor 2025-04-24 14:14:47 +01:00
b0c1f6202d [Misc] Remove OLMo2 config copy (#17066) Isotr0py 2025-04-24 21:14:32 +08:00
c0dfd97519 [V1][PP] Optimization: continue scheduling prefill chunks (#17080) Rui Qiao 2025-04-24 05:27:08 -07:00
a9138e85b1 Fix OOT registration test (#17099) Harry Mellor 2025-04-24 12:44:12 +01:00
0a05ed57e6 Simplify TokenizerGroup (#16790) Harry Mellor 2025-04-24 12:43:56 +01:00
14288d1332 Disable enforce_eager for V1 TPU sampler and structured output tests (#17016) Michael Goin 2025-04-24 03:50:09 -06:00
b411418ff0 [Chore] Remove Sampler from Model Code (#17084) Woosuk Kwon 2025-04-24 02:49:33 -07:00
2bc0f72ae5 Add docs for runai_streamer_sharded (#17093) omer-dayanandCyrus Leung 2025-04-24 11:03:21 +03:00
9c1244de57 [doc] update to hyperlink (#17096) Reidandreidliu41 2025-04-24 15:58:08 +08:00
db2f8d915c [V1] Update structured output (#16812) Reidandreidliu41 2025-04-24 14:57:17 +08:00
6167c0e5d2 [Bugfix][Core] add seq_id_to_seq_group clearing to avoid memory leak when s… (#16472) 张宇and开哲 2025-04-24 11:25:37 +08:00
ed2e464653 Addendum Fix to support FIPS enabled machines with MD5 hashing (#17043) Areeb Syed 2025-04-24 08:25:00 +05:30
2c8ed8ee48 More informative error when using Transformers backend (#16988) Harry Mellor 2025-04-24 03:54:03 +01:00
ed50f46641 [Bugfix] Enable V1 usage stats (#16986) Michael GoinandNick Hill 2025-04-23 20:54:00 -06:00
46e678bcff [Minor] Use larger batch sizes for A100/B100/B200/MI300x (#17073) Woosuk Kwon 2025-04-23 19:18:59 -07:00
6b2427f995 [Quantization]add prefix for commandA quantized model (#17017) Chen Xia 2025-04-23 17:32:40 -07:00
b07d741661 [CI/Build] workaround for CI build failure (#17070) Sangyeon ChoandMichael Goin 2025-04-24 08:14:18 +09:00
41fb013d29 [V1][Spec Decode] Always use argmax for sampling draft tokens (#16899) Woosuk Kwon 2025-04-23 14:57:43 -07:00
32d4b669d0 [BugFix][V1] Fix int32 token index overflow when preparing input ids (#16806) Yong Hoon Shin 2025-04-23 12:12:35 -07:00
3cde34a4a4 [Frontend] Support guidance:no-additional-properties for compatibility with xgrammar (#15949) Travis Johnson 2025-04-23 12:34:41 -06:00
bdb3660312 Use @property and private field for data_parallel_rank_local (#17053) Harry Mellor 2025-04-23 16:50:08 +01:00
f3a21e9c68 CacheConfig.block_size should always be int when used (#17052) Harry Mellor 2025-04-23 16:50:05 +01:00
8e630d680e Improve Transformers backend model loading QoL (#17039) Harry Mellor 2025-04-23 15:33:51 +01:00
af869f6dff [CI] Update structured-output label automation (#17055) Russell Bryant 2025-04-23 10:33:14 -04:00
53c0fa1e25 Ensure that pid passed to kill_process_tree is int for mypy (#17051) Harry Mellor 2025-04-23 15:32:26 +01:00
f7912cba3d [Doc] Add top anchor and a note to quantization/bitblas.md (#17042) Michael Yao 2025-04-23 22:32:16 +08:00
6317a5174a Categorize tests/kernels/ based on kernel type (#16799) Michael Goin 2025-04-23 07:21:07 -06:00
aa72d9a4ea Mistral-format support for compressed-tensors (#16803) Michael Goin 2025-04-23 06:46:23 -06:00
ce17db8085 [CI] Run v1/test_serial_utils.py in CI (#16996) Russell Bryant 2025-04-23 04:13:34 -04:00
8c87a9ad46 [Bugfix] Fix AssertionError: skip_special_tokens=False is not supported for Mistral tokenizers (#16964) Chauncey 2025-04-23 15:24:09 +08:00
ec69124eb4 [Misc] Improve readability of get_open_port function. (#17024) huafeng 2025-04-23 14:16:53 +08:00
d0da99fb70 [BugFix] llama4 fa3 fix - RuntimeError: scheduler_metadata must have shape (metadata_size) (#16998) Lucas Wilkinson 2025-04-23 00:49:24 -04:00
b2f195c429 [V1] Avoid socket errors during shutdown when requests are in in-flight (#16807) Nick Hill 2025-04-22 21:36:29 -07:00
047797ef90 [Bugfix] Triton FA function takes no keyword arguments (#16902) vllmellm 2025-04-23 12:35:24 +08:00
eb8ef4224d [doc] add download path tips (#17013) Reidandreidliu41 2025-04-23 12:06:30 +08:00
56a735261c [INTEL-HPU][v0] Port delayed sampling to upstream (#16949) Chendi.XueandMichal Adamczyk 2025-04-22 22:14:11 -05:00
e1cf90e099 [misc] tune some env vars for GB200 (#16992) youkaichao 2025-04-23 10:59:48 +08:00
6bc1e30ef9 Revert "[Misc] Add S3 environment variables for better support of MinIO." (#17021) Chauncey 2025-04-23 10:22:29 +08:00
7e081ba7ca [BugFix] Revert ROCm Custom Paged Attention Env Flag Check (#17022) vllmellm 2025-04-23 10:17:48 +08:00
1e013fa388 [V1][DP] More robust DP/EP dummy request coordination (#16277) Nick Hill 2025-04-22 19:12:15 -07:00
bc7c4d206b [Kernel][ROCM] Upstream prefix prefill speed up for vLLM V1 (#13305)
Aleksandr Malyshev Sage Moore root Aleksandr Malyshev qli88 root
2025-04-22 19:11:56 -07:00
f67e9e9f22 add Dockerfile build vllm against torch nightly (#16936) Yang Wang 2025-04-22 19:08:27 -07:00
36fe78769f [Bugfix] validate urls object for multimodal content parts (#16990) Guillaume Calmettes 2025-04-23 03:43:06 +02:00
83d933718c [Core][V1][TPU] Enable structured decoding on TPU V1 (#16499) Chenyaaang 2025-04-22 17:05:23 -07:00
5175b884f7 [BugFix] Remove default multiproc executor collective_rpc timeout (#17000) Nick Hill 2025-04-22 16:27:14 -07:00
5536b30a4c Fencing Kernels Tests for enabling on AMD (#16929) Alexei-V-Ivanov-AMD 2025-04-22 11:32:40 -05:00
7f58fb9718 Add assertion for no objects while hashing hf_config (#16930) Richard Zou 2025-04-22 12:32:22 -04:00
30bc3e0f66 [FEAT][ROCm]: Support AITER MLA (#15893) vllmellmandqli88 2025-04-23 00:31:13 +08:00
f34410715f [frontend] enhance tool_calls type check (#16882) Reidandreidliu41 2025-04-22 23:40:24 +08:00
68d4c33202 [Misc] Add S3 environment variables for better support of MinIO. (#16977) Chauncey 2025-04-22 22:27:36 +08:00
f961d7f6ef [BugFix] Pass in correct VLLM config in FlashInfer backend (#13207) (#16973) Zhengyuan Su (苏政渊)and苏政渊 2025-04-22 21:44:10 +08:00
d059110498 Improve configs - SpeculativeConfig (#16971) Harry Mellor 2025-04-22 13:55:36 +01:00
571e8dd65e [Bugfix] Fix distributed bug again in Qwen2.5-VL & Qwen2.5-Omni (#16974) Yang Fan 2025-04-22 20:23:17 +08:00
4b91c927f6 [Misc] refactor example series (#16972) Reidandreidliu41 2025-04-22 19:44:21 +08:00
0e237f0035 [FEAT][ROCm] Integrate Paged Attention Kernel from AITER (#15001) vllmellmandtjtanaa 2025-04-22 17:46:28 +08:00
8f7bace7c3 [Doc] Improve documentation for multimodal CLI args (#16960) Cyrus Leung 2025-04-22 16:35:35 +08:00
e4d6144232 [BugFix] Fix incremental detokenization perf issue (#16963) Nick Hill 2025-04-22 01:16:19 -07:00
8d32dc603d [Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036) Lei Wangandxinyuxiao 2025-04-22 16:01:36 +08:00
c4ab9f3e71 [V1] Remove pre-allocation for KV cache (#16941) Woosuk Kwon 2025-04-22 00:52:18 -07:00

... 172 173 174 175 176 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86