Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

2461d9e562 [CI/Build] Split out mm processor tests (#23260) Cyrus Leung 2025-08-21 11:05:20 +08:00
7be5d113d8 [CPU] Refactor CPU W8A8 scaled_mm (#23071) Li, Jiang 2025-08-21 09:34:24 +08:00
b029de9902 [Optimization] Make new_block_ids None if empty (#23262) Woosuk Kwon 2025-08-20 18:25:56 -07:00
bbea1cefdd [CI Bugfix] Fix CI by fully removing --enable-prompt-adapter (#23284) Michael Goin 2025-08-20 20:18:12 -04:00
f5aa307d77 Remove duplicate entry in vllm.attention.__all__ (#23296) Russell Bryant 2025-08-20 20:14:59 -04:00
4b795020ed [EP] Add logging for experts map (#22685) 22quinnandSimon Mo 2025-08-20 16:46:06 -07:00
c86af22f31 [Fix] remove is_marlin param in benchmark_moe (#23286) shixianc 2025-08-20 15:04:21 -07:00
10cc12ba66 Feature/mla tests (#23195) Matthew Bonanni 2025-08-20 17:46:47 -04:00
a4fbb32fab Remove chunked_prefill_enabled flag in V1 MLA (#23183) Matthew Bonanni 2025-08-20 17:43:17 -04:00
1b125004be [misc] fix multiple arch wheels for the nightly index (#23110) youkaichao 2025-08-21 05:15:34 +08:00
4fbda0b20c [Feature] use --eplb_config to set eplb param (#20562) rongfu.lengandHarry Mellor 2025-08-21 05:07:28 +08:00
4e51fa8cba Do not use eval() to convert unknown types (#23266) Russell Bryant 2025-08-20 16:28:30 -04:00
bf7c99dfc4 [Perf] Speed up function _convert_tokens_to_string_with_added_encoders by 13.7x (#20413)
Saurabh Misracodeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>Aseem Saxena
2025-08-20 13:17:11 -07:00
b95697d731 [Frontend] improve error logging of chat completion (#22957) Chen Zhang 2025-08-20 13:03:37 -07:00
582bbe6bd7 [Fix] correct tool_id for kimi-k2 when use tool_choice=required (#21259) bigmoyanandwangzhengtao 2025-08-21 03:59:54 +08:00
0cdbf5e61c [Kernel/Quant] Remove the original marlin format and qqq (#23204) Michael Goin 2025-08-20 15:13:36 -04:00
ebe56a0064 Small fix for Command-A-Vision (#23268) dongluw 2025-08-20 14:15:18 -04:00
f77a0802b7 Limit HTTP header count and size (#23267) Russell BryantandTaneem Ibrahim 2025-08-20 13:57:37 -04:00
c4477f55e5 Migrate Mistral3ImagePixelInputs to TensorSchema (#21945) Benji BeckandCyrus Leung 2025-08-20 10:37:29 -07:00
dfd2382039 [torch.compile] Support conditional torch.compile per module (#22269) Yong Hoon Shin 2025-08-20 09:52:59 -07:00
3b11b26b50 [FIXBUG ] Allow disabling rocm_aiter_fa backend for ROCm GPUs not compatible with AITER (#22795) JartXandtjtanaa 2025-08-20 18:08:29 +02:00
d6d13bd49e [Misc] Add max_seq_len to CommonAttentionMetadata (#23216) Woosuk Kwon 2025-08-20 09:05:29 -07:00
5efd6905bc [CLI][Doc] Formalize --mm-encoder-tp-mode (#23190) Cyrus Leung 2025-08-20 23:42:28 +08:00
b17109beea [Kernel] CUTLASS MoE FP8: Integrate cuda moe permute/unpermute (#23045) shixianc 2025-08-20 07:35:26 -07:00
4449235843 [Bugfix] Ensure correctness of HCXVision processing (#23254) Cyrus Leung 2025-08-20 22:19:30 +08:00
38217877aa [Fix] fix offline env use local mode path (#22526) rongfu.leng 2025-08-20 21:34:49 +08:00
c6d80a7a96 [Model] Improve olmo and olmo2 (#23228) Jee Jee Li 2025-08-20 20:47:05 +08:00
7cd17e22d7 [Model][V1] Support Ernie MTP (#22169) xyxinyangandzhouchong 2025-08-20 20:41:55 +08:00
50df09fe13 Update to flashinfer-python==0.2.12 and disable AOT compile for non-release image (#23129) Michael Goin 2025-08-20 08:05:54 -04:00
68fcd3fa73 [Bugfix] Ensure correctness of Cohere2Vision processing (#23245) Cyrus Leung 2025-08-20 19:09:18 +08:00
83e69a09d6 [Model] Support deepseek with eagle (#21086) Xin Yang 2025-08-20 04:01:31 -07:00
3aa8c10038 Fix missing quotes (#23242) Shiming Zhang 2025-08-20 18:46:59 +08:00
103f1ec8d3 [Model] use autoWeightsLoader for gptoss (#22446) Calvin Chen 2025-08-20 18:16:27 +08:00
d983769c41 fix cuda graph (#22721) who who who 2025-08-20 14:24:37 +08:00
8fd920924c [BugFix] Fix stuck stats/metrics after requests are aborted (#22995) Nick Hill 2025-08-19 22:50:29 -07:00
de7b67a023 [CI/Build] Sync multimodal tests (#23181) Cyrus Leung 2025-08-20 13:06:42 +08:00
f729023272 [CI/Build] Also check DP in benchmarks throughput script (#23038) Zhewen LiandSimon Mo 2025-08-19 21:09:27 -07:00
1a3079a15e chore: support pytorch format in lora (#22790) 길재은 2025-08-20 13:02:50 +09:00
941f56858a Fix a performance comparison issue in Benchmark Suite (#23047)
Louie Tsaigemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Li, Jiang <bigpyj64@gmail.com>
2025-08-19 20:14:32 -07:00
a634733f67 [Attention] Optimize make_local_attention_virtual_batches for Flash Attention (#23185) Zebing Lin 2025-08-19 22:57:47 -04:00
64ab3c7253 [Doc] Update V1 status of various pooling models (#23189) Cyrus Leung 2025-08-20 10:33:41 +08:00
e58c5a9768 [Core] Add torch profiler CPU traces for AsyncLLM. (#21794) Chenheli Hua 2025-08-19 19:32:47 -07:00
d46d417b58 [CI Perf] Only test bfloat16 for tests/compile/test_fusion_all_reduce.py (#23132) Michael Goin 2025-08-19 22:18:52 -04:00
0167efe20d [Core] Optimize scheduler request removal for single completions (#21917) 633WHUandchiliu 2025-08-20 09:25:59 +08:00
c32e6ad1f6 [Quantization] Bump Compressed Tensors Version (#23202)
Kyle Sayers Dipika Sikka Michael Goin
2025-08-19 20:39:28 -04:00
1630cc8d0f [Benchmarks] Add video inputs to ShareGPTDataset. (#23199) Chenheli Hua 2025-08-19 16:42:31 -07:00
14e2b0730b [BugFix] fix CUTLASS MLA full cudagraph (#23200) Lucas Wilkinson 2025-08-19 18:17:08 -04:00
0f4f0191d8 [CI/Build] Replace lm-eval gsm8k tests with faster implementation (#23002) Michael Goin 2025-08-19 18:07:30 -04:00
a38b8af4c3 [NVIDIA] Add SM100 Flashinfer Cutlass MoE fp8 backend (#22357) amirkl94 2025-08-20 01:01:53 +03:00
21dce80ea9 [CI/Build] Add support for Python 3.13 (#13164) Michael GoinandCyrus Leung 2025-08-19 16:49:34 -04:00
e61bac87ee [Misc] Minor refactoring for FlashInfer backend (#23147) Woosuk Kwon 2025-08-19 13:11:51 -07:00
80141bbf2f fix: use cache_salt for gpt-oss (#23186) Marko Rosenmueller 2025-08-19 20:12:25 +02:00
b94faf9d50 [Bugfix] Fix accuracy issue when using flashinfer cutlass moe, TP=1 and modelopt. (#23125) bnellnmandMichael Goin 2025-08-19 14:00:51 -04:00
5b5f350d67 [Misc] Enable yapf for FlashInfer backend (#23193) Woosuk Kwon 2025-08-19 10:33:47 -07:00
f7cf5b512e [Frontend] Add /collective_rpc API endpoint (#23075) 22quinn 2025-08-19 10:29:32 -07:00
03d4235fd2 [Misc] Fix the benchmark's README and improve the error messages for the benchmark's argument checks (#22654) Ruixiang Tan 2025-08-20 01:18:51 +08:00
d6a1a20973 [CI/Build] Update transformers to v4.55.2 (#23093) Isotr0py 2025-08-20 01:06:17 +08:00
a70d0bd0a3 Migrate LlavaOnevisionMultiInputs to TensorSchema (#21844) Benji Beck 2025-08-19 10:02:02 -07:00
24f4d1a224 Add return_token_ids parameter to OpenAI API endpoints (#22587)
Yuge Zhang Claude Simon Mo
2025-08-20 00:48:31 +08:00
4f510bc2a1 [Model] Removes redundant all-reduce operation in Qwen3MoeSparseMoeBlock (#23169) yiz-liu 2025-08-20 00:18:41 +08:00
1298c67795 [FEAT] [Performance] Enable DP for ViT in Qwen2.5VL (#22742)
TJian DarkLight1337 Cyrus Leung
2025-08-19 08:25:57 -07:00
4d9c61993a [Bugfix] Fix benchmark_moe.py (#23177) Jee Jee Li 2025-08-19 21:39:40 +08:00
b87cb97a53 [Model] support new model ovis2.5 (#23084)
myselvess Isotr0py Isotr0py
2025-08-19 21:12:59 +08:00
f856c33ce9 [Model] Add multi_label_classification support (#23173) wang.yuqi 2025-08-19 20:54:30 +08:00
03752dba8f [NVIDIA] Support Flashinfer TRTLLM FP8-q/kv/out Attention Kernel (#21716)
elvischenv Michael Goin Luka Govedič
2025-08-19 20:22:15 +08:00
40f26734b9 [Misc] Fix seq_lens for graph capture (#23175) Woosuk Kwon 2025-08-19 03:58:16 -07:00
2c3f557f08 [Doc] use power of 2 (#23172) Tialo 2025-08-19 13:16:23 +03:00
21bcc8263f [Misc] Avoid accessing req_ids inside a loop (#23159) Woosuk Kwon 2025-08-19 02:39:38 -07:00
5bfe0dea7a [bug fix] Fix llama4 spec decoding (#22691) qizixiandLu Fang 2025-08-19 17:53:24 +09:00
31fd3265c8 [Bugfix] Fix broken Minimax-01-VL model (#22116) Isotr0py 2025-08-19 16:49:29 +08:00
31436e8b4f [Misc] Add request_id into benchmark_serve.py (#23065) hustxiayang 2025-08-19 04:32:18 -04:00
4efd43e9b4 Fix GLM-4.5V-FP8 numerical issue (#22949) qizixiandCyrus Leung 2025-08-19 16:56:31 +09:00
3c8a787247 [Benchmark] Add flag --served-model-name to benchmark_serving_multi_turn (#22889) Daniel Serebrenik 2025-08-19 10:48:07 +03:00
01a08739e0 [misc] split engine_model into json file for nsys profile tool (#23117)
Grace Hogemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
2025-08-19 00:44:53 -07:00
fda9537c5e [Model] Support Pipeline Parallelism for moonshotai/Kimi-VL-A3B-Thinking-2506 (#23114) Jiangyun ZhuandCyrus Leung 2025-08-19 14:24:31 +08:00
90bbe0a5ad [Log] Warning Once for Cutlass MLA (#23137) Wentao Ye 2025-08-19 02:24:16 -04:00
e75f342261 Migrate InternVLImagePixelInputs (in nemotron_vl.py) to TensorSchema (#22023) Benji BeckandCyrus Leung 2025-08-18 22:48:26 -07:00
78dba404ad [Hardware][IBM Z]Enable v1 for s390x and s390x dockerfile fixes (#22725) Nikhil Suryawanshi 2025-08-19 10:10:37 +05:30
e9d6a3db69 [TPU] make ptxla not imported when using tpu_commons (#23081) Chengji YaoandChengji Yao 2025-08-18 20:46:42 -07:00
a4454e9401 chore: disable enable_cpp_symbolic_shape_guards (#23048) Xiao 2025-08-18 20:08:05 -07:00
14006840ea [V0 Deprecation] Remove V0 FlashInfer attention backend (#22776) Woosuk Kwon 2025-08-18 19:54:16 -07:00
6603288736 [CI][V0 Deprecation] Removed V0 Only Chunked Prefill and Prefix Caching Tests (#22871)
Robert Shaw Robert Shaw Woosuk Kwon
2025-08-18 20:39:01 -04:00
95e3095136 [Misc] Add @tdoublep as a maintainer of hybrid model and Triton-attention related code (#23122) Thomas Parnell 2025-08-19 02:31:38 +02:00
c9b38be8aa [Spec Decode] Make propose_draft_token_ids non-blocking for lower TTFT (#23041) Woosuk Kwon 2025-08-18 17:20:38 -07:00
0dd3f4f5ab [Misc] Minor refactoring for prepare_inputs (#23116) Woosuk Kwon 2025-08-18 16:58:05 -07:00
498259ccce Install tpu_info==0.4.0 to fix core dump for TPU (#23135) Xiang Xu 2025-08-18 16:23:33 -07:00
6d25e3fd6e Use Blackwell FlashInfer MXFP4 MoE by default if available (#23008) Michael Goin 2025-08-18 18:25:49 -04:00
ac6eb49de3 fix: OpenAI SDK compat (ResponseTextConfig) (#23126)
Breno Baldas Skukgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Michael Goin
2025-08-19 00:22:59 +02:00
bf756321c7 [CI Bugfix] Pin openai<1.100 to unblock CI (#23118) Michael Goin 2025-08-18 15:14:01 -04:00
0e3bb543f0 [Bugfix] Support compile for Transformers multimodal (#23095) Raushan Turganbay 2025-08-18 15:35:48 +02:00
569aefd134 chore: remove unnecessary patch_padding_side for the chatglm model (#23090) 杨朱 · KikiandCyrus Leung 2025-08-18 20:32:13 +08:00
d3f71f1224 [Refactor] Get prompt updates earlier (#23097) Cyrus Leung 2025-08-18 20:31:53 +08:00
5a30bd10d8 [Bugfix] fix IntermediateTensors equal method (#23027) Ning Xie 2025-08-18 17:58:11 +08:00
27e8d1ea3e [Refactor] Define MultiModalKwargsItems separate from MultiModalKwargs (#23053) Cyrus Leung 2025-08-18 17:52:00 +08:00
5c79b0d648 [XPU][CI]add xpu env vars in CI scripts (#22946) Kunshang Ji 2025-08-18 17:47:03 +08:00
5f5664b3e4 [XPU] Fix compile size for xpu (#23069) Kunshang Ji 2025-08-18 15:04:08 +08:00
89657a557c [Misc] Fix backward compatibility from #23030 (#23070) Roger WangandRoger Wang 2025-08-17 23:33:29 -07:00
08d5f7113a [Misc] refactor function name (#23029) Ning Xie 2025-08-18 13:16:21 +08:00
b2fd0b81e0 [Bugfix][CI] Machete kernels: deterministic ordering for more cache hits (#23055) Andy Lo 2025-08-18 07:10:26 +02:00
9f1c642254 [Bugfix] fix Qwen2.5-Omni processor output mapping (#23058)
double7 杨森gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-08-18 13:09:11 +08:00

... 145 146 147 148 149 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86