Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

7be3a59d8e [Misc] enhance static type hint (#23059) Ning Xie 2025-08-18 13:09:08 +08:00
8ea0c2753a [Misc] Minor code cleanup for _get_prompt_logprobs_dict (#23064) Woosuk Kwon 2025-08-17 18:16:03 -07:00
0fc8fa751a fix: gptq marlin weight loading failure (#23066) Simon Mo 2025-08-17 15:56:07 -07:00
21e39436c8 [XPU] fix xpu to set cudagraph batch sizes (#23044) Calvin Chen 2025-08-18 05:45:42 +08:00
6d243efeda [Misc] Convert use_structured_output property into constant (#23060) Woosuk Kwon 2025-08-17 12:41:38 -07:00
c55bc1db26 [Misc] Remove dead return (#23061) Woosuk Kwon 2025-08-17 10:36:46 -07:00
292084e72a [BugFix] Fix for IMA in FA3 varlen combine (#22967) Lucas Wilkinson 2025-08-17 11:52:04 -04:00
16bff144be [Misc] fix typo in the multimodal doc (#23051) Kevinzz 2025-08-17 16:56:20 +08:00
fe0411fc6f [Bugfix] should use stack instead of concat (#22972) 947132885andIsotr0py 2025-08-17 16:46:36 +08:00
4d4061b6e7 [Kernel] Add cuda kernel for gpt_oss activation (#22951) Jee Jee Li 2025-08-17 13:03:24 +08:00
87f48623a5 [Misc] method name typo fix (#23042) Ning Xie 2025-08-17 12:49:14 +08:00
5c32143b9d [Refactor] Defer tensor data construction in MultiModalKwargs (#23030) Cyrus Leung 2025-08-17 12:05:50 +08:00
94096a47c9 [UX] Separate marlin moe config logic from triton moe (#23006) Michael Goin 2025-08-16 22:16:42 -04:00
a258ad8bcc [Bugfix] fix qwen3 moe fp8 accuracy issue (#23031) Jinzhen Lin 2025-08-17 08:41:23 +08:00
bf7f470b22 [V1] Logits processors extensibility (#19912)
afeldman-nm Nick Hill Andrew Feldman Harry Mellor
2025-08-16 15:59:17 -04:00
4fc722eca4 [Kernel/Quant] Remove AQLM (#22943) Michael GoinandWentao Ye 2025-08-16 15:38:21 -04:00
3253ae765e [Flaky CI] Increase timeout tolerance for test_mp_crash_detection+test_default_mm_lora_chat_completions (#23028) Michael Goin 2025-08-16 14:33:08 -04:00
000cceca8c [Bugfix gpt-oss] Fix float32 convert for flashinfer sink support (#23016) Michael Goin 2025-08-16 14:16:00 -04:00
68373d3126 [Frontend] Added support for HermesToolParser for models without special tokens (#16890) Woonggi Min 2025-08-17 02:38:42 +09:00
52ce1420e9 Fix handling of max_num_batched_tokens for pooling tasks (#23004) Maximilien de Bayser 2025-08-16 14:36:30 -03:00
829bbd7882 [New Model]mBART model (#22883) 汪志鹏 2025-08-16 20:16:58 +08:00
4dff91c93d [Refactor] Allow optional MultiModalKwargsItem in IPC (#23022) Cyrus Leung 2025-08-16 19:30:49 +08:00
de9cb61763 Add docs for PrefixRepetitionDataset + enable usage with vllm bench throughput (#23012) Seiji EicherandRoger Wang 2025-08-16 03:21:20 -07:00
2dbccce8a6 [CI][Bugfix] Skip Ovis2 generation test because of broken remote code (#22954) Isotr0py 2025-08-16 17:44:19 +08:00
933f45334a [Core] Make cudagraph check cuda platform only (#23005)
Chengji Yao Chengji YaoLi, Jiang <jiang1.li@intel.com>
2025-08-16 00:46:00 -07:00
cc826a202b [Multimodal] Update Tensor schema test to cover arbitrary shape mm inputs (#22867) Isotr0py 2025-08-16 15:44:50 +08:00
6d3da472bc [Misc] Add --save-dir option to benchmark_moe (#23020) Jee Jee Li 2025-08-16 15:26:10 +08:00
78863f8c5c [BugFix] Add support for loading prompt embeds tensors serialized on unavailable devices and sparse tensors (#22962) Andrew Sansom 2025-08-16 01:25:10 -05:00
5157827cfc [Build] Env var to disable sccache (#22968) Lucas Wilkinson 2025-08-16 01:36:27 -04:00
7caec10e7b [XPU]avoid circular import during XPU init (#23017) Kunshang Ji 2025-08-16 13:16:34 +08:00
1f83e7d849 [misc] nsys profile output kernel classifier and visualizer (#22971) Grace Ho 2025-08-15 19:52:51 -07:00
e4e37ded56 [V1] support min_tokens for detokener (#22014) Calvin ChenandNick Hill 2025-08-16 10:28:10 +08:00
f6b5040590 [Frontend] Avoid list copies in serving_chat.py (#22947) Nick Hill 2025-08-15 19:06:30 -07:00
fbd88728b3 [Bugfix] Fix DeepSeek MTP (#22934) Benjamin Chislett 2025-08-15 21:25:06 -04:00
070da660c1 [Kernel] Simplify get_kv_cache_layout and cache use_trtllm_attention env-dependent bit (#22735) Nicolò Lucchesi 2025-08-16 02:14:08 +02:00
ad0297d113 [Misc] Support passing multiple request ids at once to AsyncLLM.abort() (#22944) Nick Hill 2025-08-15 17:00:36 -07:00
236b864e4f [BugFix] Make run_once thread-safe (#22978) Yichen Yan 2025-08-16 07:56:17 +08:00
3e2f7985a2 Support multiple attention groups for KV sharing (#22672) Yong Hoon Shin 2025-08-15 16:54:10 -07:00
c280066f9d [v1] Move block_hashes from KVCacheManager to Request.block_hashes (#19728) Or Ozeri 2025-08-16 02:52:52 +03:00
b9dc9d2607 [BugFix] Handle case where async utility call is cancelled (#22996) Nick HillandYinghai Lu 2025-08-15 16:38:42 -07:00
1fc375dc05 [Structured Outputs] [Bug] Fix misalignment in apply_grammar_bitmask causing unintended masking and NaN logits (#22963) rishitdholakia13 2025-08-15 17:25:05 -06:00
76144adf76 ci: Add CUDA + arm64 release builds (#21201) Eli Uriegas 2025-08-15 16:16:23 -07:00
f5d412bafb [BugFix] Fix regression caused by mamba state dtype PR (#22998) Thomas Parnell 2025-08-16 00:55:26 +02:00
177e55e3bd [Attention] FA3 Attention Sinks Perf Boost (#22478) Lucas Wilkinson 2025-08-15 17:41:07 -04:00
1723ef1aae minor: zero workspace buffer init for flashinfer trtllm-gen attn (#22603) eigen 2025-08-15 17:38:10 -04:00
00d6cba0cf Add PrefixRepetitionRandomDataset to vllm bench serve datasets (#20638) Seiji Eicher 2025-08-15 14:09:23 -07:00
7f89ed248f [Fix] enable swap_ab for pplx problem size computation (#22991) shixiancandShixian Cui 2025-08-15 14:02:12 -07:00
8a87cd27d9 [CI] Speed up Whisper tests by reusing server (#22859) Michael Goin 2025-08-15 16:56:31 -04:00
a344a1a7da Use regex in convert-results-json-to-markdown.py (#22989) Michael Goin 2025-08-15 16:54:20 -04:00
79899b63f6 [Bugfix] Added more env vars to hash (#22449) nvjullin 2025-08-16 04:08:37 +08:00
6e670778cd [Core] direct indexing on self.block_table_np in compute_slot_mapping (#22940) Zebing Lin 2025-08-15 15:12:12 -04:00
df5afa82e5 [Log] Debug Once for Randomizing dummy data for DP Rank (#22860) Wentao Ye 2025-08-15 14:51:50 -04:00
6cd69f51bf [Model] Granite-4 support loading quantized checkpoint (#22925) Chih-Chieh Yang 2025-08-15 14:47:56 -04:00
8ad7285ea2 [Kernels] Clean up FusedMoeMethodBase and modular kernel setup. Remove extra arguments from modular kernel methods. (#22035) bnellnmandMichael Goin 2025-08-15 14:46:00 -04:00
48b01fd4d4 [Structured Output] Make the output of structured output example more complete (#22481) Shanshan Shen 2025-08-16 02:29:25 +08:00
993d3d122b [Benchmarks] Include image data when ShareGPT4V dataset is used. (#22955) Chenheli Hua 2025-08-15 11:23:06 -07:00
68af77e51c [FIXBUG] Correctly Apply Grammar Bitmask in Mixed Batches (#22896) JartX 2025-08-15 19:42:49 +02:00
6b04039a72 [BugFix] Skip the Q component for QKVParallelLinear in the case of QKVCrossParallelLinear since its width is 0 (#22369) sstamenk 2025-08-15 19:17:31 +02:00
1c859a1387 [V0 Deprecation] Remove advance_step (#22969) Woosuk Kwon 2025-08-15 08:22:31 -07:00
74f441f4b5 [Core] Allow full cudagraph with separate attention routines and orthogonal to compilation, add support for FA2 and FlashInfer (#20059)
fhl2000 Luka Govedič Lucas Wilkinson Lucas Wilkinson
2025-08-15 22:01:39 +08:00
a0632a3e03 [Frontend] Expose do_log_stats interval to env (#22905) CsrayzandHarry Mellor 2025-08-15 21:00:20 +08:00
e8b40c7fa2 [CI] Remove duplicated docs build from buildkite (#22924) Harry Mellor 2025-08-15 13:58:06 +01:00
48f4636927 [Misc] Ignore ep_kernels_workspace (#22807) Jee Jee Li 2025-08-15 20:58:03 +08:00
75531a6c13 [V1] [Hybrid] Support using float32 for state in Hybrid Models (Mamba2, Mamba1, Minimax) (#22928)
Thomas Parnell Daniel Afrimi Burkhard Ringlein Chen Zhang
2025-08-15 14:57:06 +02:00
22341b996e Improve multimodal hasher performance for re-used Image prompts (#22825) Staszek Paśko 2025-08-15 14:32:56 +02:00
49252cf59e [MM] Allow skipping memory profiling for multimodal models. (#22950) Roger Wangandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-08-15 04:41:38 -07:00
3e6dd40016 [Bugfix] fix cuda 12.6 and 11.8 build (#22952) Jinzhen Lin 2025-08-15 18:10:22 +08:00
aa300c438d [Bugfix] Unquote file uri before reading image (#22912) Sayandip DuttaandCyrus Leung 2025-08-15 14:58:00 +05:30
fe91ce9591 [V1] - Split Prefill and Decode for Mamba1 models (#22653)
amirai21 asafg Asaf Joseph Gardin
2025-08-15 11:59:52 +03:00
5406ebf5c9 [CI] Pooling models mteb test uses enforce_eager (#22878) wang.yuqi 2025-08-15 16:16:15 +08:00
b2c06509e5 [P/D]Provide bucket algorithm rate limiter for proxy_server (#22643)
frankie Cyrus Leung Kuntai Du
2025-08-15 15:01:48 +08:00
b2f6c247a9 Revert "[ROCm][AITER] Support AITER Rope ops in RotaryEmbedding Module." (#22956) TJianandvllmellm 2025-08-14 23:39:19 -07:00
3d232dbd19 [Mamba] - refactor: Renamed mamba_attn to mamba2_attn (#22818) Asaf Joseph Gardinandasafg 2025-08-15 09:38:05 +03:00
5c3fbfe46b [Feature] Full Cuda Graph Support for Cutlass MLA and 6% E2E Throughput Improvement (#22763) Wentao Ye 2025-08-15 02:27:30 -04:00
b4cef5e6c7 refactor: Change scaling factors calculation for flashinfer FusedMoE (#22812) amirkl94andMichael Goin 2025-08-15 09:19:31 +03:00
0fe85087a9 [CI Perf] Prune tests in tests/kernels/attention/ (#22936) Michael Goin 2025-08-14 23:34:53 -04:00
d2b0e97ea6 [CI Perf] Prune tests in tests/kernels/moe/ (#22939) Michael Goin 2025-08-14 23:33:42 -04:00
590bddbfc5 [CI Perf] Prune tests in tests/kernels/quantization/ (#22942) Michael Goin 2025-08-14 23:25:34 -04:00
ae05a6d83d [BugFix] Fix port lookup in internal DP LB tests (#22252) Nick Hill 2025-08-14 20:17:11 -07:00
0933f9d518 [BugFix][KVConn] Fix use of get_required_kvcache_layout (#22734) Nick Hill 2025-08-14 18:39:43 -07:00
f1f0d2fab8 Revert "[Kernel] Add cuda kernel for gpt_oss activation" (#22948) Simon Mo 2025-08-14 17:38:10 -07:00
81f4b96481 [Kernel] Add cuda kernel for gpt_oss activation (#22538) Jee Jee Li 2025-08-15 08:21:29 +08:00
39cd09dc86 [Bugfix] use flash attn on sm90 (#22933) Yongye ZhuandMichael Goin 2025-08-14 19:37:22 -04:00
919234fe17 [BugFix] Fix initial DP request load imbalance (#22910) Nick Hill 2025-08-14 15:20:28 -07:00
ebcce2cd36 [Core] Return final response for aborted requests from AsyncLLM.generate (#22283) Nick Hill 2025-08-14 14:49:02 -07:00
4121de512e [Quantization]: Support compressed-tensors mixed-precision model loading (#22468) Dipika Sikka 2025-08-14 17:32:09 -04:00
279a5f31b3 [Kernel] Add nvfp4 gemm flashinfer backends (#22346) nvjullinandmgoin 2025-08-15 04:03:55 +08:00
b8ff05361a [CI] Temporarily disable flaky test (#22930) Lucas Wilkinson 2025-08-14 15:59:16 -04:00
637093ae26 docs: update fastsafetensors usage instructions (#22891) Nir 2025-08-14 22:56:54 +03:00
33c63e9547 [Kernel] [Quantization] Add MXFP4 and bias support for marlin kernel (#22428) +105
Jinzhen Lin rongfu.leng Huzaifa Sidhpurwalagemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Russell Bryant Varun Sundar Rabindranath Varun Sundar Rabindranath Harry Mellor Jee Jee Li Michael Goin Animesh Jain Rui Qiao XiongfeiWei Nick Hill Wentao Ye JartX fhl2000 vllmellm kf Nicolò Lucchesi Dipika Sikka Sage Moore tjtanaavllm Yong Hoon Shin Chih-Chieh Yang Roger Wang Vadim Gimpelson Yuxuan Zhang Isotr0py Cyrus Leung Thomas Parnell Yan Ma Xiao jiahanc Isotr0py Ye Qi Roberto L. Castro Ning Xie H David Ben-David David Ben-David Woosuk KwonLi, Jiang <jiang1.li@intel.com>TankNee Cyrus Leung Seiji Eicher ZiTian.Zhao 22quinn Abirdcfly Giancarlo Delfin Chenxi Yang Chenxi Yang Tyler Michael Smith Weixiao Huang Raghav Ravishankar ericehanley Zhonghua Deng Po-Han Huang PiteXChen lkchen TJian Gregory Shtrasberg tlipoca9 elvischenv wang.yuqi Benji Beck youkaichao Siyuan Liu Benjamin Chislett LiuXiaoxuanPKU simon-mo Chen Zhang Hongxia Yang Minseok Lee Yongye Zhu Lucas Wilkinson Zhang Jason Asaf Joseph Gardin asafg Lain tc-mb imning3 Maximilien de Bayser Kunshang Ji Tao He qscqesze Syed Muhammad Bin Asif Lionel Villard WeiQing Chen ycyaw66 Moritz Sanft Ming Yang Adrián García García Michael Goin JaceyShao shaojunqi Ricardo Decal Andrew Chan fxmarty-amd Andrew Sansom Zhiyu Shu Wang XIn Li Junhao Li Chauncey iAmir97 iAmir97 Hong Hanh Daniel Serebrenik yewentao256 Guy Stone yyweiss Pradyun92 Pradyun Ramadorai Nicolò Lucchesi
2025-08-15 02:23:22 +08:00
ab9f2cfd19 [CI] [Hybrid] Bump min transformers version for Bamba and Jamba (#22908) Thomas Parnell 2025-08-14 20:01:16 +02:00
dbe298046c [Bugfix] Fix parsing of --disable-mm-preprocessor-cache (#22909) Cyrus Leung 2025-08-14 23:09:44 +08:00
625ccd1c4d [Bugfix] Replace custom Encoding class with BatchEncoding in MistralTokenizer (#22786) Jiangyun Zhu 2025-08-14 23:09:27 +08:00
92ff41abea [Model] Modify the gate implementation of glm4_moe (#22832) Jee Jee Li 2025-08-14 20:28:50 +08:00
829b9a62d0 [Perf] Dont create unnecessary pooling params (#22876) Lucas Wilkinson 2025-08-14 08:28:09 -04:00
540d54ca8d [CI] Re-enable transcriptions test_long_audio_request (#22890) Nicolò Lucchesi 2025-08-14 13:34:34 +02:00
0783f13960 [Doc] fix dead link (#22898) Daniele 2025-08-14 13:06:13 +02:00
7655dc3e45 [Bugfix] Add reset prefix cache for online serving (#22726)
iAmir97 iAmir97gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-08-14 18:04:18 +07:00
f4efda821d Remove Phi 4 Flash configuration workaround (#22723) Harry Mellor 2025-08-14 12:03:49 +01:00
eb08487b18 [BugFix] Threadsafe close async zmq sockets (#22877) Nick HillandIsotr0py 2025-08-14 03:44:29 -07:00

... 146 147 148 149 150 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86