Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

4c47710bf7 [CI/Build] Apply ruff formatter to pass pre-commit (#40078) Shinichi Hemmi 2026-04-17 09:54:32 +09:00
bf9a5ddb24 [MLA] Optimize mla indexer prepare uniform decode for MTP > 1 (#39458) Giancarlo Delfin 2026-04-16 16:27:51 -07:00
79e799ebbd [Bugfix] Temporarily disable B200 fp4 MoE layer tests (#40057) bnellnm 2026-04-16 19:26:55 -04:00
c4e601c73c Bugfix: Parakeet: .conv.pointwise/depthwise_conv1/2.bias weigths can exist even if convolution_bias=False (#40007) Netanel Haber 2026-04-17 02:22:05 +03:00
29057d3bee [Compilation] Add Unit Tests for VllmFusionPatternMatcherPass (#39692) BadrBasowid 2026-04-17 06:57:16 +08:00
219bb5b8c0 [Misc] Update committers.md (#40058) Matthew Bonanni 2026-04-16 16:48:41 -04:00
ad2b1277f9 [Quantization] Consolidate experts_int8 with fp8 online quantization (#38463) Asaf Gardin 2026-04-16 23:12:20 +03:00
b897f00c9c Gate SSU dispatch setup (#40039) roikoren755 2026-04-16 23:06:01 +03:00
adf9bb3c57 [CI] Add weight transfer tests to CI (#39821)
Sumanth R Hegde Robert Shaw Cyrus Leung
2026-04-16 12:51:45 -07:00
b16fda62b7 [Misc] Add @sfeng33 to CODEOWNERS (#40048) Flora Feng 2026-04-16 15:25:29 -04:00
de111f3246 [Bugfix] Fix bench_serve UTF-8 decode crash on split multi-byte chars (#38732) Yufeng He 2026-04-17 03:01:25 +08:00
afabb5f45a [bugfix] Normalize tool message content from array to string format (#39899) Jared Wen 2026-04-17 02:54:39 +08:00
3abb7560c0 [Bugfix] Fix audioflamingo test (#40052) Roger Wang 2026-04-16 11:53:58 -07:00
617d1c2ff1 [Misc] Move pyav and soundfile to common requirements (#39997) Isotr0py 2026-04-16 23:52:37 +08:00
692db29cd4 [Bugfix] Fix Ray compiled-DAG SHM channel stalls by detaching zero-copy np.ndarray logprobs buffers (#35736) Nikita Shapovalov 2026-04-16 17:49:29 +02:00
82531edbfb [Refactor] Remove resampy dependency (#39524) Isotr0py 2026-04-16 23:48:17 +08:00
3daca38e22 [Misc] toy_proxy_server handle min_tokens (#39706) Nicolò Lucchesi 2026-04-16 17:08:22 +02:00
a302a8fd1b [Bugfix] Fix LLM priority normalization for single-string prompts (#40011) daiyu1111 2026-04-16 22:56:06 +08:00
4e8c3f1c19 [Frontend][last/5] Improve pooling entrypoints | clean up. (#39675) wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-04-16 22:53:23 +08:00
5e5afafa21 [Doc] add docs for online quant frontend (#39736) Vasiliy Kuznetsov 2026-04-16 10:52:58 -04:00
324a3d2bd8 [CI/Build] Improve stability of CPU tests (#39966) Li, Jiang 2026-04-16 21:50:36 +08:00
4269b79409 [Model] Use mm_features to compute mrope positions for PaddleOCR-VL (#39888) grYe99andgrYe99 2026-04-16 21:14:00 +08:00
edc3648966 [Kernel][Helion] Fix inductor fusion of Helion HOP (#39944) Yanan CaoandClaude Opus 4.6 2026-04-16 04:41:26 -07:00
9965f501a8 [Nixl] Bump Nixl version to 0.10.1 (#39922) Nicolò Lucchesi 2026-04-16 12:53:21 +02:00
17d87168d2 [Model] Use mm_features for Keye-VL and Keye-1.5-VL M-RoPE (#39869) lalit10 2026-04-16 02:16:06 -07:00
98700c6105 Fix #33773: Replace unconditional pandas import with PlaceholderModule (#39990) Netanel Haber 2026-04-16 12:06:51 +03:00
10e49d2638 [Docs] Update PR template to remove release notes google docs (#39982) Simon Mo 2026-04-16 00:22:03 -07:00
8d7c962833 [Bugfix] Accept **kwargs in MiniMaxM2Parser.__init__() (#39861) Tim Messerschmidt 2026-04-16 09:18:32 +02:00
f4ddaf8cf7 [XPU] use spawn multiproc method on xpu (#39671) Xinyu ChenandKunshang Ji 2026-04-16 14:42:07 +08:00
2cdf86044d Add Jina Embeddings v5 model support (fixes #38633) (#39575) Abhijit Royandwang.yuqi 2026-04-16 12:07:10 +05:30
7845379230 [Bugfix] add support for 'num_attention_groups' in ModelArchConfigConvertorBase for Step3p5 (#39796) realliujiaxu 2026-04-16 13:48:00 +08:00
4b7ca37bd4 [CPU][IBM Z][Dockefile][Docs] Fix s390x builds for torch 2.11 and update docs for s390x (#39910) R3hankhan 2026-04-16 10:56:21 +05:30
445b7093fd [perf][cpu] Accelerate BF16 GELU with LUT impl on Arm CPUs (#37469) Fadi Arafehandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-16 07:26:17 +02:00
18013df6ae [Bugfix] Reject empty tools array with HTTP 400 (#39780) jigangzandClaude 2026-04-15 21:08:04 -07:00
c0722f22de [Mistral Grammar] Fix tool and reasoning parsing (#39217) Julien Denize 2026-04-16 06:05:04 +02:00
951dca8019 [compile] Invoke split FX graph by codegen. (#38657) Zhengxu Chen 2026-04-16 00:03:41 -04:00
5f7fab881a [ROCm][FEAT] Integrate aiter gemm w8a8 ptpc (#33773) vllmellm 2026-04-16 09:55:29 +08:00
343f65234b [Model Runner V2][BugFix] fix num_sampled dtype for probabilistic rej… (#39951) Giancarlo Delfin 2026-04-15 18:09:11 -07:00
19fa90ed0d [Quantization] - Layerwise reloading of Attention/KV quantized models (#38995) Asaf Gardin 2026-04-16 04:03:32 +03:00
03f8d3a548 Update to transformers v5 (#30566)
Harry Mellor khluu Cyrus Leung jiang1.li
2026-04-16 00:29:15 +01:00
6dc9491406 [Model] Fix Gemma 4 token repetition by dynamic BOS injection for PT models (#39842) Luciano MartinsandLuciano Martins 2026-04-15 20:13:07 -03:00
27c0ca50a0 Update registry for Nemotron-v3 VL Nano/Super (#39747) Collin McCarthy 2026-04-15 16:09:11 -07:00
7c636432c6 [CI Bug] fix flaky test (#39938) Wentao Ye 2026-04-15 17:20:06 -04:00
c77e596e2e [FlashAttention] Don't overwrite flash_attn_interface.py when installing precompiled (#39932) Matthew Bonanni 2026-04-15 16:43:15 -04:00
ac3dac545b [Bugfix][Perf] Indexer upcast WK to BF16 for fusion (#38928) Benjamin Chislett 2026-04-15 16:39:32 -04:00
39ac640490 [Bug] Fix batch invariant test issue, bs=1 with max_seq_num = 1 (#39320) Wentao Ye 2026-04-15 16:28:43 -04:00
0b790a2501 [Speculative Decoding] Add DFlash speculators config parsing (#38300) zhanqiuhu 2026-04-15 16:22:15 -04:00
41488f2acd [Bugfix][NIXL] Fix _logical_to_kernel_block_ids conversion for non-mamba models (#39724) zhanqiuhu 2026-04-15 16:08:58 -04:00
102d51c9f3 [CI] Only build release Docker images when NIGHTLY=1 (#39882) Kevin H. LuuandClaude Opus 4.6 2026-04-15 12:01:13 -07:00
55e1a8e103 [Mooncake] Fix mixed MLA+Eagle block-size validation (#39596)
Zhewen Li Zhewen Li OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-15 11:36:47 -07:00
21e5a9f48e Bug/test eagle dp v2 (#39838) Monishver 2026-04-15 13:48:12 -04:00
8ad6ff0037 [Test] Fix @create_new_process_for_each_test("fork") in interactive shell pipeline (#29130) Mark McLoughlin 2026-04-15 17:22:20 +01:00
f2145efcb6 [BugFix] KeyError on scope["method"] for realtime api websocket in AuthenticationMiddleware (#36934) daniebrillandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-04-15 18:15:01 +02:00
ed33310552 [KVConnector][LMCache] Propagate cache_salt through MP connector for per-user cache isolation (#39837) Roy Huang 2026-04-15 09:10:49 -07:00
3cc328a4be [SpecDecode][Benchmark] Add SPEED-bench support to benchmarking CLI (#36029) Talor AbramovichandBenjamin Chislett 2026-04-15 19:00:07 +03:00
3beb57a238 [XPU] properly handle q_descale on XPU as quant query input not supported (#39676) Yan MaandKunshang Ji 2026-04-15 21:52:58 +08:00
8b5531933a FIX: support language_model.backbone naming in NemotronH Nano VL quantization config (#39901) danielafrimiandroot 2026-04-15 16:49:48 +03:00
db8d4a4a06 [BugFix][Graph] fix: handle empty sym_shape_indices in PiecewiseBackend. (#39395) Chauncey 2026-04-15 21:28:09 +08:00
fc701c8058 [XPU][MXFP4] add mxfp4 quant op for XPU (#39857) zofia 2026-04-15 20:28:19 +08:00
68be0f853e [Metrics] Add request_id to FinishedRequestStats to enable correlation between metrics and requests (#39710) Csrayz 2026-04-15 19:24:17 +08:00
60995c05b4 [Quantization][Autoround][CPU] Add W4A16 Support (#38192) Zhenzhong Xu 2026-04-15 18:38:31 +08:00
29e5d10205 fix online fp8 for MiniCPM models (#39862) Yan Ma 2026-04-15 17:09:20 +08:00
235e1f930a [kv_offload+HMA][3/N]: Remove block_size from KVEvents (#36644) Or Ozeri 2026-04-15 11:53:19 +03:00
431cea3eea [Bugfix] Fix tool_calls Iterable consumed when debug logging is enabled (#34844) +85
Wojciech Waismergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Michael Goin Xinyu Chen Ekagra Ranjan Rishi Puri zzaebok Cyrus Leung Yuwei An yuwei Artem Perevedentsev Ibrahim Arshad Chuan Li Chauncey Kunshang Ji Ganesh R Lucas Wilkinson Robert Shaw Kyungmin Lee Ronen Schaffer Srreyansh Sethi vnadathur vnadathur Isotr0py Elham Yan Ma Nick Hill Chaofan Wang Chendi.Xue Injae Ryou Richard Zou milesial Elvir Crnčević Claude Sonnet 4 Hexiang Wang Lalithnarayan Cgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Luka Govedič PatchyTIS PatchouliTaisa jatseng-ai Matthias Gehre xaguilar-amd Ravitez Dondeti Nicolò Lucchesi Peter Nguyen wang.yuqi zhrrr Jee Jee Li TJian Jesus Federico Manu zhanqiuhu yzong-rh Fynn Schmitt-Ulms Rahul-Tuli Harry Mellor Benjamin Chislett Tianyu Guo Lee Yongjun z1yingLi, Jiang <jiang1.li@intel.com>Vibhav Agarwal vibhav-agarwal ShubyM Wei Zhao Itay Etelis Itay Etelis EdalatiAli Andreas Karatzas r266-tech Roger Wang Martin Hickey Or Ozeri Mark McLoughlin Le Yang Animesh Jain Yongye Zhu Zhengxu Chen Kaicheng Yang maobaolong sihao_li Flora Feng Yufeng He zofia Tihomir Elek Yi Liu Wentao Ye Santino Ramos haosdent JartX Yuyi Ao Tyler Michael Smith mukesh-hai Pedram Razavi
2026-04-15 10:32:47 +02:00
799973af4e [CI][NIXL] Fix PD CI breakage: pin nixl-cu{12,13} versions (#39851) zhanqiuhu 2026-04-15 02:50:23 -04:00
bcc2306cef [Bugfix] Respect VLLM_WEIGHT_OFFLOADING_DISABLE_PIN_MEMORY in prefetch offloader (#37699) Yufeng HeandIsotr0py 2026-04-15 11:43:29 +08:00
3abf858443 [Test] Refactor hard coded device string in test files under compile/quantization/models/model_executor folders (#38901) wliao2 2026-04-14 20:02:35 -07:00
f4b42df048 [Attention Backend] TurboQuant: 2-bit KV cache compression with 4x capacity (#38479)
Vibhav Agarwal Xinyu Chen Michael Goin
2026-04-15 08:27:13 +05:30
3bfe55a037 [Model Runner V2] Disable piecewise cudagraph mode fallback for eagle draft decodes (#39773) Giancarlo Delfin 2026-04-14 17:47:57 -07:00
b569620f72 [CI] Add PyTorch nightly build and test pipeline (#37226) Andrey Talman 2026-04-14 20:13:24 -04:00
65b9808960 [Bugfix] Disable FlashInfer CUTLASS MoE on SM121 (DGX Spark) (#39825) Michael GoinandClaude 2026-04-14 19:03:57 -04:00
507df79a29 [Hybrid] Simplify accepted token counting in spec decode for hybrid models (#38372) Francesco Fusco 2026-04-15 00:19:09 +02:00
1696c864b9 [Bugfix][Mooncake] Fix thread-local CUDA context for NVLink transfers in _send_blocks (#39548) Zhewen LiandZhewen Li 2026-04-14 14:13:58 -07:00
2ad1029233 [Bug] Fix batch invariance nvfp4 support (#39820) Wentao Ye 2026-04-14 17:08:17 -04:00
b2f749dc97 fix(lmcache): correct store for cached requests while enable prefix cache (#39719) maobaolong 2026-04-15 04:51:27 +08:00
70ed01550c [Reasoning][Frontend] Add model config to adjust_request in reasoning parser (#37848)
rishitdholakia13mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Aaron Pham
2026-04-15 00:29:06 +04:00
19ec9a0a62 [MoE Refactor] Refactor ZeroExpertFusedMoE into new framework (#35549) bnellnm 2026-04-14 16:11:20 -04:00
1a9353bb02 [MoE] Move GPT OSS Triton kernel experts into fused_moe/experts/ (#39007) Jackmin801andRobert Shaw 2026-04-14 12:27:39 -07:00
ecf5ff7ce3 [Mamba] Flashinfer selective_state_update (#36162) roikoren755 2026-04-14 22:10:58 +03:00
30679319e8 [CI][KVConnector][Metrics] Update multi KV connector edge case according to prefill stats changes (#39808) zhanqiuhu 2026-04-14 14:59:15 -04:00
240f2636ca [Kernel] Support TRTLLM GEN NVFP4 MoE for non-512-aligned hidden dims via weight padding (#39510) danielafrimiandroot 2026-04-14 21:49:56 +03:00
dc8df110bc add warning when FP8 KV cache misses prefill query quantization (#39752)
Albert Cheng Albert Cheng Michael Goingemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-14 11:43:05 -07:00
be0c855ebd [KV Offload] Unified memory layout for offloading workers (#37206) omerpaz95andOr Ozeri 2026-04-14 21:33:33 +03:00
e64b39ea71 [ROCm] Align AiterFlashAttentionImpl attn_type check with backend (#39119) Andrew Barnes 2026-04-14 13:36:26 -04:00
2faad08362 [compile] Nest inductor cache under AOT compile dir (#39718) Alessandro Sangiorgi 2026-04-14 12:17:54 -05:00
23f3760217 [Bugfix][ROCm]: Allow gpt_oss_mxfp4 quantization method on rocm (#39754) Rohan Potdar 2026-04-14 10:10:04 -07:00
906a8c15d0 [Core][Metrics] Remove unused SchedulerStats.encoder_cache_usage (#39693) Mark McLoughlin 2026-04-14 17:53:57 +01:00
4f4f8eaa78 [ROCm][CI] Fix condition for test_per_token_group_quant_fp8_packed (#39730) Micah Williamson 2026-04-14 11:14:31 -05:00
b6890a120a Bugfix: use_existing_torch.py: Glob recursive subdirs in requirements (fixes #39024) (#39793) Netanel Haber 2026-04-14 18:11:46 +03:00
c08f3b2a62 Measure encoder compile time seperate from llm backbone (#39240) Lucas Kabela 2026-04-14 07:52:49 -07:00
f02b3269e7 [PluggableLayer][3/N] Apply PluggableLayer to moe-related layers. (#33556) Hexiang Wang 2026-04-14 21:55:00 +08:00
e1e318af01 [MoE Refactor] Remove MoE DP chunking (#39107) bnellnmandRobert Shaw 2026-04-14 09:48:05 -04:00
f7e62e3d66 [Bugfix] Fix mismatch between global and local attention heads in tensor-parallel mode for param2moe model (#39707) bhargav-patel-29andgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-04-14 17:43:36 +05:30
18b1c77211 fix: handle ImportError in load_audio (#39473) Yiyang "Ian" LiuandCopilot 2026-04-14 04:09:06 -07:00
1e4748c66a [Bugfix] Fix vllm bench serve to count multimodal tokens in "total input tokens" (#38654) Matthias Gehre 2026-04-14 13:00:40 +02:00
6f786f2c50 [Bugfix][Model] Fix Devstral Small 2 HF format weight loading (#39293)
Thomas thomasmaindron Claude Opus 4.6
2026-04-14 12:11:18 +02:00
4eee77b877 [fix][MOE] Fix MOE experts intermediate_size dimension not being narrowed before weight loading (#39688) fxmarty-amd 2026-04-14 11:35:28 +02:00
a1993b96fd [XPU][CI] Remove Arc in label-xpu (#39776) xiangdong 2026-04-14 17:27:38 +08:00
893b2affff feat: add TxtSlicesDataset to allow sampling slices from txt file for benchmarking (#30156) Julien Debache 2026-04-14 11:20:03 +02:00
80118853f4 [MM][Perf][CG] Support ViT full CUDA graph for Qwen3-VL video inference (#38061) Shanshan ShenandRoger Wang 2026-04-14 16:49:32 +08:00

... 74 75 76 77 78 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86