iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,845 Commits 3 Branches 0 Tags
78300cdabffbaaffe7625ed34412ec2aec75535c
Commit Graph
20845 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yufeng He de111f3246 [Bugfix] Fix bench_serve UTF-8 decode crash on split multi-byte chars (#38732) 2026-04-16 12:01:25 -07:00
Jared Wen afabb5f45a [bugfix] Normalize tool message content from array to string format (#39899) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
 2026-04-16 11:54:39 -07:00
Roger Wang 3abb7560c0 [Bugfix] Fix audioflamingo test (#40052) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-04-16 11:53:58 -07:00
Isotr0py 617d1c2ff1 [Misc] Move pyav and soundfile to common requirements (#39997) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-04-16 08:52:37 -07:00
Nikita Shapovalov 692db29cd4 [Bugfix] Fix Ray compiled-DAG SHM channel stalls by detaching zero-copy np.ndarray logprobs buffers (#35736) 
Signed-off-by: Nikita Shapovalov <nikita@poolside.ai>
 2026-04-16 23:49:29 +08:00
Isotr0py 82531edbfb [Refactor] Remove resampy dependency (#39524) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-04-16 08:48:17 -07:00
Nicolò Lucchesi 3daca38e22 [Misc] toy_proxy_server handle min_tokens (#39706) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-04-16 15:08:22 +00:00
daiyu1111 a302a8fd1b [Bugfix] Fix LLM priority normalization for single-string prompts (#40011) 
Signed-off-by: daiyu1111 <2356690121@qq.com>
 2026-04-16 07:56:06 -07:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 4e8c3f1c19 [Frontend][last/5] Improve pooling entrypoints | clean up. (#39675) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-16 07:53:23 -07:00
Vasiliy Kuznetsov 5e5afafa21 [Doc] add docs for online quant frontend (#39736) 
Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com>
 2026-04-16 07:52:58 -07:00
Li, Jiang 324a3d2bd8 [CI/Build] Improve stability of CPU tests (#39966) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-04-16 21:50:36 +08:00
grYe99andgrYe99 4269b79409 [Model] Use mm_features to compute mrope positions for PaddleOCR-VL (#39888) 
Signed-off-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: grYe99 <guorongye99@gmail.com>
 2026-04-16 06:14:00 -07:00
Yanan CaoandClaude Opus 4.6 edc3648966 [Kernel][Helion] Fix inductor fusion of Helion HOP (#39944) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-16 04:41:26 -07:00
Nicolò Lucchesi 9965f501a8 [Nixl] Bump Nixl version to 0.10.1 (#39922) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-04-16 11:53:21 +01:00
lalit10 17d87168d2 [Model] Use mm_features for Keye-VL and Keye-1.5-VL M-RoPE (#39869) 
Signed-off-by: Lalit Laxminarayan Bangad <lalitbangad@gmail.com>
 2026-04-16 02:16:06 -07:00
Netanel Haber 98700c6105 Fix #33773: Replace unconditional pandas import with PlaceholderModule (#39990) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2026-04-16 02:06:51 -07:00
Simon Mo 10e49d2638 [Docs] Update PR template to remove release notes google docs (#39982) 
Signed-off-by: Simon Mo <simon.mo@hey.com>
 2026-04-16 00:22:03 -07:00
Tim Messerschmidt 8d7c962833 [Bugfix] Accept **kwargs in MiniMaxM2Parser.__init__() (#39861) 
Signed-off-by: Tim Messerschmidt <timmesserschmidt@gmail.com>
 2026-04-16 15:18:32 +08:00
Xinyu ChenandKunshang Ji f4ddaf8cf7 [XPU] use spawn multiproc method on xpu (#39671) 
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-04-16 14:42:07 +08:00
Abhijit Royandwang.yuqi 2cdf86044d Add Jina Embeddings v5 model support (fixes #38633) (#39575) 
Signed-off-by: Abhijit <abroy@redhat.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-16 06:37:10 +00:00
realliujiaxu 7845379230 [Bugfix] add support for 'num_attention_groups' in ModelArchConfigConvertorBase for Step3p5 (#39796) 
Signed-off-by: realliujiaxu <realliujiaxu@163.com>
 2026-04-16 05:48:00 +00:00
R3hankhan 4b7ca37bd4 [CPU][IBM Z][Dockefile][Docs] Fix s390x builds for torch 2.11 and update docs for s390x (#39910) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
 2026-04-15 22:26:21 -07:00
Fadi Arafehandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 445b7093fd [perf][cpu] Accelerate BF16 GELU with LUT impl on Arm CPUs (#37469) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-15 22:26:17 -07:00
jigangzandClaude 18013df6ae [Bugfix] Reject empty tools array with HTTP 400 (#39780) 
Signed-off-by: Jigang Zhou <zjg0907008@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-16 12:08:04 +08:00
Julien Denize c0722f22de [Mistral Grammar] Fix tool and reasoning parsing (#39217) 
Signed-off-by: juliendenize <julien.denize@mistral.ai>
 2026-04-15 21:05:04 -07:00
Zhengxu Chen 951dca8019 [compile] Invoke split FX graph by codegen. (#38657) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
 2026-04-15 21:03:41 -07:00
vllmellm 5f7fab881a [ROCm][FEAT] Integrate aiter gemm w8a8 ptpc (#33773) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2026-04-16 09:55:29 +08:00
Giancarlo Delfin 343f65234b [Model Runner V2][BugFix] fix num_sampled dtype for probabilistic rej… (#39951) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-04-15 18:09:11 -07:00
Asaf Gardin 19fa90ed0d [Quantization] - Layerwise reloading of Attention/KV quantized models (#38995) 
Signed-off-by: Josephasafg <ajgard7@gmail.com>
 2026-04-15 18:03:32 -07:00
Harry MellorkhluuCyrus Leungjiang1.li
 		03f8d3a548 Update to transformers v5 (#30566) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: khluu <khluu000@gmail.com>
Signed-off-by: Kevin H. Luu <khluu000@gmail.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
 2026-04-15 16:29:15 -07:00
Luciano MartinsandLuciano Martins 6dc9491406 [Model] Fix Gemma 4 token repetition by dynamic BOS injection for PT models (#39842) 
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
 2026-04-15 16:13:07 -07:00
Collin McCarthy 27c0ca50a0 Update registry for Nemotron-v3 VL Nano/Super (#39747) 
Signed-off-by: Collin McCarthy <cmccarthy@nvidia.com>
 2026-04-15 16:09:11 -07:00
Wentao Ye 7c636432c6 [CI Bug] fix flaky test (#39938) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-15 17:20:06 -04:00
Matthew Bonanni c77e596e2e [FlashAttention] Don't overwrite flash_attn_interface.py when installing precompiled (#39932) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-15 16:43:15 -04:00
Benjamin Chislett ac3dac545b [Bugfix][Perf] Indexer upcast WK to BF16 for fusion (#38928) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-04-15 20:39:32 +00:00
Wentao Ye 39ac640490 [Bug] Fix batch invariant test issue, bs=1 with max_seq_num = 1 (#39320) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-15 16:28:43 -04:00
zhanqiuhu 0b790a2501 [Speculative Decoding] Add DFlash speculators config parsing (#38300) 
Signed-off-by: Zhanqiu Hu <zhu@redhat.com>
 2026-04-15 16:22:15 -04:00
zhanqiuhu 41488f2acd [Bugfix][NIXL] Fix _logical_to_kernel_block_ids conversion for non-mamba models (#39724) 
Signed-off-by: Zhanqiu Hu <zhu@redhat.com>
 2026-04-15 20:08:58 +00:00
Kevin H. LuuandClaude Opus 4.6 102d51c9f3 [CI] Only build release Docker images when NIGHTLY=1 (#39882) 
Signed-off-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-15 19:01:13 +00:00
Zhewen LiZhewen LiOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		55e1a8e103 [Mooncake] Fix mixed MLA+Eagle block-size validation (#39596) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-15 11:36:47 -07:00
Monishver 21e5a9f48e Bug/test eagle dp v2 (#39838) 
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
 2026-04-15 17:48:12 +00:00
Mark McLoughlin 8ad6ff0037 [Test] Fix @create_new_process_for_each_test("fork") in interactive shell pipeline (#29130) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-04-15 12:22:20 -04:00
daniebrillandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> f2145efcb6 [BugFix] KeyError on scope["method"] for realtime api websocket in AuthenticationMiddleware (#36934) 
Signed-off-by: daniebrill <50454544+daniebrill@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-15 16:15:01 +00:00
Roy Huang ed33310552 [KVConnector][LMCache] Propagate cache_salt through MP connector for per-user cache isolation (#39837) 
Signed-off-by: royyhuang <royyhuang@gmail.com>
Signed-off-by: royyhuang <roy.y.huang@gmail.com>
 2026-04-15 09:10:49 -07:00
Talor AbramovichandBenjamin Chislett 3cc328a4be [SpecDecode][Benchmark] Add SPEED-bench support to benchmarking CLI (#36029) 
Signed-off-by: talora <talora@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-04-15 12:00:07 -04:00
Yan MaandKunshang Ji 3beb57a238 [XPU] properly handle q_descale on XPU as quant query input not supported (#39676) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-04-15 21:52:58 +08:00
danielafrimiandroot 8b5531933a FIX: support language_model.backbone naming in NemotronH Nano VL quantization config (#39901) 
Signed-off-by: <>
Co-authored-by: root <root@lyris0144.lyris.clusters.nvidia.com>
 2026-04-15 13:49:48 +00:00
Chauncey db8d4a4a06 [BugFix][Graph] fix: handle empty sym_shape_indices in PiecewiseBackend. (#39395) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-15 09:28:09 -04:00
zofia fc701c8058 [XPU][MXFP4] add mxfp4 quant op for XPU (#39857) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
 2026-04-15 12:28:19 +00:00
Csrayz 68be0f853e [Metrics] Add request_id to FinishedRequestStats to enable correlation between metrics and requests (#39710) 
Enables external `StatLogger` plugins to correlate per-request metrics
with request-level context. Also, this is a pre-requisite for Prometheus
exemplars in #30972.

Signed-off-by: Csrayz <33659823+Csrayz@users.noreply.github.com>
 2026-04-15 11:24:17 +00:00