Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

c0ecaed950 [Frontend] Offload blocking preprocessing & postprocessing ops to thread pool for pooling entrypoints. (#39763) wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-04-14 16:29:25 +08:00
0008729abf [Model] Use mm_features for Ernie-4.5 VL M-RoPE (#39753) lalit10andCyrus Leung 2026-04-14 01:11:52 -07:00
d3af8c1831 [Core][Metrics][BugFix] Replace num_cached_tokens/num_external_computed_tokens with PrefillStats (#37460) Mark McLoughlinandOr Ozeri 2026-04-14 09:00:45 +01:00
25b3242d8b Fix Responses API streaming for multiple auto tool calls (#39626) noobHappylife 2026-04-14 13:28:43 +08:00
b075604da1 [Bugfix] Fix Gemma4 tool parser converting bare null to string "null" (#39679) 김의진andChauncey 2026-04-14 13:44:46 +09:00
db8a6d66bf [Refactor][Parser] Migrate chat completion auto-tool/reasoning/plain streaming to parse_delta (#39446) Flora Feng 2026-04-14 00:39:45 -04:00
d2130a47bb [Bugfix]: Fix MinimaxM2ToolParser missing tools parameter (#39683) Chauncey 2026-04-14 11:16:39 +08:00
c687bf226a [LMCache][MP] optimize save when mla enabled (#38810) chunxiaozhengandYihua Cheng 2026-04-14 08:56:43 +08:00
ccf90ba784 [Model Runner V2] Add full cuda graph support for eagle prefill (#37588) Giancarlo Delfin 2026-04-13 16:01:24 -07:00
6adacfcb65 ParakeetExtractor performance and UX enhancements (#39423) Netanel Haber 2026-04-14 00:37:35 +03:00
14cb86c187 [Refactor][Parser] Simplify parse_delta (#39728) Flora Feng 2026-04-13 17:02:13 -04:00
8213e8f880 Bug/test eagle dp v0 (#38938)
Monishvermergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nicolò Lucchesi
2026-04-13 16:50:08 -04:00
3693f922ff [Bugfix][Pooling] Fix silent weight corruption with buffer-reusing iterators (#39650) Pedram Razavi 2026-04-13 12:37:27 -07:00
5c18b961d6 [Core][Metrics] expose waiting request breakdown via labeled metric (capacity/deferred) (#38435)
mukesh-hai Claude Sonnet 4.6 Mark McLoughlin
2026-04-13 12:30:55 -07:00
f72b20976c [Bugfix] Reject non-nvfp4 dtypes when using the flashinfer_nvlink_one_sided all2all backend (#39717) Tyler Michael SmithandClaude Opus 4.6 2026-04-13 15:13:51 -04:00
610a3efcaf [Doc] Fix Python-only build 404 fallback guidance (#38052)
Yuyi Aogemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Harry Mellor
2026-04-13 15:09:31 -04:00
f414f90601 [Bugfix][Kernel][ROCm] Fix triton_w4a16 scales mismatch when BLOCK_K > group_size (#39705) JartX 2026-04-13 20:29:45 +02:00
8625ec267b [Misc] Multi-turn benchmark output performance json (#39572) Nicolò Lucchesi 2026-04-13 20:15:23 +02:00
995e9a209e [Bugfix] Use is_integrated to detect UMA GPUs for memory reporting (#35356) haosdentandDarkLight1337 2026-04-14 02:07:40 +08:00
739e5945dc [Quantization] [Refactor] Create special "GptOssMxfp4MoeMethod" (#39604) Yongye Zhu 2026-04-13 12:53:58 -04:00
4d042ed85f [Bugfix] Fix tensor shape mismatch in sparse attention with speculative decoding (#39542) Santino Ramos 2026-04-13 08:57:38 -07:00
10d9872d3a [CI][Metrics] Fix local_cache_hit assertion after prompt tokens metrics updates (#39709) zhanqiuhu 2026-04-13 11:16:56 -04:00
ccd0d1d906 [Bug] Fix rocm sparse attn indexer issue (#39225) Wentao YeandTJian 2026-04-13 10:53:45 -04:00
d8ddb31644 [Bugfix][CT] Fix KV cache scale handling (#39418) Yi Liu 2026-04-13 22:50:16 +08:00
1ce0318c68 [Bugfix] stream failure when model name not in audio endpoints (#36679) Ekagra Ranjan 2026-04-13 10:20:07 -04:00
8d825b87d6 [Bug] Fix TypeError when hf_config.architectures is None during model loading (#38849) Tihomir Elek 2026-04-13 13:13:21 +02:00
1b19bd7589 [MXFP8] [XPU] add a new compressed tensor schema and add a xpu mxfp8 gemm kernel (#38707) zofia 2026-04-13 16:59:20 +08:00
200a727e94 [Bugfix] Fix Responses API instructions leaking through previous_response_id (#37727) Yufeng HeandChauncey 2026-04-13 16:46:33 +08:00
edbc1abd1c feat: add max_tokens_per_doc in rerank request. (#38827)
Jesus Federico Claude Opus 4.6 wang.yuqi
2026-04-13 04:24:09 -04:00
0e39202ca9 [Bugfix] Fix GLM tool parser streaming with MTP or stream interval (#39253) Flora Feng 2026-04-13 01:10:30 -04:00
9dd5ee0117 [XPU]Enhance environment collection for Intel XPU and optimize layout (#35698) sihao_liandKunshang Ji 2026-04-13 12:51:46 +08:00
fa6ae31177 feat: rename logit_bias/logit_scale to logit_mean/logit_sigma for affine score calibration (#39530)
Jesus Federico Claude Opus 4.6 wang.yuqi
2026-04-13 00:43:44 -04:00
2a3c32ce67 fix(lmcache): correct store for cached requests and num_scheduled_tokens in lmcache_mp_connector.py (#39655) maobaolong 2026-04-13 11:29:19 +08:00
4beeb0689c fused qknorm+rope kernel optimization for SM9.0 (#37376) Kaicheng YangandClaude Sonnet 4.6 2026-04-13 10:58:37 +08:00
cae984060f [compile] Enable AOT compile with batch invariance mode. (#39201) Zhengxu Chenandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-12 22:58:33 -04:00
715681c127 [LoRA] Support dual CUDA streams-Linear Layer (#35721) Jee Jee Li 2026-04-13 10:57:07 +08:00
dc02271d76 [XPU] revert torch-xpu to 2.10 (#39656) Kunshang Ji 2026-04-13 10:50:29 +08:00
4e4ad41d11 [ROCm][CI] Removed stale tests and extended acceptance test (#39651) Andreas Karatzas 2026-04-12 21:40:26 -05:00
620e8924d9 [Bugfix] [Tests] Enforce out tensor device in kernel/moe/test_cutedsl_moe.py (#39644) Yongye Zhu 2026-04-12 20:08:08 -04:00
f00c5539d7 [compile] Bug fix for _decompose_size_nodes (#38360) Animesh Jain 2026-04-12 13:20:24 -07:00
21fab0a3db fix(moe): fix RoutedExpertsCapturer assertion failure with DP>1 and MK path (#37879) Le Yang 2026-04-12 22:28:17 +08:00
3244a2ebf2 [KVConnector][NIXL] Organize NIXL connector into its own directory (#39354) Nicolò Lucchesi 2026-04-12 15:10:50 +02:00
72ff142c37 [Core][Metrics] Remove vllm:prompt_tokens_recomputed metric (#38709) Mark McLoughlin 2026-04-12 10:22:01 +01:00
ee3c0c83db [Pooling] Disable async scheduling by default for pooling models (#39592) Nick Hill 2026-04-12 00:23:42 -07:00
cc07dad789 [HMA] [KVEvent] Enable GPU-side KV events for HMA (#37688) Martin HickeyandOr Ozeri 2026-04-12 08:01:02 +01:00
17e787a779 fix(kimi_k25): resolve media_placeholder_token_id from tokenizer (#39344) r266-techandRoger Wang 2026-04-12 12:10:24 +08:00
639402f5a2 Support FP8 KVCache on XPU (#37731) Xinyu ChenandKunshang Ji 2026-04-12 11:53:32 +08:00
0f7be0f2f7 [ROCm][CI/Build] Fix memory cleanup in MM test (#39555) Andreas Karatzas 2026-04-11 22:13:34 -05:00
394ff86965 [XPU][CT] support per-channel quantization in xpu fp8 linear method (#38316) Yan Ma 2026-04-12 10:46:28 +08:00
df1e30e74b [Quant] add CompressedTensorsW8A8Mxfp8 for linear and MoE layers (#38815) EdalatiAli 2026-04-11 19:21:36 -04:00
bd8bd52308 [Bugfix] Runtime driver check for cuMemcpyBatchAsync in swap_blocks_batch (#38919)
Itay Etelis Itay Etelis Michael Goin
2026-04-11 20:02:34 +03:00
59b2f7b640 [Perf] Fuse Zero Initializer for FP8 DeepGemm Block Quant Kernel (#39547) Wei Zhao 2026-04-11 10:16:51 -04:00
92feb9991d [Gemma4][Bugfix]: Enable Gemma4ForCasualLM to load lora adapters correctly (#38844) ShubyM 2026-04-11 02:06:49 -07:00
d4cb783c10 [Bugfix] Fix GDN FLA kernel crashes with NULL_BLOCK_ID=0 CUDA graph padding (#39064)
Vibhav Agarwal vibhav-agarwal Claude Opus 4.6
2026-04-11 14:05:19 +05:30
eb92ba740a [CI/Build] Fix sentence-transformers version in CPU test (#39557) Li, Jiang 2026-04-11 15:04:25 +08:00
a3e750c0a5 [Misc] Update deprecation warning for --model flag (#39518) z1ying 2026-04-10 23:25:20 -07:00
da72daced2 [Bugfix] add SupportsMultiModal to Exaone4_5_MTP (#39526) Lee Yongjun 2026-04-11 14:57:26 +09:00
8d0aabdde9 Fix the order of _free_encoder_inputs (#38907) Tianyu Guoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-11 13:47:48 +08:00
0f3ce4c74b [XPU] Fix spec-decode UTs under tests/v1/spec_decode (#38491) Yan MaandKunshang Ji 2026-04-11 09:31:00 +08:00
af661a182d Revert "Add nightly b200 test for spec decode eagle correctness (#38577)" (#39512) Benjamin Chislett 2026-04-10 20:07:32 -04:00
7f0b8f2020 [Docs] Use --torch-backend=auto for editable install docs (#39511) Michael Goin 2026-04-11 00:27:02 +02:00
11e2375fe2 [Refactor] Move MXFP8 GEMM management into MxFp8LinearKernel (#39205) Michael Goin 2026-04-10 23:02:03 +02:00
fc645f1acc Add structure to requirements/ directory (#39024) Harry MellorandRobert Shaw 2026-04-10 21:46:41 +01:00
2d80cf9d6e Fix pre-commit labeled trigger system (#39523) Fynn Schmitt-Ulms 2026-04-10 15:54:49 -04:00
e7cfd7c5b9 Add Gemma4 Eagle3 support (#39450)
Fynn Schmitt-Ulms Rahul-Tuli Claude
2026-04-10 15:35:35 -04:00
e816a8811f [Bugfix] Fix FlashInfer crash with kv_cache_dtype_skip_layers (#39002) yzong-rh 2026-04-10 14:50:47 -04:00
e281cb721c [CI] Add MultiConnector (Nixl+Offloading) e2e edge case tests (#39343) zhanqiuhu 2026-04-10 13:35:03 -04:00
51cfc0e76c perf(moe): add tuned fused_moe config for RTX PRO 6000 Blackwell Server Edition (#39183) Manu 2026-04-10 19:32:42 +02:00
b87575d24b feat: add logit_scale to PoolerConfig for affine score calibration (#39435) Jesus FedericoandClaude Opus 4.6 2026-04-10 13:21:14 -04:00
42c6bb4b75 [ROCm] [AITER] Revert AITER version to v0.1.10.post3 (#39509) TJian 2026-04-11 00:25:52 +08:00
ecd1ea1363 [Kernel] Porting the TRTLLM minimax_allreduce_rms kernels (#37045) Jee Jee Li 2026-04-11 00:20:20 +08:00
8f121f7879 [Model Runner V2] support auto resolve cudagraph mode/sizes based on attn backend (#32936) zhrrr 2026-04-10 23:27:15 +08:00
cb5f7501cb [New Model]: jinaai/jina-reranker-v3 (#38800) wang.yuqi 2026-04-10 23:20:40 +08:00
8d0f908b98 [Model] Implement LoRA support for Qwen3ASRForConditionalGeneration (#37247) Peter Nguyen 2026-04-10 07:34:31 -07:00
c9dddc144b [CI] Add Nixl+OffloadingConnector e2e integration tests (#39200) Nicolò Lucchesi 2026-04-10 15:40:40 +02:00
c1cc7344fb [ROCm] Add RDNA 3.5/4 device IDs (gfx1150, gfx1151, gfx1201) (#38455) Ravitez DondetiandClaude 2026-04-10 06:35:07 -05:00
f976e3b98b [Performance] Remove unnecessary zero-fill of MLA decode output tensor in Aiter backend (#37539) xaguilar-amd 2026-04-10 13:27:35 +02:00
d468322dc1 [Kernel][Hardware][AMD] Add TritonW4A16LinearKernel for ROCm (#37352)
jatseng-ai Claude Matthias Gehre
2026-04-10 03:25:27 -07:00
967146e7bd [model] support FireRedLID (#39290) PatchyTISandPatchouliTaisa 2026-04-10 16:43:58 +08:00
8e8a3becd1 [ZenCPU] Make PT Backport Patch Accessible to vLLM (#38205)
Lalithnarayan Cgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Luka Govedič
2026-04-10 13:59:35 +05:30
1dfd64c1cc [PluggableLayer][3/N] Apply PluggableLayer to llm_head and vocab embedding layer (#33465) Hexiang Wangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-10 16:12:59 +08:00
ad720aefe9 [Bugfix] Fix V1 dummy run writing NaN to KV cache null block (#39444) Elvir CrnčevićandClaude Sonnet 4 2026-04-10 10:09:46 +02:00
270e8a4102 Nemotron Nano VL: Streamline pixel shuffle (#37580) milesial 2026-04-10 00:31:19 -07:00
f44afef6d6 [compile] Allow strings in custom ops without regressing compilation times (#38123) Richard Zou 2026-04-10 09:26:37 +02:00
447ce22212 [GGUF] Support non-standard quant types with prefix (e.g. UD-IQ1_S) (#39471) Injae RyouandIsotr0py 2026-04-10 16:22:53 +09:00
65e4e46f66 update CODEOWNERS file (#39439) Chendi.Xue 2026-04-10 02:05:31 -05:00
49d20346e4 [Perf] Reduce H2D pageable memory copies (#38794) Chaofan WangandIsotr0py 2026-04-10 15:03:26 +08:00
ef076c1b73 [Core] Change max_model_len in EngineCoreReadyResponse to be non-None (#39442) Nick Hill 2026-04-09 23:34:57 -07:00
ec68d53b2b Add platform manual_seed_all API (#38468) Yan Ma 2026-04-10 13:43:50 +08:00
13e6b1b908 [BugFix][CPU] Add CPU profiler summary file output (#38366) Elham 2026-04-10 01:41:15 -04:00
58c0a928c9 [Bugfix] Fix broken explicit unquantized kv cache dtype support (#38922) Isotr0py 2026-04-10 13:27:53 +08:00
3dd60971de [feat]: make DCP error msg clearer (#28443)
Srreyansh Sethi vnadathur vnadathur
2026-04-09 22:27:22 -07:00
a5b17fba8f [KV Offload] Implement shutdown() in OffloadingConnector and related classes (#39182) Ronen Schaffer 2026-04-10 08:06:22 +03:00
c48b2b83bd [Mergify] Update model vendor auto-label rules (#39312) Cyrus Leung 2026-04-10 12:25:37 +08:00
e7a1387e73 Add EXAONE-4.5 (#39388) Kyungmin Leeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-10 12:53:26 +09:00
f83de7196f [BugFix] Fix OOB read in CUTLASS grouped GEMM with epilogue (#38571) Lucas WilkinsonandRobert Shaw 2026-04-09 23:52:52 -04:00
445a2a4d1a feat(cpu): add CPU support for draft model speculative decoding (#32662) Ganesh R 2026-04-10 09:19:52 +05:30
55d037e2e5 [CT][FP8][Marlin] refactor CompressedTensorsW8A16Fp8 to use kernel abstraction (#38244) Kunshang Ji 2026-04-10 09:58:35 +08:00
ecbfbb8d61 [Feature] Add auto-detection for reasoning_config when only reasoning_parser is set (#38214) Chauncey 2026-04-10 09:36:26 +08:00
e0613702ad [ROCm] Fix AITER ops fake impl and minor bugs (#36092) Chuan (Richard) Li 2026-04-09 17:56:17 -07:00

... 75 76 77 78 79 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86