Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

6290470843 [Bugfix] Fix dtype mismatch in RMSNormGated.forward_native() during torch.compile (#35256) haosdent 2026-03-02 04:14:46 +08:00
72f4d16262 [Model Runner V2] Use block table apis for capture inputs (#35671) Woosuk Kwon 2026-03-01 10:31:13 -08:00
5a435507d8 fix(mxfp4): return is_monolithic=False when LoRA is enabled for Triton backend (#35382) Seungho Yoon 2026-03-01 23:59:30 +09:00
59d7af9c6c [MISC] Fixing a null reference by removing parallel_utils from mypy EXCLUDE (#35630) Taneem Ibrahim 2026-03-01 08:26:44 -06:00
bbf81f9a92 [Mamba1] - Kernel Level Chunk Alignment for Prefix Caching (#34798) Asaf Gardin 2026-03-01 14:40:23 +02:00
da543d1abe [Model Runner V2] Minor refactoring for EncoderRunner (#35628) Woosuk Kwon 2026-03-01 00:15:39 -08:00
87d319c52f [AMD][CI] Support Triton attention with ExampleConnector (#34931) Ryan Rock 2026-03-01 01:58:07 -06:00
a9ec392c86 Fix typo: implictly -> implicitly in isaac.py docstring (#35646) lin-shh 2026-03-01 02:34:37 -05:00
afd089f231 [Bugfix][Model] Fix Qwen3.5/Qwen3Next ignoring --dtype flag on older GPUs (#35617) lailoo 2026-03-01 11:27:37 +08:00
3ecd0bf9fc Add TMA support to fused_moe_lora kernel (#32195) gnovackandJee Jee Li 2026-02-28 18:55:25 -08:00
e3eb146f7a [Model Runner V2] Add ModelStateInterface [4/N] (#35621) Woosuk Kwon 2026-02-28 13:19:45 -08:00
95a395dbec [Bugfix] Fix Anthropic API base64 image handling in Messages endpoint (#35557) Martin Vit 2026-02-28 21:57:08 +01:00
e94b263bd6 [Chore] Cleanup BNB utilization dead code (#35620) Isotr0py 2026-03-01 03:22:41 +08:00
e113a30113 [Deprecation] Deprecate code in 0.17 as scheduled (#35441) Wentao YeandCyrus Leung 2026-02-28 12:32:37 -05:00
1dafb29f91 [Benchmark] Avoid unnecessary video download in MMVU (#35618) Cyrus Leung 2026-03-01 01:07:02 +08:00
49b9ae32e9 [Fix] Avoid sending image input to other PP ranks (#35405) emricksini-handRoger Wang 2026-02-28 17:14:29 +01:00
63d7972f13 Fix Qwen3_5MTP packed_modules_mapping for gate_up_proj (#35581) cwazai 2026-02-28 22:50:55 +08:00
c68e69f144 custom dataset img support base64 (#35280) flutist 2026-02-28 19:49:52 +08:00
7e08c22b8c [Feat] Add CUDA torch fallbacks for fp8_mqa_logits/fp8_paged_mqa_logits_torch function (#35271) Chauncey 2026-02-28 18:12:00 +08:00
8e75d88554 add io_process_plugin for sparse embedding (#34214) Augusto YaoandCyrus Leung 2026-02-28 17:16:37 +08:00
0892d1ab1f [Feature]Supports Anthropic Thinking Block (#33671)
Mario Hong zetaohong Chauncey
2026-02-28 17:02:33 +08:00
7600642eae Add padding support to wvSplitK solution for skinny GEMMs (#33762) Hashem Hashemi 2026-02-28 01:02:05 -08:00
1e69c04887 [ROCm][CI] Parametrize vision score tests across attention backends with per-backend tolerances (#35571) Andreas Karatzas 2026-02-28 02:59:26 -06:00
4292e3b807 [Benchmark] Improve UX of sweep scripts (#35600) Cyrus Leung 2026-02-28 16:36:02 +08:00
24d6ea8afd [Benchmark] Rename SLA Finder to Workload Explorer (#35586) Cyrus Leung 2026-02-28 15:31:55 +08:00
57c86c0741 [Misc] Change logging level from info to debug for tool parser import (#35575) Chauncey 2026-02-28 14:51:35 +08:00
06254d4cbb [CI] add trainer_send_weights for MockWeightTransferEngine (#35589) Chauncey 2026-02-28 14:47:43 +08:00
f5d1281c9d [ROCm][CI] Expose tests to AMD production CI and fix amdsmi heap corruption (#35071) Andreas Karatzas 2026-02-27 23:57:31 -06:00
94029ffaf0 [ROCm] Derive device capability from GCN arch string without CUDA init (#35069) Andreas Karatzas 2026-02-27 23:55:28 -06:00
88e8525f2e [ROCm][CI] Adding infiniband mappings for moriio tests (#35170) Andreas Karatzas 2026-02-27 23:53:28 -06:00
b2d8b422b2 [EPLB] Enforce sync eplb for NCCL-based all2all backend (#35212) Ilya Markov 2026-02-28 06:47:12 +01:00
1d5ab5d603 [Bugfix] Move chat completion response_format validation to Pydantic model_validator (#35510) Umut Polat 2026-02-28 08:26:19 +03:00
7b346ba8ed [Bugfix] Propagate compilation_time from workers to main process for TP>1 (#35503) Huy Do 2026-02-27 21:03:22 -08:00
dea268336f [1/N] Elastic EP Milestone 2 (#34861)
Itay Alroy Yongji Wu Tyler Michael Smith Ron Tourgeman
2026-02-28 06:46:42 +02:00
90805ff464 [CI/Build] CPU release supports both of AVX2 and AVX512 (#35466) Ma Jianandjiang1.li 2026-02-28 12:35:21 +08:00
2562e0271e [MTP] Validate that MTP weights are actually loaded (#35548) Matthew Bonanni 2026-02-27 23:27:40 -05:00
fd68cd132b [Bugfix] Fixes for SLA finder (#35537) Cyrus Leung 2026-02-28 12:20:55 +08:00
0edf101d2b [ROCm] Add stablelm Head Size 80 To Supported Head Sizes For ROCM_ATTN (#35527) Micah Williamson 2026-02-27 22:16:34 -06:00
d5b6f3ba36 [ROCm][Quantization] Add Composable Kernel (CK) backend support for M… (#34301)
Douglas Lehr Doug Lehr Cursor Rohan Potdar
2026-02-27 21:37:01 -06:00
1a014a0a93 [Model Runner V2] Move MM encoder to Model States [3/N] (#35564) Woosuk Kwon 2026-02-27 18:32:38 -08:00
86ac7bcf84 [Model Runner V2] Support pooling models (#35120) Woosuk Kwon 2026-02-27 18:03:01 -08:00
405f28d38d [Misc] Clean up ResponsesRequest model validators (#35531) Umut Polat 2026-02-28 04:19:21 +03:00
5323672bc2 [misc] cleanup one level of error stack when nixl fails to initialize (#35517) youkaichao 2026-02-28 08:42:37 +08:00
a201ad72d8 [Refactor][Kernel] Add global helper to deduplicate vectorized memory ops (#35105) Roberto L. Castro 2026-02-28 01:28:17 +01:00
e3691988d0 [ROCm]: fix aiter rope functionalization (#35533) Rohan Potdar 2026-02-27 16:42:30 -06:00
9fa6c68fa6 [ROCm] Enabling encoder and encoder-decoder on ROCm and AITER unified backends (#35334) Gregory Shtrasberg 2026-02-27 15:32:55 -06:00
2ce6f3cf67 [Feat][RL][2/2] Native Weight Syncing API: IPC (#34171) Aaron Hao 2026-02-27 12:45:21 -08:00
1f3dbd95fd [Bugfix][Model] Fix gpt-oss batch invariance (#35404) Jakub Zakrzewski 2026-02-27 21:41:24 +01:00
1d532f9d8f [DP] Only use DP padding when cudagraphs are actually used (#34102) Lucas Wilkinson 2026-02-27 15:14:31 -05:00
234a65b781 [Bugfix] Add monkeypatch to prevent race condition from writing (#35420) Lucas Kabela 2026-02-27 11:51:36 -08:00
2decec9856 [Transformers backend] Ignore MTP weights when num_nextn_predict_layers=0 (#34888)
SteadfastAsArt Claude Opus 4.6 Harry Mellor
2026-02-28 03:39:23 +08:00
29b35477b0 [compile] Fix caching error over pytree slice node. (#35308) Zhengxu Chen 2026-02-27 14:34:16 -05:00
b1d9f5372d [Model Runner V2] Warmup kernels (#35172) Nick Hill 2026-02-27 10:43:30 -08:00
fd6de37fca [BugFix] Fix 3D rope in transformers backend (#35097) Raushan TurganbayandHarry Mellor 2026-02-27 19:34:49 +01:00
c8aca0c9e1 Support parakeet as audio encoder for nemotron-nano-vl (#35100) Netanel HaberandRoger Wang 2026-02-27 20:07:38 +02:00
b602e4f299 [Doc] Fix link to Llama chat template for usability (#35525)
Martin Hickeygemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
2026-02-27 17:51:09 +00:00
157722da75 [perf] Use pinned memory for async H2D transfer in do_mamba_copy_block (#35480) Huamin Li 2026-02-27 09:50:37 -08:00
1d897ff04f [Misc] Fill in some v1 CODEOWNERS gaps (#35524) Nick Hill 2026-02-27 09:34:37 -08:00
905d76b51d [Model] Add huggingface skt/A.X-K1 model (#32407)
fort726 Sungwan Kimgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>TJian
2026-02-28 02:26:02 +09:00
9098ce690c [Kernel] [Helion] [7/N] Use HOP to represent Helion Kernel call to enable fx tracing and pattern matching (#34390) Yanan Cao 2026-02-27 09:21:35 -08:00
876312f0b5 [Core] Fix gpu_worker.py pre-commit errors (#35312) Nick Hill 2026-02-27 07:54:24 -08:00
5de98abc12 Add @BoyuanFeng to CODEOWNERS (#35317) Boyuan Feng 2026-02-27 07:53:47 -08:00
9251ed5c4f [Bugfix] Handle case when kimi ends reasoning with a tool call (#33646)
Koushik Dutta mondaylord Chauncey Roger Wang
2026-02-27 06:58:28 -08:00
e8249378e4 [Bugfix] Fix check_interleaved_audio_video false positive for batched non-interleaved requests (#35487) Yueqian LinandRoger Wang 2026-02-27 09:48:25 -05:00
6d4f9d3ad5 [Bugfix] Fix DCP + FA3 crash due to missing num_splits in _forward_with_dcp (#35082) haosdent 2026-02-27 22:27:06 +08:00
fbe3f0120a Revert "Add GlmOcrConfig for GLM-OCR model type recognition" (#35512) Harry Mellor 2026-02-27 14:13:27 +00:00
66c1751d13 [compile] Cleanup: Remove unnecessary +rms_norm forcing for sequence parallelism (#35410) Jason Li 2026-02-27 05:36:37 -08:00
6467b635b6 [Bugfix] Add missing activation attr to RMSNormGated (#35423) TibandtibG 2026-02-27 13:53:35 +01:00
9c3fe9936b Flashinfer cuDNN backend for Qwen3 VL ViT attention (#34580)
Max Hu Max Hu Shang Wang
2026-02-27 20:20:23 +08:00
b66a74649e [Bugfix] Replace assert with ValueError for response_format validation in completions endpoint (#35456) Umut Polat 2026-02-27 11:01:06 +03:00
07bdabef03 [Bugfix] Use 'sum' reduction instead of 'avg' in Async TP reduce-scatter (#33088) Wang XingranandHongjian Zhang 2026-02-27 15:06:08 +08:00
a572baff5e [Model Performance] Add Qwen3MoE tuned MoE configs for H200 (#35457) Chengyi NieandChengyi Nie 2026-02-26 21:51:14 -08:00
516cf26698 [Bug] correct out dtype of rms_norm_gated native path (#35369) zofiaandKunshang Ji 2026-02-27 13:19:51 +08:00
487e5c51f7 [Bugfix] disable allreduce_rms_fusion by default when pp size > 1 (#35424) Jiangyun Zhu 2026-02-27 12:18:52 +08:00
1a8c71674e [BugFix] Repo utils debug print patch (#35434) Daniel Huang 2026-02-26 19:50:56 -08:00
062b789632 [Bug] Fix outdated links in source code (#35314) Wentao Ye 2026-02-26 22:50:46 -05:00
a532c83849 use 'max_active_experts' for moe lora input size (#33197) gnovack 2026-02-26 19:50:43 -08:00
1e5ad9b74f [Bugfix] Fix Qwen3NextForCausalLM packed_modules_mapping (#35413) Jee Jee Li 2026-02-27 11:46:30 +08:00
cabdaa7619 [Misc] Move GPUModelRunner.prepare_kernel_block_sizes to utils (#35400) Nicolò Lucchesi 2026-02-27 04:42:51 +01:00
06be53563b [Core]Extract is_last_rank in Ray for tpu to override (#33012) Chenyaaang 2026-02-26 19:18:52 -08:00
c29ee9c326 [compile] Invalidate cache for cpu flags (#35119) Angela Yi 2026-02-26 18:54:11 -08:00
d43048ce05 [Bugfix] Emit reasoning_part events in simple streaming path for Resp… (#35184) daniel-salib 2026-02-26 17:49:06 -08:00
4fec53cfcb [CI] Actually run tests/kernels/quantization/test_block_fp8.py in CI (#34274) Michael Goin 2026-02-26 19:58:03 -05:00
38c498b8e3 [Performance] Cublas Bf16 Gate with Fp32 Output (#35121) roikoren755 2026-02-27 02:51:28 +02:00
56a6371706 [Update] Use FlashInfer fast_decode_plan directly instead of replication (#34687) Andrii SkliarandAndrii 2026-02-27 01:31:43 +01:00
6283021142 [Bugfix] Fix KV Scale loading for MLA Models (#35430) Pavani Majety 2026-02-26 15:38:19 -08:00
01923eec70 [ROCm][Quantization] GPT OSS Upstream MoE wmxfp4_afp8 with static scales (#30357) Aleksandr MalyshevandAleksandr Malyshev 2026-02-26 14:50:16 -08:00
31fb6f43da [Kernel][perf] optimize NCCL symm_mem vs custom_AR selection thresholds (#33839)
pkousha Pouya Koushagemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Michael Goin
2026-02-26 14:35:58 -08:00
eb19955c37 [WideEP] Remove pplx all2all backend (#33724) Tyler Michael SmithandClaude Opus 4.6 2026-02-26 17:30:10 -05:00
0f2f24c8b2 [Bugfix] Fix MessageQueue connect_ip for cross-node data parallelism (#35429) Lucia FangandLu Fang 2026-02-26 14:08:16 -08:00
d0105b84f0 add mixed precision support for modelopt (#35047) sychen52 2026-02-26 13:56:24 -08:00
832a780f3a Nemotron: use per-layer config in NemotronHMLPDecoderLayer for heterogeneous models (#35396) danielafrimi 2026-02-26 23:55:19 +02:00
98217b09f9 [Performance] Extract KV cache update op from flashinfer forward (#35422) ElizaWszola 2026-02-26 22:29:01 +01:00
967572dd5f fix(reasoning): Qwen3ReasoningParser returns truncated output as reasoning (#35230) 不做了睡大觉andstakeswky 2026-02-27 04:30:45 +08:00
3d66502e1b [Model Runner V2] Prepare attn metadata in ModelState [2/N] (#35383) Woosuk Kwon 2026-02-26 11:47:02 -08:00
c66aa48e99 [Model Runner V2] Add model states [1/N] (#35350) Woosuk Kwon 2026-02-26 11:20:35 -08:00
b6d5a17298 [Model Runner V2] Fix error-handling (#35063) Nick Hill 2026-02-26 11:00:19 -08:00
5e58bdc711 [Bugfix] Remove erroneous lower bound on LoRA vocab size constraint (#35354) Lucas Wilkinson 2026-02-26 13:44:50 -05:00
a1f53addb1 [BugFix] Align fused MoE-LoRA kernel config with actual weight shapes (#34396) Runkai Tao 2026-02-26 13:03:10 -05:00
05970c772c [Refactor] Remove dead code for attention benchmark script (#35418) Wentao Ye 2026-02-26 12:53:46 -05:00

... 89 90 91 92 93 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86