Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

f95c11a848 [Feat] dflash support for ROCm (#39703) hangy-amd 2026-04-21 14:58:20 +08:00
257015d5e5 [MoE] Triton MoE Perf regression - restore low latency path (#39016) milesial 2026-04-20 23:37:11 -07:00
b47840019e [MM][Misc] Support image+video mixed inputs (per prompt) for VLM examples (#40335) Shanshan Shen 2026-04-21 11:43:25 +08:00
989cc12d88 [Fix] Add missing space in IP fallback warning (#40359) SeongJun Lee 2026-04-21 12:26:06 +09:00
301024aa9c [Deprecation] Deprecate cprofile and cprofile_context (#39100) Wentao Ye 2026-04-20 23:25:22 -04:00
8256833fe6 [Startup] Parallelize torch/transformers import + weight prefetch + forkserver prewarm (#40331) Simon Mo 2026-04-20 19:49:32 -07:00
8097591286 [Doc] Update ViT CUDA graph doc for mixed (image+video) inputs (#40355) Shanshan Shen 2026-04-21 10:31:09 +08:00
20d3743491 [Bugfix] Gemma4: fix multimodal embedder norm order to match HF reference (#40411) Luciano MartinsandLuciano Martins 2026-04-20 19:28:26 -07:00
18563f2072 [Misc] Reduce attention logging levels (#40086) Chauncey 2026-04-21 10:09:25 +08:00
0e884fe638 [Bugfix] Fix _CONFIG_REGISTRY types getting wrong config class when on-disk model_type differs (#39554) MisaandMisael Casarez 2026-04-20 19:04:48 -07:00
fe5c115ee4 [vLLM IR] Add IR op testing and benchmarking infrastructure (#40167)
Yanan Cao Theresa Shan Claude Opus 4.6
2026-04-20 17:23:03 -07:00
6867bcd076 [Bugfix] Replace code that disabled shared expert overlap (#39222) bnellnmandRobert Shaw 2026-04-20 19:36:16 -04:00
c075702eae [Misc][UX] Suppress confusing num_gpu_blocks log lines (#40402) Matthew Bonanniandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-20 18:32:46 -04:00
21b086d0aa [ROCm] Hotfix: guard MLA dual RMS norm fusion against older AITer versions (#40386) Rita Brugarolas 2026-04-20 14:20:05 -07:00
3173441b0f [EPLB] Consolidate is_unchanged/is_received_locally into TransferMetadata (#37341) Sage Moore 2026-04-20 14:12:42 -07:00
8b1f3bebca [LMCache MP Connector] Add num_lmcache_extra_cached_token in KVTransferParams (#39843) Cao Qian 2026-04-20 13:42:49 -07:00
2390caf157 Enable building MoRI with AMD AINIC stack (#38371)
Theresa Shan Theresa Shan TJian
2026-04-21 02:17:59 +08:00
87805fa11e [Core] Cache InductorPass.hash_source with functools.cache (#39328) Frederik Gossen 2026-04-20 14:06:15 -04:00
304d5ba1a0 [Bugfix][CI] Fix tests/distributed/test_torchrun_example_moe.py (#40349) Nicolò Lucchesi 2026-04-20 20:05:44 +02:00
81d954f454 [WideEP] Remove naive all2all. Use allgather_reducescatter instead (#33728) Tyler Michael Smith 2026-04-20 13:53:55 -04:00
47fcb8ca68 [Core] Pass donate_graph_module=True to standalone_compile (#39733) Frederik Gossen 2026-04-20 13:40:52 -04:00
191e3fdaa1 Update flashinfer to 0.6.8 (#39959) bai 2026-04-20 20:37:23 +03:00
b9cf629bd0 [Core] Label torch trace logging overhead with dynamo_timed (#39329) Frederik Gossen 2026-04-20 13:31:03 -04:00
3461c8b027 [EPLB] Refactor Async EPLB synchronization logic (#37601) Sage MooreandTyler Michael Smith 2026-04-20 10:05:41 -07:00
726efe177b [MoE Refactor] Move the shared/fused expert output sum into MoERunnerBase (#35949) bnellnmandRobert Shaw 2026-04-20 12:28:46 -04:00
595562651a [XPU] fix MoE triton backend in online fp8 quantization (#40109) Yan Ma 2026-04-20 23:31:39 +08:00
3a30eaa1d7 Properly enable wvSplitK fp8 path for RDNA (#37712) Hashem Hashemi 2026-04-20 08:09:24 -07:00
fb5635d3f9 [ROCm] Add MLA dual RMS norm fusion (Q, KV) pass for DeepSeek/Kimi-K2 (#39242) Rita Brugarolas 2026-04-20 07:56:27 -07:00
b42e878ec0 [Bug] Fix dcp error message (#40053) Wentao Ye 2026-04-20 10:52:32 -04:00
7243e02aa1 [ROCm][Feature] Enable AITER MLA attention backend to work with Eagle3 speculative decoding on ROCm (#39616) larryli2-amdandTJian 2026-04-20 22:44:43 +08:00
def8f52200 [CI][EPLB] Add Async EPLB end-to-end integration test to CI (#40168) Sage Moore 2026-04-20 07:22:54 -07:00
38fa87caca mxfp8 online quant move to new frontend (#40152) Vasiliy Kuznetsov 2026-04-20 09:26:12 -04:00
a023edfa5b [bugfix] Use only onlines CPUs in lscpu (#40161) Galigatorandkse 2026-04-20 15:19:57 +02:00
b82fc1364d [Anthropic][Frontend] Added chat_template_kwargs to /v1/messages (#40125) aleksandaryanakievandAleksandar Yanakiev 2026-04-20 16:10:45 +03:00
e06de7f005 [XPU] enable triton attention test on XPU by removing cuda device binding (#39627) Yan Ma 2026-04-20 20:57:11 +08:00
cc3993b05d nixl refactor [2/N]: unify TpKVTopology + HeteroTPTransferConfig into TransferTopology (#39529) zhanqiuhu 2026-04-20 06:39:08 -04:00
50dd4cb427 [EPLB] Add nixl-based eplb communicator (#36276) Ilya Markov 2026-04-20 12:24:23 +02:00
f774ba028a [kv_offload+HMA][4/N]: Support sliding window lookup (#36645) Or OzeriandNicolò Lucchesi 2026-04-20 12:53:51 +03:00
2aab9acf48 [CPU][BugFix] Fix inter-node pipeline parallel (#40150) Fadi Arafeh 2026-04-20 10:21:12 +01:00
58631d7c3f [Bugfix] Fix scaled_mm output narrowing for 3D input tensors (#38093) nemanjaudovic 2026-04-20 10:58:39 +02:00
a943839e9a [ROCm][CI] Introducing new MI300 nodes (#39531) Andreas Karatzas 2026-04-20 03:09:58 -05:00
6d8b80802b [Docs] Fix thinking_token_budget docs (#40316) milesial 2026-04-20 01:09:44 -07:00
77fd2c8631 [Bugfix] Forward mm_processor_kwargs in offline generate APIs (#40251) wuyingjun 2026-04-20 15:56:56 +08:00
e729cc823d [Fix] Add Spacing when Requesting Output Token > max_model_len (#40324) San-Nguyen 2026-04-20 02:25:06 -05:00
ec7aafc02a [CPU][RISC-V] Support multiple RVV VLEN targets via compile-time dispatch (#39478) velonica0 2026-04-20 14:36:59 +08:00
6097afb9bd [BUGFIX] Fix Pixtral consolidated format vision weight loading (#39916) Julien Denize 2026-04-20 07:25:03 +02:00
4f4713f96e [XPU] [torch.compile] Skipping CUDA graph memory estimation to avoid startup errors. (#39977) Chaojun ZhangandKunshang Ji 2026-04-20 13:04:39 +08:00
8936118134 [Qwen][Bugfix] Fixes sigmoid activation in torch impl of RMSNormGated. (#40245) Tao He 2026-04-20 12:28:19 +08:00
67ed01c353 fix: Do not make function calls when request has no tools for /v1/responses (#40314) Yuan Tang 2026-04-20 00:17:30 -04:00
6e10cb54f6 [Bugfix][Responses API] Fix streaming tool calls on /v1/responses (#39892) Hoang NguyenandClaude 2026-04-20 10:24:52 +07:00
fcb31c1ac3 [Bugfix] Properly initialize PerTensorScaleParameter for fused-on-disk checkpoints (#39765) Shinichi Hemmiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-04-20 11:53:04 +09:00
d886c26d4d [Doc] Fix typos in token_embed pooling documentation (#40266) Lxx 2026-04-19 19:27:32 -07:00
898beca5a8 [BugFix][XPU] fix lora ops bgmv_expand size not match (#39989) Liangliang MaandKunshang Ji 2026-04-20 08:24:50 +08:00
629d45eacb [ci] Make ecr authenticate non blocking (#40305) Kevin H. Luu 2026-04-19 15:37:53 -07:00
f150107efd [ROCm] Fix cu_seqlens_q off-by-one in AITER FA speculative decode path (#39120) Andrew Barnes 2026-04-19 14:34:33 -04:00
d1135a5087 Fix MoE backend selection for LoRA (unquantized MoE) (#40273) danisereb 2026-04-19 20:18:40 +03:00
982beae809 Optimize nemotron VL image/video preprocessing (#40283) Netanel Haberandmilesial 2026-04-19 18:06:20 +03:00
45232a454e [FEAT] [Perf] [Gemma4] Fused Gemma4 Routing Function Triton (#39083) TJian 2026-04-19 17:57:39 +08:00
03ce1c6ed9 [Bugfix] Kimi-K2 tool parser streaming - fix token leakage, argument truncation, and content dropping (#38579) Flora Feng 2026-04-19 04:30:27 -04:00
4353c9cb4a [KV Offload] Pass request context (#39185) omerpaz95 2026-04-19 08:54:59 +03:00
4b7f5ea1a0 [KV Connector] Allow metrics of multiple connectors of same types in multi connector. (#40010) omerpaz95andOr Ozeri 2026-04-19 07:49:10 +03:00
38907e4391 [Frontend] Preserve structured output special tokens in offline LLM.chat (#39352)
Luciano Martins Luciano Martins Flora Feng
2026-04-18 20:46:07 -03:00
d0359f3e04 [Bugfix] Guard mxfp4_experts_quant bindings on ENABLE_NVFP4_SM100 (#40191)
mysterious hhhh mgoin Claude Opus 4.7
2026-04-19 04:58:46 +08:00
ed0622e3a8 [Attention] TurboQuant: remove redundant random signs, add prior art attribution (#40194) Dan Alistarh 2026-04-18 20:31:59 +02:00
b5f6c5f834 Added general ND x ND matmul and unit test for it (#39909) Yusuf Mohammad 2026-04-18 15:05:21 +01:00
bfde49e287 [DOC] Add fuse_minimax_qk_norm (#39782) Jee Jee Li 2026-04-18 15:41:37 +08:00
153ba7f0f3 [Refactor] Drop direct dependency on librosa (#39079) Nick CaoandClaude 2026-04-18 02:55:38 -04:00
87518c3027 [ZenCPU] AMD Zen CPU Backend with supported dtypes via zentorch weekly (#39967) Chinmay-Kulkarni-AMD 2026-04-18 11:52:37 +05:30
aeee7ef939 [Bugfix] Fix k_proj's bias for GLM-ASR (#40160) Rishapveer Singh 2026-04-18 07:34:33 +02:00
cda19ecf4d [Doc] Fix outdated source reference comment in anthropic/serving.py (#40189) z1ying 2026-04-17 22:31:13 -07:00
80b18230e0 [Frontend] Add multimodal support to /inference/v1/generate endpoint (#38405) Nithin ChalapathiandCyrus Leung 2026-04-17 20:31:56 -07:00
d0697cc7b6 [Doc] Add Realtime Transcription section to supported_models.md (#39845) z1ying 2026-04-17 20:26:14 -07:00
b0755523dc [Core] Reduce mm scheduler, get_num_embed overhead (#40143) milesialandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-04-17 20:25:49 -07:00
993859ceb0 [XPU] fix all_reduce all-zero accuracy issue under torch.compile (#39844) Chaojun Zhangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-18 10:33:07 +08:00
48a65ccb02 [CI] Speed up test_fused_marlin_moe (#40178) Michael Goin 2026-04-17 22:26:21 -04:00
55842a8d69 [XPU]fake impl for xpu fp8_gemm (#39984) Xinyu ChenandKunshang Ji 2026-04-18 08:53:56 +08:00
1f45e83756 Remove outdated tests test_mixtral_moe and test_duplicated_ignored_sequence_group (#40175) Michael Goin 2026-04-17 19:49:43 -04:00
a8bffaa133 [Kernel] Add MXFP4 W4A4 CUTLASS MoE kernel for SM100 (#37463) Michael Goin 2026-04-17 19:42:32 -04:00
5cdddddd4a [Kernel] [Helion] Force disable HOP path due to performance regression (#40171) Yanan CaoandClaude Sonnet 4 2026-04-17 14:36:49 -07:00
6ef1efd51f [ROCm] Fix TurboQuant on ROCm: backend routing, flash-attn compat, int64 overflow (#39953) aditi-amd 2026-04-17 13:08:37 -07:00
58da4ee047 [AMD][CI] Update DeepEP branch (#38396) Ryan Rock 2026-04-17 14:30:20 -05:00
1ae11e2bfc [ROCm][CI] Build fastsafetensors from source so it links against libamdhip64 (#39978) Andreas Karatzas 2026-04-17 14:30:08 -05:00
251c18d1f8 skip fp8e4b15 on xpu (#39957) Xinyu ChenandKunshang Ji 2026-04-18 00:55:08 +08:00
512765d52d [Misc][UX] Map mimo reasoning and tooling parsers (#40089) Roger WangandChauncey 2026-04-17 09:49:21 -07:00
640cc9dd7d feat: Add LoRA support for Gemma4ForConditionalGeneration (#39291) allgatherandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-04-17 09:39:19 -07:00
ceade1952c [BugFix] Support custom tool parsers when tool_choice is required and named function (#39870) Jared Wenandsfeng33 2026-04-18 00:38:10 +08:00
747256bb5d [Bugfix][Core] Fix stuck chunked pipeline parallelism with async scheduling (#38726) Jing WangandCopilot 2026-04-18 00:02:50 +08:00
1174723eba Fix TURBOQUANT backend selection in cuda.py (#40060) Michael Goin 2026-04-17 10:31:41 -04:00
6b2b7bd0eb Add nvfp4 support to reshape_and_cache_flash (#37332) sychen52 2026-04-17 07:28:00 -07:00
70770268c3 Add @bbrowning to CODEOWNERS (#40141) Ben Browning 2026-04-17 09:51:48 -04:00
7a51b3e415 [Bugfix] Fix empty delta detection in Qwen3XMLToolParser streaming (#40090) Chauncey 2026-04-17 21:34:55 +08:00
d02421a7db [CPU] Refactor CPU affinity and memory management (#39781) Li, Jiang 2026-04-17 21:01:08 +08:00
b1dc87a098 [Models][Gemma4] Prevent GPU/CPU sync in embed_input_ids (#39234) Lukas Geiger 2026-04-17 13:37:21 +01:00
79a5b63253 [kv_offload]: Fix num CPU blocks for UniformTypeKVCacheSpecs (#39617) Or Ozeri 2026-04-17 15:13:55 +03:00
c0c98b8b9a [Bugfix] Add Marlin kernel in block scaled mm kernel selection. (#40105) Maral 2026-04-17 18:20:32 +08:00
8d2cff8140 [Examples] Resettle Observability examples. (#40123) wang.yuqi 2026-04-17 18:13:31 +08:00
4f436782af [Misc] Improve new PR bot trigger condition (#40114) Cyrus Leung 2026-04-17 16:56:22 +08:00
bf45e6d0a5 [Doc] Add Gemma 4 to supported models list (#39607) z1ying 2026-04-16 22:42:52 -07:00
978a4462bb [CI Failure] Fix Plugin Tests (2 GPUs) Failure (#40083) wang.yuqiandMichele Gazzetti 2026-04-17 12:17:39 +08:00
1948d0c467 [UX] Defer some imports on CLI paths to save ~2s (#40056) Michael Goin 2026-04-16 22:48:37 -04:00

... 73 74 75 76 77 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86