Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

b4092176b9 [Bugfix] Complete one-shot fused all-reduce PDL at end to avoid NaN (#45448) Ace Eldeib 2026-06-17 20:54:39 -04:00
ebbb2d55ac [CI/Build][Bugfix] Fix SD LoRA (#45941) Jee Jee Li 2026-06-18 08:34:15 +08:00
2959a9273a [XPU][CI] add model runner v2 into CI (#44650) liuzhenwei 2026-06-18 08:28:34 +08:00
1797576237 Revert "[DSV4 Perf] Optimize dsv4 cudagraph by reducing eager_break_during_capture" (#45309) (#45972) Woosuk KwonandClaude Opus 4.8 2026-06-17 17:20:59 -07:00
0d339cf135 [Bugfix] Fix NixlConnector handshake block_len validation for GQA-replicated KV heads (#45879) Bryan Shanandwaynehacking8 2026-06-17 15:11:29 -07:00
5fd21eb0b2 [BUG] fix hidden states nan for hybrid attention models (#45849) shanjiazandshanjiaz 2026-06-17 18:02:24 -04:00
9d4b87f4f0 [Bugfix][Model] Validate DefaultModelLoader / LoadConfig and fail with clear errors (#45196) Ting SUN 2026-06-18 05:46:33 +08:00
58b2e89642 [Bugfix][Gemma4] Render reasoning on assistant turns without tool_calls (#45867) Luciano MartinsandLuciano Martins 2026-06-17 17:44:15 -03:00
2659f60a1a [Refactor] Remove dead quantization code and tests (#45454) Wentao Ye 2026-06-17 16:12:01 -04:00
091386a99b [Bugfix] MiniMax-M3 (AMD): add packed_modules_mapping and pass swiglu… (#45794)
wangjiaxin99 TJian Douglas Lehr
2026-06-18 03:15:46 +08:00
d112eb1ac7 [feature] MiniMax-M3-MXFP4 support added (#45896) qli88 2026-06-17 13:50:48 -05:00
2a47a9ff0f [DSV4 Perf] Optimize dsv4 cudagraph by reducing eager_break_during_capture, 26.8% ~ 27.9% E2E TTFT improvement (#45309) Wentao Ye 2026-06-17 12:34:53 -04:00
9c7c74bf10 [Log] Update deepgemm log (#45857) Wentao Ye 2026-06-17 11:34:22 -04:00
5e27b2baf4 [Bugfix] Pass TP group to FlashInfer all-reduce fusion (#45917) danisereb 2026-06-17 18:24:26 +03:00
eb0fdeb1e8 [Bugfix][PD] Fix DSV4 disaggregated serving (#45831) zhanqiuhu 2026-06-17 11:17:14 -04:00
46f74e144b [Kernel][Helion][1/N] Add Helion kernel for rms_norm_dynamic_per_token_quant (#34432) Xiaohong (Sean) ChenandYanan Cao 2026-06-17 11:03:54 -04:00
0a7bacdcac [DSv4 Perf] DSv4 flashinfer sparse index cache for metadata, 2%~4% TTFT improvement (#45863) Wentao Ye 2026-06-17 10:55:48 -04:00
8b2b566ea7 Feature: Enable Flashinfer non-gated MoE bf16 (#43853) amirkl94 2026-06-17 17:32:49 +03:00
0b131b16c9 [ROCm][AITER][Quark] Tag per-channel FP8 weights as PER_CHANNEL so AITER pre-shuffled GEMM is selected (#44626) xaguilar-amd 2026-06-17 16:05:34 +02:00
bcb518ad7a [quant][autoround]Refactor INC quantization into package with INCScheme orchestrator (#40601)
Yi Liu n1ck-guo Zhenzhong1
2026-06-17 21:51:32 +08:00
06e1e0885c [XPU] Fix test_logprobs_e2e import error: pin lm-eval[api]>=0.4.12 (#44469) Chaojun Zhang 2026-06-17 20:26:47 +08:00
1a59078c87 [CI/Build] Avoid duplicate ViT CG test introduced by accident (#45654) Isotr0py 2026-06-17 20:23:44 +08:00
fa85ead2f3 [MM][Perf][CG] Support ViT full CUDA graph for Kimi-VL (#41992) Oğuzhan KIR 2026-06-17 15:14:01 +03:00
e28e8c8782 [ROCm][Quant] Minimax-M3: Enable fp8_per_channel for bf16 weights on mi300x (#45854) Hongxia Yangandtjtanaa 2026-06-17 08:02:40 -04:00
ee0fd6984a docs, kv_offloading: add docs for selective offload (#45279) Angelo Ruocco 2026-06-17 13:58:00 +02:00
d537122398 [ROCm][Bugfix]: Fallback GFX942 sparse MLA ops to Triton (#45782) vllmellm 2026-06-17 19:41:29 +08:00
3d20275bb4 fix(security): enforce audio decode duration limit in chat completions path (#45908) Juan Pérez de Algaba 2026-06-17 13:07:13 +02:00
f694d43b33 [Bugfix][test] Use Salesforce/wikitext for ppl tests (#45913) wentian-byteandwentian-byte 2026-06-17 18:37:19 +08:00
3c6084bb0d [Bugfix][Gemma4] Pre-initialise streaming reasoning state when prompt ends inside an open <|channel> (fixes #45834) (#45852) Nikhilesh Chhetri 2026-06-17 05:16:02 -05:00
68ff30d40e [Bugfix] Fixes MiniCPM-O resampler device placement to avoid tensor device mismatch (#42332) Joel Smith 2026-06-17 09:35:27 +01:00
6d8fff5698 [KV Connector][Offloading] Avoid blocking the engine to flush offloads on idle (#45595)
Itay Etelis Itay Etelis Or Ozeri Itay Etelis
2026-06-17 11:35:07 +03:00
e2c58570ea [Rust Frontend] Support hybrid/external DP LB in Python supervised bootstrap (#45805) Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-17 15:32:40 +08:00
43fa24e832 [Misc] Validate Cohere Embed Mixed Content Payloads (#45873) Taneem Ibrahim 2026-06-17 01:57:12 -05:00
93bbe94d3a [Kernel] Add weightless RMSNorm CUDA kernels for has_weight=False (#41430) (#44109) arghyadeep sarkar 2026-06-17 12:15:55 +05:30
17bc144556 [Rust Frontend] Add serde defaults for omit_defaults fields in EngineCoreSamplingParams (#45848) Will Eaton 2026-06-17 02:40:49 -04:00
295232a26a [Rust Frontend] Add /abort_requests endpoint (#44382) Sahil Singh 2026-06-17 12:10:47 +05:30
56e4345226 [Rust Frontend] Support prompt-only completions (#44938) Reid 2026-06-17 14:38:06 +08:00
e9993a52aa [BugFix][CI] Fix scheduler plugin test (#45897) Nick Hill 2026-06-16 23:30:49 -07:00
a46abb7ae6 [Bugfix][Quantization] Reject unsupported compressed tensors KV cache schemes (#45312) Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-17 13:08:41 +08:00
4c62663315 [M3] Enable FP8 sparse GQA (#45744) Thien TranandYongye Zhu 2026-06-17 12:38:03 +08:00
d78650cf97 [CI][NIXL] Pin NIXL to 1.2.0 (#45843) Itay Alroyandovidiusm 2026-06-17 07:29:34 +03:00
5bdc01bcc3 [M3] Tune Triton indexer score decode for spec-decode (#45743) Thien Tranandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-17 12:07:34 +08:00
20a5f8b43b [FlexAttention] make custom mask mods fully cudagraphable (#45232) liangel-02 2026-06-16 23:53:12 -04:00
7b5d60cc37 [Bugfix][V1] Clean up compiled-model bytecode hooks on VllmRunner exit (#45195) Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-17 11:31:17 +08:00
14b438a98b [ModelRunnerV2] Various model/config compatibility fixes (#45868) Nick Hill 2026-06-16 20:23:01 -07:00
2785a5e0e6 [Bugfix][ROCm] Fix FP8 per-tensor scale rank mismatch causing Inductor assertion failure (#44912)
nehmathe2 TJianmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Divakar Verma Andreas Karatzas
2026-06-16 20:17:42 -07:00
efd15e192a [Bugfix][ROCm] Fix MiniMax-M3 FP8 KV cache dtype (#45720)
Cameron Quilici Hongxia Yang TJian
2026-06-16 22:14:45 -05:00
556b063e45 [XPU] Fix test_spec_decode_logprobs: use FLASH_ATTN for XPU in GPU_DETERMINISM_KWARGS (#44468) Chaojun ZhangandKunshang Ji 2026-06-17 11:07:04 +08:00
aa0ac8a661 [CI] Run pre-commit on self-hosted vllm-runners (#45865) Kevin H. LuuandClaude Opus 4.8 2026-06-16 19:49:22 -07:00
b831374cf1 [Bugfix][Gemma4] Fix parsing when thinking is disabled (#45832) Federico 2026-06-17 04:41:36 +02:00
71bc19dbdd [Bugfix] Fix MoE model load OOM in FlashInfer_TRTLLM backend with sleep mode (#45589) Dakai Anandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-17 10:36:51 +08:00
ef2c40dc00 [XPU][CI] fix server test file path (#45870) Kunshang Ji 2026-06-17 09:06:25 +08:00
4bf699d310 [Kernel] Support DS Mamba tail copy for MTP align mode (#45473) sungsoo haandThomas Parnell 2026-06-16 15:50:30 -07:00
520828789c Apply LRU policy only to proper cache entries (#42656) Stan Wozniak 2026-06-16 23:49:15 +02:00
9d4dc4ca2f [Kernel] Support GLM-5 dimensions for TRT-LLM ragged MLA prefill (#43525) Mohammad Miadh AngkadandMatthew Bonanni 2026-06-17 04:49:47 +08:00
b9684d99e9 [Bugfix] Gemma4: skip forced JSON for required/named tool choice (#45795) Federico 2026-06-16 22:38:29 +02:00
4fadf9c92c [ROCm][CI] fix multimodel run cmds (#45858) Divakar Verma 2026-06-16 16:31:52 -04:00
d8d95998dc [Core] Add prefill step cadence for better non-PD DP balancing (#44558) Nick Hill 2026-06-16 13:17:18 -07:00
475a6ad18a [Misc] Update Mergify tool-calling label (#45853) Flora Feng 2026-06-16 15:08:00 -04:00
f2beaa80c8 [ROCm][Quant] mxfp8 moe/linear gfx950 tuning for MiniMax-M3 (#45725) Hongxia Yang 2026-06-16 14:50:40 -04:00
8e27a9c215 [PERF] Fuse multi-group block table staged writes (#44944) Song ZhixinandNick Hill 2026-06-17 01:53:27 +08:00
7d567172fc [Bugfix] Fix Qwen3 prompt tool-call reasoning false positive (#45763) alexbi29andAlex Bilichenko 2026-06-16 10:48:01 -07:00
f00e163f35 [Frontend] Add Streaming Parser Engine and new MinimaxM2 Parser (#45701) Chauncey 2026-06-17 01:38:17 +08:00
44b2512767 [KV Connector][Mooncake] Add cache_prefix to namespace store keys (#45767) Dao007foreverandClaude Opus 4.8 2026-06-16 10:24:20 -07:00
188c68798e [KVConnector][MoRIIO] Allow overriding the advertised host IP (#45488) kourosh hakhamaneshiandClaude Opus 4.8 2026-06-16 10:18:37 -07:00
c45f681932 [Bugfix][Core] Fall back when numactl --membind is blocked in constrained containers (#45438) Ting SUNandShengqi Chen 2026-06-17 00:49:56 +08:00
89e8645a9e [Model] Remove Dots1ForCausalLM (#45637) Tiezhen WANGandClaude 2026-06-17 00:32:18 +08:00
88a9cdd439 [Model Runner V2] Enable GraniteMOE for MRv2 by default (#45461) Wentao Ye 2026-06-16 12:31:32 -04:00
6f612fbedf [ROCm][CI] Patch conftest to resolve occasional OOMs (#45722) Micah Williamson 2026-06-16 10:00:15 -05:00
506ec6d656 Upgrade tpu-inference to v0.22.1 (#45793) Sting Lin 2026-06-16 22:54:57 +08:00
a52205bccf [Model] Add HrmTextForCausalLM (Hierarchical Reasoning Model — Text) (#43098) yifei wuandHarry Mellor 2026-06-16 22:41:41 +08:00
3d34f8cbdc [ROCm][Cleanup] Remove stale AITER FA hybrid KV-cache TODO (#44178) Tuukka SarviandCyrus Leung 2026-06-16 17:28:06 +03:00
eb04c769d3 feat: MLA prefill enable FA4 fp8 output (#43050) Carl Y 2026-06-16 07:10:59 -07:00
ce3ef17bec [Kernel][Helion][1/N] Add Helion kernel for rms_norm_per_block_quant (#36895) Xiaohong (Sean) ChenandYanan Cao 2026-06-16 10:09:52 -04:00
bf5149b516 [Bugfix] Fix FlashMLA sparse accuracy with topk_length and zero-init padding (#36616) Ajay AnuboluandMatthew Bonanni 2026-06-16 07:09:00 -07:00
cca3365b73 [Rust Frontend] Add CORS support (#45753) Tahsin Tunan 2026-06-16 19:47:11 +06:00
040df8f2ea [CI] Fix attention benchmark smoke test (#45728) Matthew BonanniandClaude 2026-06-16 09:43:37 -04:00
ced32bb474 [Perf] Add VLLM_TRITON_FORCE_FIRST_CONFIG to skip Triton autotuning (#42425) Francesco FuscoandClaude 2026-06-16 15:16:45 +02:00
c5e5c33fcd [Bugfix][MoE] Restore routed output unpadding before shared expert add (#45707) Netanel Haberandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-16 16:06:28 +03:00
a8c86eeb16 [Quant] Support modelopt_mixed on Ampere (SM80/SM86) (#45306) Mike G 2026-06-16 05:43:44 -07:00
7e179e4bc0 [ROCm][CI] Gate incompatible HF references on Transformers v5 (#41532) Andreas Karatzas 2026-06-16 07:34:11 -05:00
405c7cf283 [ZenCPU] Add zencpu Platform Runtime Logging and Docs (#42726)
Lalithnarayan C Tyler Michael Smith Claude Tyler Michael Smith
2026-06-16 17:53:12 +05:30
3f53e2138f [Refactor] Remove Fp8OnlineLinearMethod as scheduled (#45463) Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-06-16 07:35:58 -04:00
d53f4593ce [KV Connector][Mooncake] Pipeline-parallel support for PD-disaggregated serving with Mooncake connector (#44528) Hank Han 2026-06-16 19:35:38 +08:00
ad32608e24 [MM][Perf][CG] Support dual-path ViT full CUDA graph for DeepSeek-OCR (#43586)
Shanshan Shen Roger Wang Isotr0py
2026-06-16 19:35:20 +08:00
b2cfae777d Add Triton recompile detection (#45631) Thien Tran 2026-06-16 18:25:28 +08:00
3f1ff1ff14 [Misc]Clean up useless test (#45792) wangxiyuan 2026-06-16 17:53:08 +08:00
c69c73418a [XPU][CI] add intel xpu cases for nightly CI (#44372)
wenjun liu zengxian Kunshang Ji
2026-06-16 16:35:08 +08:00
ebf3a6d705 [Bugfix] Fix trtllm fused allreduce+rms_norm for transformers backend (#45307) Thomas Parnell 2026-06-16 10:34:27 +02:00
c4fd9794e9 [Frontend] Remove AsyncMicrobatchTokenizer. (#45759) wang.yuqi 2026-06-16 16:02:11 +08:00
7ad894c86a [Bugfix] Prevent cuMemcpyBatchAsync segfault with MTP and KV offloading (#44784)
joshua abraham joshua Or Ozeri
2026-06-16 13:28:39 +05:30
a7fdfeef72 [CPU] Support Gemma Diffusion (#45690) Li, Jiang 2026-06-16 14:39:56 +08:00
8bf374955f [Bug Fix] Allow pinned memory for WSL2 (#41496) Jimmy Lee 2026-06-15 22:56:26 -07:00
9096659edb [Cleanup] Remove dead env (#45777) Cyrus Leung 2026-06-16 13:56:23 +08:00
81d8f4ebac [Misc] Added validation for Cohere /v2/embed input field exclusivity (#45640) Taneem Ibrahim 2026-06-16 00:42:43 -05:00
a9a8a32dcd Register parsed config classes before tokenizer init (#40299) Andrew BarnesandOpenAI Codex 2026-06-16 01:33:08 -04:00
9d808e2309 [Core] Use fastsafetensors ParallelLoader for weight loading (#40183)
gitbisector Claude Cyrus Leung
2026-06-15 22:32:05 -07:00
f3858d5422 [Frontend] [Parser] Migrate Nemotron V3 to streaming parser engine (#45755) Ben Browning 2026-06-16 01:31:21 -04:00
259ff891be [Rust Frontend] Require ModelConfig.vocab_size to be present (#45696) Bugen Zhao 2026-06-16 13:30:25 +08:00
6607a80dab [Bugfix][Gemma4] Fix offline parser truncation, adjust_request token leak, and chat template sync (#45553) Luciano MartinsandLuciano Martins 2026-06-16 01:31:53 -03:00

... 55 56 57 58 59 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86