Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

fe5b4e0fe7 [Model Runner V2] Apply synthetic mode to probabilistic rejection sampler (#41035) Giancarlo Delfin 2026-05-12 15:37:03 -05:00
0ce6613b9c platforms: add uses_cpu_device() hook to Platform for DeviceConfig (#42313)
Viktor Pus Cursor Wentao Ye
2026-05-12 21:39:17 +02:00
379f0ec369 [CI] Migrate 6 verified jobs from gpu_1_queue to h200_18gb MIG (#42446) Kevin H. LuuandClaude Opus 4.6 2026-05-12 11:52:01 -07:00
67c89fe40a [Model][Bugfix] Fix Step3-VL image_embeds input path (#42333) KaivalyaMDabhadkar 2026-05-13 00:17:55 +05:30
d9b4990783 [MoE Refactor] EPLB refactoring for FusedMoE (#41055) bnellnmandRobert Shaw 2026-05-12 14:16:31 -04:00
4d591db470 [MoE Refactor] Introduce RoutedExperts alias for FusedMoE and don't store SharedExperts in MK (#40735)
bnellnm Robert Shaw Robert Shaw
2026-05-12 13:37:44 -04:00
6ff7405b81 [Bugfix] [Frontend] Responses API, fix merging of messages (#42189) yzong-rh 2026-05-12 12:09:59 -04:00
bcb9c133ba feat(kv-events): emit KV cache metadata (#40984) Yan Ru Pei 2026-05-12 08:58:48 -07:00
c8a6e272e0 [CPU] Fix rotary embedding for CPU without flash-attn ops (#42225) Jonathan MamouandWentao Ye 2026-05-12 18:05:35 +03:00
a1b2d87498 [Refactor] Clean up pooling models build_tok_params logic (#42341) Wentao Ye 2026-05-12 11:05:05 -04:00
418ba8ef14 [kv_offload][BugFix] Fix store deferral (#41945) Martin Hickey 2026-05-12 16:04:44 +01:00
5a6a9fc6f6 [docs] Added one new contact to the Vulnerability Management team (#42145) Juan Pérez de Algabaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-12 16:59:59 +02:00
289cee0473 [vLLM IR] Minor improvements (#39362) (#39558) Avishek GoswamiandAvishek Goswami 2026-05-12 20:28:36 +05:30
6ccb10d794 Added peagle speculators support (#41826) shanjiaz 2026-05-12 10:55:57 -04:00
7a9cc5e7f0 [Model] Support MiniCPM-V 4.6 (#41254) tc-mbandcaitianchi 2026-05-12 22:28:10 +08:00
d077622d60 [Build] Build bundled DeepGEMM _C per-Python so the wheel imports on every CPython (#41516) Michael GoinandClaude Opus 4.7 2026-05-12 10:27:29 -04:00
dd6b3a5ef5 [Perf] Use 2D-grid to eliminate divmod in W8W8 group quant (#42153) Jiahan Chang (Cyrus)andYongye Zhu 2026-05-12 22:01:30 +08:00
593d5a4033 [Bugfix] Fix mismatched kernel-per-logical blocks in NIXL HMA transfer (#42097) zhanqiuhuandNickLucche 2026-05-12 09:53:30 -04:00
6427603ae8 [MoE Refactor] Move remaining experts classes to experts directory (#42334) bnellnm 2026-05-12 09:19:46 -04:00
206eaed08d [MoE Refactor] Move expert map related code into ExpertMapManager class (#41046)
bnellnm Robert Shaw Robert Shaw
2026-05-12 09:18:27 -04:00
8f89381fc6 [Hybrid] Warmup Mamba2 SSD kernel (#39822) Thomas ParnellandClaude Opus 4.6 2026-05-12 14:46:22 +02:00
a7b801e26d [MXFP4] Support for linear layers + compressed-tensors integration (#41664) Dipika Sikka 2026-05-12 07:49:33 -04:00
4df1be9547 [XPU] bump up vllm-xpu-kernels to v0.1.8 (#42410) Kunshang Ji 2026-05-12 19:47:37 +08:00
bc03f280c8 [XPU] keep generator state of sycl kernel align with pytorch (#41771)
Yan Ma Qiming Zhang Kunshang Ji
2026-05-12 19:44:47 +08:00
997132911e [Doc] Fix typo in llm-d documentation link (#42397) Florian WoernerandCyrus Leung 2026-05-12 13:26:46 +02:00
fc8bf6eedb [CI] De-flake Language Models Test (Extended Generation) test_models(False-False-5-32-bigcode/starcoder2-3b) (#42392) haosdent 2026-05-12 18:46:48 +08:00
07a40ede19 [UT][XPU] fix test_parallel_sampling due to global random state (#42388) liuzhenwei 2026-05-12 18:03:23 +08:00
e1c8776e90 [CI] Move DockerHub and PyPI publish steps to end of release pipeline (#42355) Kevin H. Luu 2026-05-12 02:17:42 -07:00
1ff9d33535 [CI] Migrate remaining B200 jobs to b200-k8s with test fixes (#42387) Kevin H. Luu 2026-05-12 02:00:37 -07:00
7f65f84428 [Bugfix] Fix empty channel/recipient in harmony for /v1/responses (#35540) chazenandChauncey 2026-05-12 04:45:51 -04:00
ef34592a1a [Bugfix] Fix double reduce in flashinfer_nvlink_two_sided and flashinfer_nvlink_one_sided backends (#41382) amitz-nv 2026-05-12 10:47:47 +03:00
f69644caf8 [CI] Migrate more B200 jobs to b200-k8s queue (#42356) Kevin H. Luu 2026-05-12 00:38:31 -07:00
d37e25ffbe [Frontend] Consolidate Speech to Text entrypoints. (#42370) wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-12 15:06:57 +08:00
8517cdaf90 [XPU] update dp rank w/o env-var isolation (#39856) liuzhenweiandKunshang Ji 2026-05-12 14:49:54 +08:00
4e498b5e5c [Bugfix][Performance Improvement] Improve penalties triton kernel performance (#40657) Lucas Kabela 2026-05-11 22:47:20 -07:00
28ee78af54 Implement custom dataset class for ASR benchmarking (#41576) Yasmin Moslemandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-12 05:17:58 +01:00
630492da30 [Fix] Gemma4 Mixed-Resolution Image Co-Batching Crash (#42217) ZiTian Zhao 2026-05-12 11:13:03 +08:00
920bf3ec84 [Bugifx] [Qwen3CoderTool] Restore supports_required_and_named for required tool_choice (#42292) Chauncey 2026-05-12 10:09:56 +08:00
39dff5ff39 Add VLLM_USE_SPINLOOP_EXT to use more efficient busy polling (#36517) pschlan-amd 2026-05-12 01:11:49 +02:00
d7af6b34d8 [Model Runner V2] Bug fix: logprob dtype int64/int32 issue (#41761)
Wentao Ye Nick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-11 17:55:43 -04:00
bbee532988 [Perf][1/n] Eliminate various GPU<->CPU syncs (#41429) Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-11 13:36:03 -07:00
53181384e0 [Bugfix] Fix DSV4 swiglu_limit on marlin backend (#42287) Jee Jee LiandYongye Zhu 2026-05-12 04:03:56 +08:00
a0dc7a0f36 [CI] Consolidate Speech to Text tests (#42274) wang.yuqi 2026-05-12 03:50:17 +08:00
56e5810ff1 [BugFix] Prevent orphaned process on NCCL destroy (#39846) Jeffrey WangandTyler Michael Smith 2026-05-11 12:25:26 -07:00
639cbfd274 [CI] Add tests/parser to CI coverage (#41877) Flora Feng 2026-05-11 15:08:54 -04:00
a721315488 [ROCm][Perf] Fix RMSNorm+Quant fusion for gfx950 (non-fnuz) (#41825)
frida-andersson Markus Hartikainen Cursor Chuan Li Luka Govedič Frida Andersson TJian
2026-05-11 21:00:51 +02:00
6fdb49392e [Bugfix] Fix int32 overflow in DeepGEMM SiLU/mul FP8 Triton kernel (#42201) Vensenandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-12 01:52:31 +07:00
cf0d279142 [Docs] Add Apple Silicon documentation for vLLM-Metal GPU support (#41987) Alexa GriffithandClaude Sonnet 4.5 2026-05-11 14:34:25 -04:00
5497ffbf7c Add documentation about vLLM FIPS compliance (#42190) Vinay R DamodaranandRussell Bryant 2026-05-11 11:17:02 -07:00
9af6a5ed75 [Model Runner V2] Fix seq_lens_cpu_upper_bound (#42202) Nick Hill 2026-05-11 10:37:50 -07:00
7863fff6e5 [ROCm][DSv4] implement flash sparse mla with triton kernels (#41812) Hexiang Wang 2026-05-12 00:27:11 +08:00
0d453e2336 [Perf] Batch invariance with Cutlass fp8 support, 28.9% E2E latency improvement (#40408) Wentao Ye 2026-05-11 12:20:58 -04:00
3f9c0c25b3 [Bug] Fix kimi dtype issue with mm_projector_forward (#42081) Wentao Ye 2026-05-11 11:45:24 -04:00
a2e776d716 [Bugfix] Accept canonicalized modelopt_* quant_method in _extract_modelopt_quant_algo (#42181) Vadim Gimpelson 2026-05-11 19:10:57 +04:00
4955990f1b [kv_offload] Move FilterReusedOffloadingManager logic to CPUOffloadingManager (#41727) Martin Hickeyandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-11 16:09:29 +01:00
4b64fc2cbf [Refactor] Cleanup batch invariant dead code (#41993) Wentao Ye 2026-05-11 10:48:39 -04:00
5f1b313900 [ROCm] Clean up a bit the AITER FA backend (#41942) pschlan-amd 2026-05-11 16:45:18 +02:00
724ed2fc35 [DSv4] Improved dequant gather K cache kernel (#42236) Thien TranandYongye Zhu 2026-05-11 21:41:12 +07:00
a51376b3f0 [Performance][DSR1]: Fused RoPE+KVCache+q_concat for MLA (#40392) Rohan PotdarandElizaWszola 2026-05-11 07:10:50 -07:00
8415bf2cdb [kv_offload] Set offloading connector to prefer HND layout (#41928) Martin Hickey 2026-05-11 13:05:41 +01:00
ac062147fa Avoid silent weights corruption when loading Nemotron Nano VL with reusable-buffer loaders like runai distributed streaming (#42244) Noa Neria 2026-05-11 15:03:14 +03:00
617239b70c [Frontend]Responses API supports chat_template_kwargs (#42272) Chauncey 2026-05-11 19:59:39 +08:00
27ae676364 Fix EXAONE-4.5 to align with Transformers update (#42246) Kyungmin Lee 2026-05-11 19:25:31 +09:00
17ed5e61f5 [CI] Make Python-only Installation optional (#42293) haosdent 2026-05-11 17:47:16 +08:00
5672d100ed [KV Connector][NIXL][Bugfix] Fix NIXL handshake failures not honoring kv_load_failure_policy (#40364) Nicolò Lucchesi 2026-05-11 11:37:21 +02:00
770e9bd6b3 [Nixl][PD] Lease renewal TTL KV blocks on P (#41383) Nicolò Lucchesi 2026-05-11 11:27:30 +02:00
9efdddca28 [Model] Fix missing maybe_prefix (#42280) Cyrus Leung 2026-05-11 17:04:06 +08:00
b1b59720b2 bugfix(flashinfer,dcp): remove kv_cache_layout for BatchDCPPrefillWrapper._new_tokens. (#38895) Qiu 2026-05-11 16:11:49 +08:00
f9f770ca0b fix nixl side-channel host selection (#41806) shaharmor98andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-11 10:40:37 +03:00
5cba6839e6 Document MolmoWeb hf_overrides (#42163) Haoqing Wang 2026-05-11 14:58:22 +08:00
05d610e5cd [CI/Build] Reduce LoRA model tests. (#42266) Jee Jee Li 2026-05-11 14:49:08 +08:00
581b5e9afc [Frontend] Return rendered prompt text in chat completion response (#42052)
汪志鹏Wang, Zhipeng | RASIA <zhipeng.wang@rakuten.com>Cursor Chauncey
2026-05-11 13:53:39 +08:00
5536fc0c01 [Misc] Replace mamba_type string literals with MambaAttentionBackendEnum (#41188) wangxiyuan 2026-05-11 11:59:36 +08:00
7f95e66a11 [ROCm][Bugfix]: dynamically align BLOCK_DMODEL with Lv in MLA decode kernel (#41119) vllmellm 2026-05-11 11:14:19 +08:00
b1687527b8 [Bugfix] Gemma 4 chat template crash with missing tool name and tool id (#42188) yzong-rh 2026-05-10 23:07:45 -04:00
171019ab19 add fused mhc_post_pre kernel (#41536) gnovackandgeorge 2026-05-10 19:56:52 -07:00
879a8c3180 Fix Molmo2 image token metadata (#42162) Haoqing Wang 2026-05-11 09:19:21 +08:00
1b57eb41f2 [MoE] Move various experts classes to fused_moe/experts/ (#41979)
bnellnm Jackmin801 Claude Robert Shaw Robert Shaw Jackmin801
2026-05-10 19:54:33 -04:00
21943d4c25 [Performance] Make safetensors checkpoint prefetch settings configurable (#41499) Mohammad Miadh Angkad 2026-05-10 23:55:15 +08:00
f396bee56f [DSV4] Add PP support for deepseek-v4 (#41694) Isotr0pyandqizixi 2026-05-10 23:47:26 +08:00
215e2f7990 [Bugfix][Mamba] IMA in causal_conv1d kernel for long sequences (#41617) Vensenandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-10 19:38:28 +07:00
e175192d33 [KV Offload] Pass ReqContext to touch(), complete_load(), and complete_store() (#41366) Ronen Schaffer 2026-05-10 15:09:25 +03:00
a54f0d1049 [CPU] Fix spec decode kernel signatures for synthetic mode compatibility (#41932) Jonathan MamouandBenjamin Chislett 2026-05-10 15:07:15 +03:00
48698b1b9b [Bugfix] Fuse Qwen3.5 in_qkvz_proj forwarding with LoRA enabled (#37912) Isotr0py 2026-05-10 18:59:02 +08:00
0a309b5ee9 [ROCm] Cap Triton paged attention block size to fix ROCm shared memory OOM (#38502) Andreas Karatzas 2026-05-10 05:03:00 -05:00
84f7a55340 [CI] Trigger LoRA test when changing MoE code. (#42196) Jee Jee Li 2026-05-10 16:26:09 +08:00
a2c9d548d7 [Docs] Fix broken local links (#42160) Ethan Feng 2026-05-10 16:15:38 +08:00
301305c093 Add @zyongye to CODEOWNERS (#42200) Yongye Zhu 2026-05-10 04:07:32 -04:00
efd0e7789d Fix mypy failure on main (#42197) Mohammad Miadh Angkad 2026-05-10 15:55:57 +08:00
a5d0a5afba [Frontend][Bugfix] Abort ASR engine requests on cancellation (#41266)
abdulrahman-cohere Cursor Agent Nicolò Lucchesi
2026-05-10 02:51:11 -04:00
f2840120f6 [ROCm][CI] Fix NIXL spec-decode acceptance startup and diagnostics (#41313) Andreas Karatzas 2026-05-10 01:50:16 -05:00
3f5bd482f5 [Bugfix][KV Transfer][NIXL] Notify P node on pre-admission rejection to free stranded KV blocks (#41269) Dao007forever 2026-05-09 22:52:09 -07:00
fb1ac806c5 [ROCm][CI] Stabilize ROCm shutdown and distributed compile CI (#41573) Andreas Karatzas 2026-05-09 22:47:40 -05:00
986edc858a [Bugfix] Fix DeepSeek v4 topk numerical issue for unaligned max-model-len (#42169) Wei Zhao 2026-05-09 23:30:08 -04:00
27d3bac272 docs: clarify Gemma 4 assistant speculative decoding (#42180) Abhishek GuptaandAbhiOnGithub 2026-05-09 20:08:44 -07:00
00b0618a03 Use CU_MEMCPY_SRC_ACCESS_ORDER_ANY for batch KV cache swaps (#39306)
Itay Etelis Itay Etelis Or Ozeri Itay Etelismergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-10 05:57:09 +03:00
0d382ecde8 Handle optional bool-or-string CLI args in get_kwargs (#40951) Christian VanandChristian Van 2026-05-09 22:47:21 -04:00
1029e5ef28 [CI/Build] Use modelscope's international site for regression test (#42176) Isotr0py 2026-05-10 10:47:09 +08:00
0b272a6e01 [Bugfix] Fix SP pass for multimodal models and PP+SP residual handling (#33322) Wang XingranandHongjian Zhang 2026-05-10 10:44:16 +08:00
dcb3135af7 Fix: Nemotron 3 rescue whitespace-only final_content, not just None (#41846) Nave Assafandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-10 05:07:58 +03:00

... 67 68 69 70 71 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86