iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
16,821 Commits 3 Branches 0 Tags
f2ace1d57d28df8d4c5e973dd62d87f47d628cb3
Commit Graph
5597 Commits
Author SHA1 Message Date
Nick HillandBugen Zhao f2ace1d57d [Frontend][RFC] Rust front-end integration (#40848) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-05-21 12:24:48 +08:00
손세정세덩sejung-sonsfeng33
 		d97ba29fdc [ToolParser][Bugfix] Re-land: Fix anyOf/oneOf/$ref type resolution in Qwen3CoderToolParser (#37831) (#38973) 
Signed-off-by: AAISSJ <maze0717@g.skku.edu>
Signed-off-by: <>
Signed-off-by: sejung-son <sejung.son@nhn.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: 세덩 <saison@sedeong-ui-MacBookAir.local>
Co-authored-by: sejung-son <sejung.son@nhn.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-05-21 12:24:08 +08:00
Ben Browningandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 346cf163a1 [Frontend] Normalize reasoning_content to reasoning for client compatibility (#42664) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-20 21:23:47 -07:00
haosdent 7e5070934e [CI] Fix "test_vit_cudagraph_[image|video][step3_vl]" failure (#43082) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-05-20 21:22:10 -07:00
Ace EldeibandFlora Feng 63ea11709b [CI] Add composed-schema regression tests for DeepSeek V3.2/V4 parsers (#43255) 
Signed-off-by: Ace Eldeib <aeldeib@coreweave.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-05-21 00:36:16 +00:00
akii96 bde560ed6e [ROCm] Add QuickReduce min-size override and codec threshold (#41675) 
Signed-off-by: <>
 2026-05-20 17:46:51 -05:00
Michael Goin 5774aad9c5 [Perf][gpt-oss] Downgrade triton_kernels to v3.5.1 (#43135) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-05-20 14:13:12 -07:00
meena-at-workandClaude Sonnet 4.6 363fc84407 Integrate flashinfer b12x MoE and FP4 GEMM kernels for SM120/121 (#40082) 
Signed-off-by: Meenakshi Venkataraman <meenakshiv@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-05-20 17:21:11 +00:00
haosdentandNickLucche f2d5e3d3ae [CI] Lower granite-4.0-h-tiny gsm8k threshold for Hybrid SSM NixlConnector PD accuracy tests (4 GPUs) (#43186) 
Signed-off-by: haosdent <haosdent@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
 2026-05-20 17:00:24 +00:00
Aaron HaoandIsotr0py 2d6b3489b9 [R3] Add routed experts to openai entrypoint (#38939) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-05-20 09:07:59 -07:00
Flora Feng a10d69116c [Bugfix] Use shared coerce_to_schema_type in DeepSeekV32 tool parser (#43019) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-05-20 10:21:00 -04:00
rishitdholakia13andCursor ded871201a [Bug][Structured Outputs] Fix bug that leads to unconstrained generations with structural tags (#42452) 
Signed-off-by: rishitdholakia13 <rishit+github@cohere.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-20 07:08:58 -07:00
Dipika Sikka df84fb07a6 Remove additional dead code as a follow-up to #42889 (#43144) 
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
 2026-05-20 10:01:45 -04:00
Kebe 19cf334207 [Feature] Support manually enabling the cumem allocator (#33648) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2026-05-20 08:58:30 -04:00
Ronen Schaffer 4f940896a3 [KV Offload] Pass OffloadingSpec instead of VllmConfig to secondary tiers (#43076) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
 2026-05-20 03:32:08 +00:00
Michael Goin cd0ff26e7a [CI] Add DSV4-Flash to gsm8k moe-refactor/config-b200.txt (#42111) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-05-19 20:21:01 -07:00
pmaybankgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		fadf5d332c add enqueue all option to throughput benchmark (#42975) 
Signed-off-by: Philip Maybank <pmaybank@amd.com>
Signed-off-by: pmaybank <113125070+pmaybank@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-19 20:16:02 -07:00
Benjamin Chislett c628a93a64 [Perf][Bugfix] Update dflash aux layer indexing (#40727) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-05-19 20:15:57 -07:00
Terrence Zhao 5774aaed0c [Cohere] Enable Cohere MoE (#43143) 
Signed-off-by: Terrencezzj <terrence@cohere.ai>
 2026-05-19 19:32:06 -07:00
Aaron Hao 73dd2f33b7 [bug] fix WeightTransferConfig.backend to allow for all strings (#43121) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
 2026-05-19 21:01:29 -04:00
Max de Baysergemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>wang.yuqi
 		117afeea46 Fix error in Dynamic NTK scaling (#41277) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-19 17:27:54 -04:00
tomeras91 f54721bcc3 [Bugfix][MoE] FlashInfer one-sided: workspace union across heterogeneous layers (#42976) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
 2026-05-19 14:43:04 -04:00
Dom Brown d247a931cc [feat] Add FP8 per-tensor Q scale support to Triton attention backend (#42080) 
Signed-off-by: Dom Brown <3886319+DomBrown@users.noreply.github.com>
 2026-05-19 09:02:05 -07:00
Flora Feng 42b4f1fdf7 [Refactor] Extract extract_types_from_schema utility from Minimax M2 tool parser (#43025) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-05-19 11:21:12 -04:00
Wang Yiwen 1c6158083a [Model] Openvla support (#42654) 
Signed-off-by: Wang Yiwen <121547057+yiwen101@users.noreply.github.com>
 2026-05-19 08:17:42 -07:00
Nick Hill b82e908b4c [Perf][4/n] Eliminate various GPU<->CPU syncs (#42347) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-05-19 10:35:54 -04:00
Sage a78b842d0e [Bugfix] Fix top logprobs token placeholders in /inference/v1/generate (#42887) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-05-19 10:21:49 +00:00
zhanqiuhuandNicolò Lucchesi 129019f334 [CI] Add MTP + PD disagg test for Qwen3.5 (#42677) 
Signed-off-by: ZhanqiuHu <zhu@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-05-19 11:44:33 +02:00
Woosuk Kwon 07beaed842 [Model Refactoring] Rename deepseek_v4.py to model.py [4/N] (#43077) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-05-19 01:12:46 -07:00
Yifan Qiao 056bc2e166 [KVConnector][DSV4] HMA support for Mooncake store connector (#42828) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-05-19 01:07:46 -07:00
Woosuk Kwon b14be81c1f [Model Refactoring] Move deepseek_v4_ops to models/deepseek_v4 [3/N] (#43073) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-05-19 00:52:54 -07:00
Qiuyang YueClaude Opus 4.7gemini-code-assist
 		fab07e4d0f [Bugfix][KV Connector] Fix SimpleCPUOffloadScheduler TOCTOU between Phase A and Phase B (#42289) 
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: gemini-code-assist <noreply@google.com>
 2026-05-18 21:22:33 -07:00
gnovackandgeorge 3ca8db2ef8 add cutedsl dsv4 indexer fp8 kernel (#42899) 
Signed-off-by: george <george@inferact.ai>
Co-authored-by: george <george@inferact.ai>
 2026-05-18 21:17:56 -07:00
Nicolò LucchesiandNick Hill fba010dd74 [Bugfix][MRV2] Fix KVCache tensor explicit kernel_block_size dim (#42766) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-05-18 20:25:41 -07:00
Mohammad Miadh Angkad da03e549b3 [UX] Add a persistent cache for FlashInfer autotuning (#42537) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-05-18 20:25:37 -07:00
Woosuk Kwon 287471b994 [Model Refactoring] Migrate DeepSeek V4 to vllm/models/ [1/N] (#43004) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-05-18 19:50:02 -07:00
Wentao Ye 37ece593c1 [Perf] Padded nvfp4 quant kernel to remove additional copy, 2.4%~5.7% e2e performance improvement (#42774) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-18 16:38:12 -07:00
Flora Feng 57fef4e0bf [Refactor] Extract shared coerce_to_schema_type utility from Minimax M2 tool parser (#43006) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-05-18 17:55:39 -04:00
Ronen Schaffer 84747489de Tier offload followup (#42529) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
 2026-05-18 19:41:58 +00:00
Bowen Bao a2c8fc6657 [ROCm][Quantization][3/N] Refactor quark_moe w4a4 w/ oracle (#41436) 
Signed-off-by: Bowen Bao <bowenbao@amd.com>
 2026-05-18 13:46:13 -04:00
Netanel Haber 47829b1159 [Bugfix] mamba: run single-token extends as decodes (#42430) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2026-05-18 15:26:00 +00:00
Blanc Swan 4a39b4f553 [Model] Add Apertus Tool Parser (#41154) 
Signed-off-by: Blanc <swan.blanc@infomaniak.com>
 2026-05-18 11:20:04 -04:00
Siddharth BedekarRobert ShawOpenAI Codexgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		78e7a7b9b0 Refactor AWQ Marlin MoE onto modular WNA16 oracle (#42483) 
Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com>
Signed-off-by: Siddharth Bedekar <104613085+bedeks@users.noreply.github.com>
Co-authored-by: Robert Shaw <robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-18 08:02:43 -07:00
liranschourOr Ozerimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e5417657e5 [KV Connector][Offloading] Flush all pending jobs on last step (#42611) 
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: liranschour <liranschour@users.noreply.github.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-18 12:59:42 +00:00
Nicolò Lucchesi 69c91d010a [MRv2] Default to MRv1 when a connector is present (#42955) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-18 20:34:16 +08:00
roikoren755 737bfa3a43 [Bugfix][Hybrid][NemotronH] Fix mamba_cache_mode=all + speculative decoding crash (#41233) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2026-05-18 14:54:00 +03:00
Yuwen Zhou 88a860d754 [CPU] Add MXFP4 W4A16 MoE support (#41922) 
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Yuwen Zhou <yuwen.zhou@intel.com>
 2026-05-18 03:04:45 -07:00
Jee Jee Ligemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Roger Wang
 		7d5b033782 [LoRA] Support 2D and 3D MoE LoRA adapter at the same time (#42242) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-18 15:22:26 +08:00
Nguyễn Thế DuyJee Jee LiJee Jee Li
 		e3aeee5ff8 [Bugfix] moe lora align kernel grid (#40131) 
Signed-off-by: TheDuyIT <nduy250299@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: dtnguyen <dtnguyen@nvidia.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-05-18 00:17:53 -07:00
Andreas Karatzas b50646e5ef [ROCm][CI] Stabilize ROCm pooling and multimodal CI (#42909) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-18 03:57:59 +00:00