iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,318 Commits 3 Branches 0 Tags
39eba6ac368133ab4e69a0589fcd53c481dddc44
Commit Graph
7366 Commits
Author SHA1 Message Date
Jensen ChenandClaude Opus 5 39eba6ac36 [Misc] Use VLLMValidationError in offline inference input validation (#53432) 
Signed-off-by: Jensen Chen <a1043904820@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-23 08:43:51 -07:00
Wyett b26039b09f Add routed expert loading for gpt-oss (#52209) 
Signed-off-by: Wyett <wyettzeng@gmail.com>
Signed-off-by: wyett <wyettzeng@gmail.com>
 2026-08-23 17:59:08 +08:00
Cyrus LeungandQwen Code ebcd606467 [MM] Simplify prompt updates: replace PromptSeq with list[int] (#53372) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Qwen Code <noreply@alibaba-inc.com>
 2026-08-23 14:50:32 +08:00
anmolgupt a3561ef8e4 [Bugfix][Model] Mistral3: fix image placeholder grid for processor size overrides (#52874) 
Signed-off-by: Anmol Gupta <14880251+anmolgupt@users.noreply.github.com>
 2026-08-22 21:34:38 -07:00
Andreas Karatzas 30b34171b1 [ROCm][CI] Restore attention coverage after KV-cache layout refactor (#53351) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-22 20:00:25 -05:00
Brian WoodsClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		bbe8b23e1a [Deprecation] Remove dead use_prefill_decode_attention flag (#52557) 
Signed-off-by: Brian Woods <me@brian.biz>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-22 22:38:10 +00:00
Zhou248 2f55ef254c [Model] Add Qwen3-Omni DSpark support (#52560) 
Signed-off-by: Zhou248 <630563665@qq.com>
 2026-08-22 15:35:22 -07:00
Shuolei Wangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 236f78cc5c [Core][RL] Support sparse checkpoint updates through native weight loaders (#50723) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-22 17:39:58 +00:00
Wentao YeZiming HuangOpenAI Codex
 		9eb9d9d395 [Perf] Support internal prefill checkpoints for Mamba prefix caching, 9%~25% TTFT improvement (#52789) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-22 10:32:40 -07:00
Cyrus Leung 0b19ebcacd [MM] Simplify _apply_hf_processor_main (#53275) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-08-22 01:49:37 -07:00
Tianyu Guo 7ca49fbe4b [Refactor][Model Runner V2][Multimodal] Move the encoder-only path out of the shared runner (#53176) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-08-22 05:11:44 +00:00
Yongye ZhuandOpenAI Codex e9d1398d9e [Bugfix][Kimi K3] Enable deferred MoE finalization before weight loading (#53327) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-22 11:23:32 +08:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 b2db227a7c [Bugfix][R3] Unwrap UniformTypeKVCacheSpecs when selecting the routed-experts KV group (#53240) 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
 2026-08-21 19:13:53 -07:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00
Misha Goin 9ff7041b55 [Bugfix][Spec Decode] Use group geometry for FlashAttention metadata (#53002) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-21 17:07:19 -07:00
Misha Goin a34f2ab70a [Test] Add focused hybrid MTP prefix-cache regressions (#53189) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-21 17:05:39 -07:00
Maroon Ayoub a556f3fccb Forward Anthropic vllm_xargs to sampling params (#53308) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
 2026-08-21 15:23:21 -07:00
andrewbcohere f94dcde5c5 Fix Cohere ChatV2 citation and tool handling issues (#52175) 
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
 2026-08-21 15:23:17 -07:00
stefankoncarevic f37d5868b9 [CI/Build][ROCm] Run the TileLang HIP symbol checks in their own interpreter (#53117) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-08-21 16:17:42 -05:00
Kevin H. LuuandAndreas Karatzas 592e06f2ae Revert "[ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill" (#53294) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-21 13:34:13 -07:00
Lucas WilkinsonandOpenAI Codex 0a3a4f7f35 [Bugfix][KV Connector][NIXL] Support PCP producers (#52779) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-21 16:22:01 -04:00
Andreas KaratzasItay EtelisOpenAI Codex
 		3e47a9a824 [CI] Fix MultiConnector accuracy test lifecycle (#53023) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-21 13:19:19 -07:00
Divakar Verma 29b7c2f7d4 [ROCm][CI] aiter kernel ops - enable rope test (#52854) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-08-21 15:17:34 -05:00
Andreas Karatzas a0af854f5d [ROCm][CI] Stabilize MI355 FlyDSL MoE accuracy test (#53024) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-21 13:17:18 -07:00
Divakar Verma b5f7fcc79d [ROCm][CI] Add float16 dtype and unsupported head size tests for paged attention (#53177) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-08-21 15:16:45 -05:00
Andreas Karatzas 88eb946cb1 [ROCm][CI] Stabilize MI355 FusedMoE test group (#53025) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-21 13:15:25 -07:00
djramic f15ea66b30 [ROCm][Test] Use platform FP8 dtype in ModelOpt FP8_PB_WO test (#53268) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
 2026-08-21 15:06:19 -05:00
Wentao Ye e6f35d3c69 [DSv4 Perf] Adaptive topk width for dsv4, making #50004 back (#52823) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-21 12:11:45 -07:00
Summer Yang d6c2fec9fd [Cleanup][MLA] Remove FlashInfer DSpark DCP support (#53139) 
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
 2026-08-21 11:42:52 -07:00
vllm-agentandvllm-ci-failure-analyzer ba53da60bb Revert "[Bugfix][MoE] Tune FlashInfer experts to scheduler token limit" (#52989) (#53186) 
Co-authored-by: vllm-ci-failure-analyzer <kevin@inferact.ai>
 2026-08-21 12:39:28 -06:00
Wentao YeYongye ZhuOpenAI Codex
 		7a2fdbaac4 [K3 Perf] Fuse MXFP4 top-k finalization into latent-tail, ~5% E2E latency reduction (#53152) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-21 13:29:59 -04:00
Fangzhou Ai fe76112ff2 [ROCm][Perf] Optimize DeepSeek V4 C4A top-k with AITER (#52882) 2026-08-21 10:00:39 -07:00
Tan Chao 1baf372bfc [Frontend] Use VLLMValidationError for batch request URL validation (#50191) 
Signed-off-by: Chao Tan <chaos.tc@gmail.com>
 2026-08-21 09:07:24 -07:00
Wei-Cheng (Wayne) Chiu b41dc4ec72 [Bugfix] Return HTTP 500 for non-streaming generate errors (#49195) 
Signed-off-by: WEI CHENG CHIU <waynehacking8@gmail.com>
 2026-08-21 09:07:17 -07:00
kliuaeClaudeTJian
 		463aa5e30f [ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill (#52606) 
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-08-21 09:07:13 -07:00
SoluMilken 27ec8ac626 [Bugfix] Fix MTP draft model using local cache path instead of S3 URL with runai_streamer (#42079) 
Signed-off-by: SoluMilken <ypiheyn.imm02g@g2.nctu.edu.tw>
 2026-08-21 09:07:08 -07:00
Lucas WilkinsonOpenAI CodexClaude Opus 5
 		72aedcc426 [DCP] Default query replication for GLM sparse attention (#50382) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-21 09:07:05 -07:00
Hongxia YangandYifan Qiao cd7b7c265a [ROCm] Cpu offload for ROCm 7.13+ to align the hipMemcpyBatchAsync params and perf in 7.14x (#43018) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-08-21 09:07:03 -07:00
Morrison TurnanskyandTyler Michael Smith 1183f04b74 [Bugfix] test_batch_inference_correctness now uses batch invariance (#48040) 
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-08-21 09:06:59 -07:00
Greg Pereira 18aa245a99 using existing uvicorn configuration for dp supervisor (#52473) 
Signed-off-by: Gregory Pereira <grpereir@redhat.com>
 2026-08-21 09:06:54 -07:00
Bugen ZhaoandOpenAI Codex 08b73d07cc [Frontend] Prevent Kimi K3 reserved markers in response text (#52889) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-21 09:06:52 -07:00
Lee Yongjunandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 6f74337c47 [Bugfix][Spec Decode]Preserve user --speculative-config overrides for speculators-format models (#42376) 
Signed-off-by: leeyongjun <jqueen.astro@gmail.com>
Signed-off-by: Lee Yongjun <35302114+elwhyjay@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-08-21 09:06:47 -07:00
yimdevandusberkeley 574e6a00ef [Bugfix][Attention] Normalize FlashInfer prefill LSE before merging (#52796) 
Signed-off-by: yimdev <5779256+yimdev@users.noreply.github.com>
Co-authored-by: usberkeley <150880684+usberkeley@users.noreply.github.com>
 2026-08-21 11:31:14 -04:00
Luke Alonsoandmgoin 2740c817ff [Kernel] Add b12x FP4 MoE backend (#52018) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-08-21 08:04:36 -07:00
Frederik GossenandClaude 5ee84d3c52 [CI][Bugfix] Use a prompt that survives offload-resume rounding in mamba offload test (#53146) 
Signed-off-by: Frederik Gossen <frgossen@meta.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-21 10:14:52 -04:00
Thanaji Rao ThakkalapelliandKunshang Ji 9fd750f00d [XPU][INC] Add int4 w4a8 (dynamic int8 activation) backend for INC linear layers (#50501) 
Signed-off-by: Thanaji Rao Thakkalapelli <thanaji.rao.thakkalapelli@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-21 14:38:59 +08:00
Wei ZhaoandOpenAI Codex f8e0602713 Support kimi k3 nvfp4 checkpoint (#53132) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-21 05:59:44 +00:00
Zihan Zhang b389ac2946 [Spec Decode] DFlash2: local convolution + candidate selector (#52816) 2026-08-20 22:27:22 -07:00
Thien TranOpenAI Codexwzhao18
 		2785c72a14 [Kimi-K3] Extend GEMM-RS to GEMM-AR (#53053) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-08-21 03:44:15 +00:00
Cyrus Leung b00f475f09 [MM] Remove text components from ProcessorInputs (#53093) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-08-20 20:42:53 -07:00