iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
19,601 Commits 3 Branches 0 Tags
b976e980efdfb95d0ade2ff6658ca736c737f1bf
Commit Graph
19601 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
zofiaopencodeKunshang Jimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b976e980ef [XPU] Route AWQ linear through choose_mp_linear_kernel (#50840) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: opencode <noreply@opencode.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 13:50:11 +08:00
liuzhenwei 2a0323b4e0 [XPU][Test] Support MultiConnector accuracy testing on XPU (#51160) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-08-06 13:49:48 +08:00
Xiaochang WuCopilotKunshang Ji
 		d779835a11 [XPU] Support MXFP8 linear weights for INC DeepSeek V4 model (#48476) 
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-06 13:17:48 +08:00
Qiuyang Yue 777b01d1a8 [MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference (#45254) 
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com>
 2026-08-06 05:04:19 +00:00
Li JuqiandCyrus Leung 821717118f [BugFix][Pooling] Skip weight-prefix probe when model has WeightsMapper (#50890) 
Signed-off-by: Juqi Li <2223621784@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-05 21:12:33 -07:00
Yunzhu LuandCyrus Leung f84df12c6a [Bugfix][MM] Fix MiniCPM-V placeholder replacement and image processor loading on Transformers v5 (#48413) 
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com>
Signed-off-by: Yunzhu Lu <lucia.yunzhu@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-05 21:12:00 -07:00
Chauncey 22170354d8 [Feature] Parse request priority from HTTP header (#51089) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-08-05 21:10:29 -07:00
baodi 2e09247c2d [Docs] List Intel XPU attention backends (#51215) 
Signed-off-by: baodii <di.bao@intel.com>
 2026-08-06 10:58:28 +08:00
liuzhenwei d54b58cc48 [Hardware] Use torch.accelerator.empty_host_cache() for host cache cl… (#51107) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-08-06 10:27:57 +08:00
Giancarlo Delfin 81bc196913 [Model Runner V2] Cache draft logits in model's LM head dtype (#50910) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-08-05 19:12:46 -07:00
Jee Jee Li f85c1d2f84 K3: remove the add operation for megamoe path (#51146) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-08-06 01:48:25 +00:00
Aarushi Jain 71f975af43 [ROCm][CI] Add MLA decode accuracy and determinism tests (#50480) 
Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com>
 2026-08-05 20:35:56 -05:00
Thien TranandOpenAI Codex 9f3169960a [MoE] Align TRTLLM MXFP4 autotune buckets (#50942) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-05 18:17:53 -07:00
Silen NaihinDakai AnJiangyun Zhu
 		b50fdebce0 [Bugfix] Fix level-2 sleep/wake/reload with enable_lora=True (#39935) 
Signed-off-by: Silen Naihin <silen.naihin@gmail.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-08-06 09:13:10 +08:00
aoshen02andAndy Lo 8cfa01cdd2 [BugFix] Dense multinode DP rescope with regression test (#49212) 
Signed-off-by: Andy Lo <andy@mistral.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Andy Lo <andy@mistral.ai>
 2026-08-05 18:12:03 -07:00
gabriel-peracioandKimi K3 47a4e410ba [Bugfix][Spec Decode] Fix NaN handling in rejection sampler tl.argmax (#50183) 
Signed-off-by: Gabriel Peracio <gabriel@sigsonic.com>
Co-authored-by: Kimi K3 <noreply@moonshot.ai>
 2026-08-05 18:01:09 -07:00
LZWandRoger Wang d36f24b66a [KV Connector][Mooncake] Add tenant ID support to MooncakeStoreConnector (#48069) 
Signed-off-by: lzw <2994587066@qq.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-08-05 17:31:18 -07:00
aoshen02Matej SirovatkaRishi PuriHsiao-Yuan ChenClaude Opus 5
 		7c77868cdf [Bugfix] Size and iterate w13 by shard count for non-gated MoE (#51125) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Matej Sirovatka <matej.sirovatka@gmail.com>
Co-authored-by: Rishi Puri <riship@nvidia.com>
Co-authored-by: Hsiao-Yuan Chen <littlecircle0730@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-05 17:24:47 -07:00
QiuandLucas Wilkinson 811622c410 [Refactor][PCP] Make PCPManager construction extensible (#50066) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-08-05 23:05:14 +00:00
Tianyu Guo b30291cf0c [EPD] Remove duplicate image preprocessing in EPD and enable preprocess on GPU (#50390) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-08-05 15:59:54 -07:00
Rohan PotdarandClaude Opus 4.8 38ebd97bca [ROCm] Relax MLA rope+cache test tolerances for bf16 (#51083) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-05 17:50:52 -05:00
Wentao Ye 877975897d [K3 Perf] Combine multiple all gather together for SP, 1.5~3x kernel level performance improvement (#51070) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-05 17:42:41 -04:00
bnellnm 373fe8b83e [MoE Refactor] Remove MoE legacy code (#51078) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-08-05 14:35:36 -07:00
Micah WilliamsonandAndreas Karatzas 4282fe52d5 [ROCm][CI] Add More AITER quantization/MoE kernel tests (#49375) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-05 21:27:33 +00:00
Netanel Haber 482c613423 [Bugfix][Humming] Preserve ModelOpt FP8 weight dimensions (#51093) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2026-08-05 14:08:55 -07:00
Wentao Ye e76b71f659 [CI Bug] Fix pydantic_core._pydantic_core.ValidationError: Input should be a valid integer (#51179) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-05 20:57:47 +00:00
stefankoncarevicandAndreas Karatzas 65addac701 [ROCm][CI] Keep rocprofiler-sdk out of DeepEP HT MoE test workers (#51173) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-05 15:28:41 -05:00
Rohan PotdarandClaude Opus 4.8 c2d8009044 [ROCm] Work around DeepEP teardown SIGSEGV in MoE test harness (#51174) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-05 14:44:42 -05:00
Michael Goin e6d67fddb4 [CI] Prune PyTorch Fullgraph Test (#51074) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-05 12:27:27 -07:00
Michael GoinandOpenAI Codex 8f158d0ee2 [MoE] Share apply_moe_activation support metadata (#44359) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-05 12:05:47 -07:00
Zhewen LiZhewen LiClaude Opus 5
 		14e57ad47d [Docker][KVConnector] Install mooncake from official wheels instead of a custom build (#51067) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-05 11:58:44 -07:00
Wentao Ye 613411a90c [CI Bug] Fix Chunked prefill is required for mamba cache mode 'align'. (#51177) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-05 11:33:45 -07:00
fxmarty-amd bc37fc970e Revert [Misc] Avoid importing nixl_ep on every vllm serve config (#50879) (#51176) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-08-05 11:04:40 -07:00
Wentao Ye 23c0f337b7 [CI bug] Fix Each KV cache group's real block_size must be divisible by has h_block_size (#51180) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-05 14:03:17 -04:00
AlexHuangOpenAI CodexOr Ozeri
 		9833aa53d5 [Bugfix] Fail fast with a clear error when CPU offload region exceeds available space (#50358) 
Signed-off-by: Alex <jihui.huang@daocloud.io>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-08-05 20:59:55 +03:00
Michael GoinandShengqi Chen e8586b2878 [Build] Remove Ubuntu build-stage option from the CUDA dockerfile (#51060) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-08-05 10:50:58 -07:00
Ronen SchafferandOr Ozeri 8543522ca7 [KV Offload] Support out-of-tree secondary tier managers via module_path (#51007) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-08-05 20:30:53 +03:00
zexplorerhj d4da0c55af [Model][Frontend] Add Ling 3.0 Flash BF16, MTP, and parser support (#51045) 
Signed-off-by: zexplorerhj <zhjoneson@163.com>
 2026-08-06 00:38:28 +08:00
Li JuqiMatthew BonanniClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		66b3c0e61f [Kernel][Model] Optimize FA4 mm_prefix range lookup (#50294) 
Signed-off-by: Juqi Li <2223621784@qq.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-05 07:33:49 -07:00
ChaunceyandOr Ozeri 62c5e21621 [KV Offloading] Support partial-tail prefix reuse with fine-grained prefix matching (#50507) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-08-05 22:31:50 +08:00
Gabriel WuandPavani Majety 08b8613b7b [Bugfix][Model] Fix MiniMax-M3 NVFP4 inference correctness (#48929) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Pavani Majety <pmajety@nvidia.com>
 2026-08-05 07:25:46 -07:00
Jacob ZhangMatthew Bonannimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		cd930c8e78 [Bugfix] Fix MLA kv_b_proj activation dtype with Marlin FP8 (#38771) 
Signed-off-by: Jacob Zhang <jaco8123@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-05 07:25:41 -07:00
LucainOpenAI CodexClaude Opus 5Harry Mellor
 		397094da17 Resolve revision to commit_hash once per model load, via huggingface_hub's resolve_revision (#49990) 
Signed-off-by: Wauplin <lucainp@gmail.com>
Signed-off-by: Lucain <lucainp@gmail.com>
Signed-off-by: Lucain Pouget <lucain@huggingface.co>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-05 13:49:56 +00:00
music-dino a9b39d6dcb [Bugfix] Enable chunked prefill for qwen3.5-0.8B ppl test (#51153) 
Signed-off-by: Dino Music <Dino.Music@amd.com>
 2026-08-05 13:21:43 +00:00
Ziming Huang beca88e59e [BugFix][K3] Skip moe_intermediate padding when EP is enabled (#51131) 2026-08-05 09:22:55 +00:00
Kevin H. LuuandOpenAI Codex a3b86752fb [CI] Fix CI authorization notification fallback (#51095) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-05 01:34:48 -07:00
SageandBugen Zhao 5bf32605a9 [Rust Frontend] Deduplicate request preprocessing for /tokenize (#50448) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-05 08:27:35 +00:00
SchattenOpenAI CodexYifan Qiao
 		fd859d2e06 [KV Connector][Mooncake] Add store group semantics (#44956) 
Signed-off-by: Schatten <czhengt@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-08-05 08:25:38 +00:00
kliuae f5cd862dbd [ROCm][Bugfix] Kimi-K3 Fix KDA NaN on mixed batches and racy autotune config (#50649) 
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
 2026-08-05 08:07:21 +00:00
MINJUN GIL b92352ca2c [Perf][KV Offload] Avoid quadratic ARC batch eviction (#50992) 
Signed-off-by: MINJUN GIL <alswnsrlf12@naver.com>
 2026-08-05 10:58:36 +03:00