iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,845 Commits 3 Branches 0 Tags
78300cdabffbaaffe7625ed34412ec2aec75535c
Commit Graph
20845 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Hill b1e12d142d [V1] Copy NaN-in-logits counts to host asynchronously (#51304) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 00:05:03 +00:00
Nicolò Lucchesi d35eb6c440 [CI] Exclude KV-connector subtree from broad source dependencies (#51046) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-06 16:51:10 -07:00
Matej SirovatkaOpenAI Codexaoshen02Claude Opus 5Roger Wang
 		8170c23c4f [Quantization] Share online weight scales across TP (#49764) 
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-08-06 16:34:33 -07:00
Giancarlo DelfinandNick Hill 27930df9c2 [Model Runner V2] Fix -1 placeholder draft token ids in rejection sam… (#50939) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-06 16:32:00 -07:00
Simon Mo 9bca7d840d docs(governance): refresh committers list, add TSC note, update project leads (#51300) 2026-08-06 16:28:17 -07:00
Taneem Ibrahim 9464529612 [ModelRunner v2] Enable decoder token-wise pooling (#50931) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-06 22:21:52 +00:00
Tejas 4d341ca829 fix: resolve silent request skipping in PRIORITY scheduling (#49206) 
Signed-off-by: Tejas-Raj01 <rajtejas.xyz@gmail.com>
 2026-08-06 22:21:21 +00:00
rongfu.lengNick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		a07086e403 [Misc] Upgrade fastsafetensors version, fix metadata is null (#50827) 
Signed-off-by: lengrongfu <lenronfu@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 14:48:56 -07:00
Nick Hill c5d470ac4c [ModelRunner V2] Minor indexing optimizations (#51210) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-06 14:44:13 -07:00
wangxian001andNick Hill d6af803f43 [Bugfix] Fix packed KV block zeroing stride (#50276) 
Signed-off-by: wangxian001 <120719093+wangxian001@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-06 13:49:06 -07:00
Wentao Ye adc3e03517 [Mypy Fix] Mypy fix for "vllm/model_executor/models/[aA][bB]" (#48977) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-06 19:30:15 +00:00
vanshbhatia-amdvanshbhatia-amdCursorMarkus HartikainenLiuyinfeng01
 		d8eabdbfbe [ROCm][MLA] Use asm decode for non-divisor small head counts (#50578) 
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Markus Hartikainen <markus.hartikainen@amd.com>
Co-authored-by: Liuyinfeng01 <yinfeliu@amd.com>
 2026-08-06 14:56:53 -04:00
Bugen ZhaoOpenAI CodexNick Hill
 		7b9f2dad89 [Frontend] Watch frontend processes during engine startup (#43417) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-06 11:17:24 -07:00
Divakar Verma 4f851bef6c [ROCm][CI] Update AITER AR+RMS e2e fusion counts for final-norm coverage (#51273) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-08-06 13:05:16 -05:00
Yifan QiaoHernanyanghui1-archClaude Opus 5
 		c56f169d9a [Bugfix] Keep mamba align prefill chunks block-aligned past last_cache_position (#51113) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Hernan <kodek@eat1337.com>
Co-authored-by: yanghui1-arch <3053034939@qq.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-06 10:21:00 -07:00
fxmarty-amdClaudemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		81be2e09ae [Weight processing] Copy over new_data attributes in replace_parameter (#49601) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 13:03:37 -04:00
Matthew BonanniClaude Opus 5OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b38e111d3e [Attention][MLA] Per-request scheduling for MLA chunked context (#50613) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 16:45:44 +00:00
Harry Mellor e7b8d59460 Fully generalise input embedding handling in Transformers modelling backend (#51247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-06 16:21:24 +00:00
Michael GoinandOpenAI Codex 566c80edf9 [CI] Run basic fullgraph correctness on one GPU (#51271) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-06 12:09:30 -04:00
gnovack 7b4ed49628 attn_res kernel latency improvements (#50185) 2026-08-06 08:53:20 -07:00
Chris LeonardandShengqi Chen 41e7746b82 Update vllm to point to flash-attention commit that builds FA3 with torch stable API. (Retry) (#49599) 
Signed-off-by: Chris Leonard <chleonar@redhat.com>
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-08-06 08:37:46 -07:00
Ning Xie 62a86318de [VocabParallelEmbedding] fix extra_repr fields concat (#51224) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-08-06 05:22:18 -07:00
Liangqiusong 1e05b21d61 Remove the XPU branch of topk_softplus_sqrt (#51242) 
Signed-off-by: xiaolong <xiaolong.guo@intel.com>
 2026-08-06 12:18:45 +00:00
AlexHuangandOpenAI Codex 46e6a83ce1 [Bugfix][KV Offload] Clean up resources after initialization failure (#51227) 
Signed-off-by: Alex <jihui.huang@daocloud.io>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-06 14:42:13 +03:00
yjzandJianDan0212 5fba75aefe [Bugfix][Model] Add missing fused_qkv_a_proj to Kimi-Linear packed_modules_mapping (#51249) 
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
Co-authored-by: JianDan0212 <zhangyj0212@gmail.com>
 2026-08-06 11:39:17 +00:00
Sage 865781e627 [Rust Frontend] Add standalone Rust renderer (#50289) 2026-08-06 11:23:26 +00:00
AlexHuangGuy GirmonskyClaude Opus 5OpenAI Codex
 		ef2615c2e0 [Bugfix][KV Offload] Fall back when MADV_POPULATE_WRITE is unsupported (#51116) 
Signed-off-by: Guy Girmonsky <guygir@gmail.com>
Signed-off-by: Alex <jihui.huang@daocloud.io>
Co-authored-by: Guy Girmonsky <guygir@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-06 13:48:01 +03:00
Lyu Han 22013f74ff Interns2mobius support (#51149) 
Signed-off-by: Lyu Han <lvhan_028@163.com>
 2026-08-06 10:39:35 +00:00
Jared Wen e07532b035 [MISC][Bench] refactor throughput and reuse serve's get samples (#50981) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
 2026-08-06 10:31:23 +00:00
anhtra3889andKhang Ngo 872fd5973e [Bugfix][LoRA] Guard TrtLlm BF16 MoE LoRA gate on activation type (#51002) 
Signed-off-by: Anh Tran <anhtra@nvidia.com>
Co-authored-by: Khang Ngo <khngo@nvidia.com>
 2026-08-06 18:12:54 +08:00
wang.yuqiandCyrus Leung 2fa490470d [Model] Fused mm preprocess normalisation on the Device (#50411) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-08-06 03:11:14 -07:00
Matej Sirovatka 9c22668436 [Quantization] Preserve precision in online NVFP4 expert packing (#50029) 
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
 2026-08-06 02:23:54 -07:00
Chaojun ZhangandKunshang Ji 276f0bb5c1 [XPU] Register fake meta kernel for fp4_gemm (#50946) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-06 17:18:42 +08:00
maobaolongandLi, Jiang <jiang1.li@intel.com> 13726c80fe [CPU] Add MLA backend so DeepSeek-V2/V3 can run on CPU (#49453) 
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-08-06 16:51:41 +08:00
Wentao Ye ad5280255b [GLM Perf] DSv32/glm use skip topk for MTP case, 2.0x kernel performance improvement (#50904) 2026-08-06 01:48:36 -07:00
xiaozhoupyClaudemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		2dfb8ba590 [Perf][CUDA] Programmatic dependent launch for the DSA decode kernels (#50230) 
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 00:53:17 -07:00
zofiaandopencode febea17f6a [Model] Fix weight prefix mapping for native Qwen3.5 text-only checkp… (#50355) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: opencode <opencode@anomaly.co>
 2026-08-06 07:40:44 +00:00
Cheng JiangchengvjiangVAthree
 		470297c143 [HPC Attention Backend] hpc attention backend support bf16 kv cache with fp8 weight (#50980) 
Signed-off-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: VAthree <56064364+VAthree@users.noreply.github.com>
 2026-08-06 15:40:14 +08:00
Andreas Karatzas 7e724dca89 [ROCm] Fix AITER all-reduce fusion coverage (#50802) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-06 14:52:23 +08:00
Canlin Guo 5370461649 [BugFix][KV Cache] Fix hybrid prefix caching with hidden-state extraction (#51108) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-08-06 06:42:37 +00:00
zofiaandKunshang Ji 8060260e1d [Linear] [Kernel] add block-wise scaled_mm (#49932) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-06 14:40:26 +08:00
liuzhenweiandKunshang Ji 8c0f029ac7 [XPU] update warning of XPU Graph (#50236) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-06 06:33:33 +00:00
Gabriel WuandYongye Zhu 2e35c529b8 [Bugfix][Quantization] Fix MXFP4 conversion for FlashInfer CUTLASS (#51038) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-08-05 23:19:22 -07:00
JooHo Lee c0202c5603 [Bugfix][Spec Decode] Auto-enable async scheduling for draft models (#48341) 
Signed-off-by: BWAAEEEK <jooho414@gmail.com>
 2026-08-06 13:51:40 +08:00
zofiaopencodeKunshang Jimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b976e980ef [XPU] Route AWQ linear through choose_mp_linear_kernel (#50840) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: opencode <noreply@opencode.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 13:50:11 +08:00
liuzhenwei 2a0323b4e0 [XPU][Test] Support MultiConnector accuracy testing on XPU (#51160) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-08-06 13:49:48 +08:00
Xiaochang WuCopilotKunshang Ji
 		d779835a11 [XPU] Support MXFP8 linear weights for INC DeepSeek V4 model (#48476) 
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-06 13:17:48 +08:00
Qiuyang Yue 777b01d1a8 [MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference (#45254) 
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com>
 2026-08-06 05:04:19 +00:00
Li JuqiandCyrus Leung 821717118f [BugFix][Pooling] Skip weight-prefix probe when model has WeightsMapper (#50890) 
Signed-off-by: Juqi Li <2223621784@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-05 21:12:33 -07:00
Yunzhu LuandCyrus Leung f84df12c6a [Bugfix][MM] Fix MiniCPM-V placeholder replacement and image processor loading on Transformers v5 (#48413) 
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com>
Signed-off-by: Yunzhu Lu <lucia.yunzhu@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-05 21:12:00 -07:00