iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,845 Commits 3 Branches 0 Tags
78300cdabffbaaffe7625ed34412ec2aec75535c
Commit Graph
1021 Commits
Author SHA1 Message Date
aoshen02Claude Fable 5.1vx120
 		560ef78bfe [Perf][Model Runner V2] Compact sampling masks on GPU instead of unpacking the full-vocab bitmask on CPU (#54901) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: vx120 <893600387@qq.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: vx120 <893600387@qq.com>
 2026-09-03 19:03:36 -07:00
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
Nick Hill e55b93f296 [Core] Deprecate "all" mamba cache mode (#55041) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-03 08:56:59 -07:00
Kevin H. LuuandKimi Code 6c6376a09e [CI] Remove deleted nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 and its arch aliases (#55026) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-02 15:18:46 -07:00
Isotr0py 1356635d83 [New model][Multimodal] Add DeepSeek-V4-Flash-Vision-Exp support (#54566) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-02 11:09:09 -07:00
fxmarty-amd 7894394b07 [Online quantization] Add targeted online quantization configuration based on user patterns (#51285) 2026-09-02 08:55:15 -04:00
Nicolò Lucchesiandcjackal f81eb41934 [Bugfix] adjust_dcp_kv_cache_interleave_size for NixlConnector only (#54803) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-09-02 12:31:14 +02:00
Wentao Ye 6c58595c2d [Feature] Avoid flashinfer autotune each time when vllm source change (#54794) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-01 15:03:30 -04:00
Andrii SkliarandAndrii Skliar adebc41b7e [Mamba] Add FlashInfer ReplaySSM backend (#52506) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
 2026-09-01 10:26:43 -04:00
Ziming HuangandOpenAI Codex 481839ad9e [Feature][Spec] Support disabling trailing prefix-cache block dropping (#53388) 
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com>
Signed-off-by: Ziming Huang <48115868+ZeldaHuang@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-01 19:12:36 +08:00
aoshen02aoshen02Claude Fable 5
 		4ac452ad98 [Core] Release NCCL communicator memory in sleep mode (#51485) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-01 14:31:16 +08:00
sychen52 7c5dc571cb [Attention][DSA] Enable W4A16 DSA (#51724) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
 2026-09-01 12:37:47 +08:00
Rohan PotdarClaude Opus 4.8mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ce2e343be1 [ROCm] Keep GLM-5.2 on MRV1 and disable default breakable cudagraph (#53155) 
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-31 23:17:54 -05:00
afriedriandAndreas Karatzas dc9114b201 [ROCm][MoE] Split AITER CK and Triton MXFP4 W4A16 into separate backends (#50622) 
Signed-off-by: afriedri <afriedri@amd.com>
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-31 23:14:14 -05:00
Lain 882ca8d696 [Kernel] add Flashinfer cutedsl w4a16 linear (#53014) 
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
 2026-09-01 02:49:14 +00:00
Nicolò Lucchesi dafbef15a1 [Core] Add max_num_queued_reqs and max_num_queued_tokens for queue size management (#49445) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-31 14:26:49 +00:00
范裕达Jee Jee LiJee Jee Li
 		da0b2d8b17 [Performance] Optimize Dots3 NOTE runtime (#53517) 
Signed-off-by: KurodaKanbei <mistergalahad@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-31 00:05:18 -07:00
+5
Harry Huangzjy0516KungYorkThien TranYifan Qiaoandyluo7Isotr0pyandyluo7OpenAI Codexlishunyang12Vadim GimpelsonIsotr0pyLucas WilkinsonClaude Opus 5OpenAI Codex
 		e126687a9a [Model] Support Qwen3.8-Flash-Next (#53896) 
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: KungYork <firelikest@163.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-31 05:57:56 +00:00
Kevin H. Luu b2dc864bb6 [Bugfix][Spec Decode] Keep default CUDA graph sizes memory-safe (#54418) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-08-30 18:02:31 +00:00
Nicolò Lucchesi 7f4793eaa3 [Nixl][PD] DCP support for MLA models (#50611) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-29 16:03:53 +00:00
Rahul ChalamalaJanelle CaiDevin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>Devin
 		d3d79ffc1e [Bugfix][Spec Decode] Capture the widest uniform decode batch by default (#50488) 
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Rahul Chalamala <rahul@modal.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin <devin@cognition.ai>
 2026-08-28 22:38:13 -07:00
Cheng JiangchengvjiangJee Jee LiBugen ZhaorussellfengJee Jee Li
 		b2f685834a [Hy4] support Hy4-preview model (#54160) 
Signed-off-by: chengvjiang <chengvjiang@tencent.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: russellfeng <russellfeng@tencent.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-29 12:07:28 +08:00
Nick Hill 8cfba696ee [Misc] Separate adaptive verification config validation (#54108) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-27 20:03:01 -06:00
Jiahan Chang (Cyrus)OpenAI CodexJee Jee Li
 		7c877062ac [kernel] Integrate FlashInfer BF16 CuTeDSL Low Latency GEMM (#50572) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-27 22:29:12 +00:00
Nick Hill 4aab2b0ebe [Model Runner V2] Use MRV2 for all models by default (#53183) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-27 11:59:34 -07:00
Itay Alroy 6d11122607 [Elastic EP] Preserve AOT cache reuse during scaling (#53378) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
 2026-08-27 14:55:44 +00:00
vllm-agentandKevin Luu d1e5e66ee3 [Bugfix] Restore portable all2all backend default (#53952) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-08-26 16:19:49 -07:00
Artem Perevedentsev 2cd6c664c1 [MoE] enable all2all fi_one_sided by default (#53311) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-08-26 17:03:45 +05:00
Qiu Chunshuo e376d45e82 [Config] Delegate PCP compatibility checks to PCP manager (#53853) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
 2026-08-26 01:16:01 -07:00
Donald Tolleyyewentao256mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		80771bbbdd [Core] Disable fuse_allreduce_rms under VLLM_BATCH_INVARIANT (non-deterministic under TP) (#51292) 
Signed-off-by: Don Tolley <tolleybot@gmail.com>
Signed-off-by: tolleybot <tolleybot@gmail.com>
Co-authored-by: yewentao256 <yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-25 15:16:09 -04:00
Tianyu Guo 12c84b98dd [Config][EC] Normalize producer-only encoder config (#53656) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-08-25 07:53:04 +00:00
Md Saidul Hoque AnikClaude Fable 5Anerudhan Gopal
 		8fe9317f2e [Model][MoE] DeepSeek-V4: add opt-in FlashInfer moe_ep expert backend (#49636) 
Signed-off-by: Md Saidul Hoque Anik <mhoqueanik@nvidia.com>
Signed-off-by: Anerudhan Gopal <agopal@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Anerudhan Gopal <agopal@nvidia.com>
 2026-08-25 03:35:01 +00:00
drakoshaandClaude 9bf0d4717e [Bugfix] Keep grid dims for XD-RoPE models on a prefix-cache hit (#53456) 
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-25 01:14:32 +00:00
Giancarlo Delfin d154d90d6c [Model Runner V2] batch-sharded sample (#50465) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-08-24 15:49:57 -07:00
Jee Jee LiandCyrus Leung 4f686e182a [MISC] Cleanup deprecated parameters (#53559) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-24 10:19:04 -07:00
Aaron Hao 6cd9713463 [RL] P2P RDT weight sync (#43375) 2026-08-23 09:42:22 -07:00
Brian WoodsClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		bbe8b23e1a [Deprecation] Remove dead use_prefill_decode_attention flag (#52557) 
Signed-off-by: Brian Woods <me@brian.biz>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-22 22:38:10 +00:00
Zhou248 2f55ef254c [Model] Add Qwen3-Omni DSpark support (#52560) 
Signed-off-by: Zhou248 <630563665@qq.com>
 2026-08-22 15:35:22 -07:00
Tianyu Guo 7ca49fbe4b [Refactor][Model Runner V2][Multimodal] Move the encoder-only path out of the shared runner (#53176) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-08-22 05:11:44 +00:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00
SoluMilken 27ec8ac626 [Bugfix] Fix MTP draft model using local cache path instead of S3 URL with runai_streamer (#42079) 
Signed-off-by: SoluMilken <ypiheyn.imm02g@g2.nctu.edu.tw>
 2026-08-21 09:07:08 -07:00
Lucas WilkinsonOpenAI CodexClaude Opus 5
 		72aedcc426 [DCP] Default query replication for GLM sparse attention (#50382) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-21 09:07:05 -07:00
Luke Alonsoandmgoin 2740c817ff [Kernel] Add b12x FP4 MoE backend (#52018) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-08-21 08:04:36 -07:00
Zihan Zhang b389ac2946 [Spec Decode] DFlash2: local convolution + candidate selector (#52816) 2026-08-20 22:27:22 -07:00
Matthew KotilaReed MeyersonClaude Opus 4.8
 		7cfb97e337 [Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915) 
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-20 15:42:49 -07:00
Isotr0py 00f7f25828 [Misc] Don't allow language-model-only used with encoder CG together (#53127) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-20 17:26:47 +00:00
Zupeng WangOpenAIMisha GoinOpenAI Codex
 		01af92e175 [Feature][Model Runner V2] Support extract_hidden_states speculation (#49811) 
Signed-off-by: Zupeng Wang <71580390+zupengwang@users.noreply.github.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI <support@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-20 11:13:43 -04:00
Prudhvi Vuda de216b6e64 [Bugfix] Skip MM processor cache inserts larger than capacity (#53016) 
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
 2026-08-20 13:42:50 +00:00
Harry Mellor f0c14b4f77 Fix weight tying (#51665) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-20 10:31:55 +00:00
Zhixia LiuianzxliuClaudeaoshen02CodexNick Hill
 		a1c5b1fd9f [Core][V1] Support trace_decode_token_ids for deterministic decode replay (#46701) 
Signed-off-by: ianzxliu <ianzxliu@tencent.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Zhixia Liu <liuzhixia.nju@gmail.com>
Co-authored-by: ianzxliu <ianzxliu@tencent.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-20 15:12:43 +08:00