iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
20,220 Commits 3 Branches 0 Tags
00f7f258282ad3ea400a760ba3bc4679ef4e40fe
Commit Graph
20220 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Isotr0py 00f7f25828 [Misc] Don't allow language-model-only used with encoder CG together (#53127) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-20 17:26:47 +00:00
Dennis Yeh ae25628995 upgrade tpu-inference to v0.27.0 (#53088) 2026-08-20 16:21:39 +00:00
Harry Mellor 1fe3a1571a Reduce AutoWeightsLoader kwargs (#53106) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-20 15:40:11 +00:00
Harry Mellor 2dd17225c6 Fix Transformers modelling backend RMSNormFuser.fuse performance (#52766) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-20 15:21:23 +00:00
Zupeng WangOpenAIMisha GoinOpenAI Codex
 		01af92e175 [Feature][Model Runner V2] Support extract_hidden_states speculation (#49811) 
Signed-off-by: Zupeng Wang <71580390+zupengwang@users.noreply.github.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI <support@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-20 11:13:43 -04:00
Yiqinandshenyiqin cb09dd7488 [Core][Multimodal] Skip redundant placeholder scan when token match succeeds (#52925) 
Co-authored-by: shenyiqin <shenyiqin1@huawei.com>
 2026-08-20 15:04:29 +00:00
SeonjinandMisha Goin bd8865a299 [Kernel] Add FlashInfer TRTLLM MXFP8 linear backend (#52204) 
Signed-off-by: seonjinn <sna@nvidia.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-08-20 10:07:22 -04:00
Prudhvi Vuda de216b6e64 [Bugfix] Skip MM processor cache inserts larger than capacity (#53016) 
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
 2026-08-20 13:42:50 +00:00
Sandeep MaddipatlaandKunshang Ji 4b7cb949a9 [Docker] Update to nixl-1.3.2 (#51777) 
Signed-off-by: Sandeep Maddipatla <sandeep.maddipatla@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-20 21:03:32 +08:00
Zupeng Wang df1376907b [Model] Add tower and connector LoRA support for LFM2-VL (#51498) 
Signed-off-by: zupengwang <71580390+zupengwang@users.noreply.github.com>
 2026-08-20 20:33:43 +08:00
Thien TranandOpenAI Codex 6259572b28 [Docs] Use incremental builds for C++ changes in AGENTS.md (#53098) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-20 04:08:50 -07:00
Kevin H. Luu 6df7adc17f [Bugfix][GDN] Reset speculative decode count for an empty draft schedule (#53077) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-08-20 10:33:41 +00:00
Harry Mellor f0c14b4f77 Fix weight tying (#51665) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-20 10:31:55 +00:00
Harry Mellor 6e85feb1b7 [3/N] Harden Transformers modelling backend multi-modal path (#51827) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-20 10:24:41 +00:00
Turner JabbourandClaude Sonnet 5 1eab6fef01 [CI] replace shellcheck script with shellcheck-py hook (#52572) 
Signed-off-by: Turner <doubleujabbour@gmail.com>
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-08-20 10:09:59 +00:00
Ziming Huang 5b1e7a812b [BugFix][Mooncake] Fix Mooncake saves from sparse Mamba block tables (#51362) 
Signed-off-by: ZeldaHuang <zelda.huanghuang@gmail.com>
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com>
 2026-08-20 03:05:43 -07:00
Sagemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Bugen Zhao
 		727274a75b [Rust Frontend] Fix Qwen parser auto-detection (#51169) 
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-08-20 09:39:05 +00:00
Eoin-HoustounandClaude Fable 5 30e2394c83 [Bugfix] Record non-ImportError attention backend probe failures instead of crashing engine init (#51703) 
Signed-off-by: Eoin <eoin@turintech.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-20 02:36:00 -07:00
Andreas Karatzas 6b68db441e [ROCm] Fix DeepSeek V4 indexer numerics and coverage (#50803) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-20 09:34:00 +00:00
pmanczak 44cf3f0466 [XPU][Tests] Make tests device-agnostic (#51968) 
Signed-off-by: pmanczak <pawel.manczak@intel.com>
 2026-08-20 02:25:41 -07:00
JC-ut0 38e9cefdef [Bugfix] Return HTTP 400 instead of 501 for unknown chat roles in DeepSeek encoders (#53071) 
Signed-off-by: JC-ut0 <809602657@qq.com>
 2026-08-20 09:23:41 +00:00
rongfu.lengandIsotr0py c8de519917 [Kernel][Kimi] fused vision q/k roper kernel (#50400) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-20 09:15:47 +00:00
Taneem Ibrahim 5d4d470470 [CI] Fix nonexistent dependency for data-parallel example test selection (#53026) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-20 09:11:00 +00:00
Juan Pérez de Algaba c0a25c089a [Bugfix][Security] Guard _load_ov2_processor with resolve_trust_remote_code (#52952) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-08-20 09:09:49 +00:00
Roy WangClaude Fable 5Bugen Zhao
 		963fcfa48c [Rust][Benchmark] Align speed-bench CLI flags with Python and add flag parity test (#51592) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: esmeetu <jasonailu87@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-20 08:50:51 +00:00
Cyrus Leung 4666a8ba9e [Refactor] Remove InputPreprocessor (#53064) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-08-20 08:39:48 +00:00
Anton AandRoger Wang 14617c2b6c [Bugfix] DeepEP-V2: expert_tokens_meta must be None on the decode/cudagraph path (empty recv_expert_num_tokens) (#52632) 
Signed-off-by: Anton Alexander <dmvevents@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-08-20 01:11:44 -07:00
Kevin H. Luu a34fd69106 [CI][Bugfix] Update distributed DP API server test path (#52939) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-08-20 01:10:06 -07:00
Quentin GallouédecandBugen Zhao 16cfe728d8 [Bugfix][Rust Frontend] Reject n > 1 in the /inference/v1/generate route (#52844) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-20 07:41:06 +00:00
Ankit Nakhawa d66300a1ba [Bugfix][EPD] Fix encoder round-robin fan-out (#52491) 
Signed-off-by: AnkitNakhawa <ankitnakhawa@gmail.com>
 2026-08-20 07:30:40 +00:00
Zhixia LiuianzxliuClaudeaoshen02CodexNick Hill
 		a1c5b1fd9f [Core][V1] Support trace_decode_token_ids for deterministic decode replay (#46701) 
Signed-off-by: ianzxliu <ianzxliu@tencent.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Zhixia Liu <liuzhixia.nju@gmail.com>
Co-authored-by: ianzxliu <ianzxliu@tencent.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-20 15:12:43 +08:00
Qiming Zhang 754e1c3de4 [CI][XPU] Skip test_fused_shared_expert.py on XPU (#53035) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
 2026-08-20 14:56:09 +08:00
Shanshan Shen d626108b18 [ROCm][Perf] Fuse DeepSeek-V4 mHC post/pre and RMSNorm with AITER (#52737) 
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-08-20 04:51:04 +00:00
akii96 cd5035379c [ROCm] [Bugfix] Preserve CPU query offsets during capture (#51585) 
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
 2026-08-20 12:33:37 +08:00
Amit MoryossefClaude Fable 5Isotr0py
 		9b2aef5124 [Bugfix] Video loading: sample over presentable frames, not header sample count (MP4 edit-list trims) (#48608) 
Signed-off-by: AmitMY <amitmoryossef@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-20 04:23:09 +00:00
Xin Yang 11baa0ebd8 [Attention] Avoid redundant mask compute in GDN metadata build (#52078) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-08-20 11:52:42 +08:00
Andreas Karatzas 4f66bc3e0d [CI][ROCm] Standardize AMD test job labels by device (#52976) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-19 22:33:25 -05:00
Yufeng HeandChauncey e85dd21497 fix: report stop_sequence stop_reason in Anthropic Messages API (#45807) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-08-20 03:23:43 +00:00
pmanczakandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> f85e060ec2 [Doc] Update Gaudi HPU committers (#52726) 
Signed-off-by: pmanczak <pawel.manczak@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-20 10:59:22 +08:00
92hyungjunClaudeNick Hill
 		76fb6d210a [Bugfix][Core] Reserve the KV null block when validating max_model_len (#47272) 
Signed-off-by: hyungjun oh <92hyungjun@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-20 02:49:23 +00:00
Woosuk KwonandOpenAI Codex c0233fcf01 [Model] Remove unused DeepseekV32Indexer forward (#53021) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 19:39:52 -07:00
Micah WilliamsonAndreas KaratzasAndreas Karatzas
 		fbb4c04db4 [ROCm][CI] Speed up test_rocm_aiter_qk_norm_rope_kvcache_fusion (#53004) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-19 21:20:21 -05:00
Woosuk KwonandOpenAI Codex 6a962071bd [Distributed] Enable FlashInfer all-reduce by default (#52998) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 18:17:43 -07:00
Summer YangandWoosuk Kwon 58e5ee0158 [refactor] consolidate cp attn ops (#52839) 
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-19 18:04:11 -07:00
Guanyi Chenandz-zanez bf2866f8bf [KV Connector] Add decode offloading to Mooncake Store consumers (#52466) 
Signed-off-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: Guanyi Chen <939416532@qq.com>
Co-authored-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
 2026-08-19 17:40:35 -07:00
Giancarlo Delfin d4f4d3f40f [Model Runner V2][Spec Decode] Fix draft logits cache column stride in gumbel_sample (#53017) 2026-08-19 17:39:30 -07:00
Ronen Schaffer 0a111cca2c [kv_offload] fix(metrics): rename kv_offload_tiering_block_{queries,hits} → chunk (#52812) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
 2026-08-20 00:24:07 +00:00
Sage Moore 5bf0dbd6fd [Bugfix] vLLM crashes at startup when DeepEP v2 is used with --enforce-eager wiht TRTLLM Bf16 (#51824) 2026-08-19 20:18:47 -04:00
Rohan Potdar 823ec22b78 [ROCm]: Bump triton 3.7 commit (#52819) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-08-19 17:39:08 -05:00
jcotant-inferactClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c205726108 [CI] Fix and extend PR/issue auto-labeling (#51459) 
Signed-off-by: Joe Cotant <joe@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-19 21:45:44 +00:00