iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
19,422 Commits 3 Branches 0 Tags
f7097a92fd5374ff8a44167b7bc16eec42393559
Commit Graph
19422 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Andreas KaratzasandOpenAI Codex f7097a92fd [Bugfix][CI] Prevent common ops imports from initializing CUDA (#50639) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-01 00:13:29 -04:00
Reid 62195e9784 [Rust][Benchmark] Prevent invalid token IDs in random benchmarks (#50058) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2026-08-01 03:20:29 +00:00
Jane (Yuan) Xu eb6453d95b [Build] Update pin to build ABI stable FA2 (#50474) 2026-07-31 20:09:36 -07:00
Amir Samani c4a4a42d80 [Bugfix][Test] Fix monolithic routing replay test buffer capacity (#50640) 2026-07-31 19:37:39 -07:00
Michael GoinOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		fcdc7c2e9c [CI] Organize speculative decoding E2E tests by coverage (#50330) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-31 18:05:42 -07:00
Shangdi YuOpenAI CodexClaude Opus 4.8
 		b40d859c7b [Kernel][Helion] Add numerics checks to benchmark script (#48968) 
Signed-off-by: Shangdi Yu <shangdiy@meta.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-31 20:16:02 -04:00
Mike GMichael Goinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e3be89673d Enable ModelOpt FP8 emulation on SM80 (#50019) 
Signed-off-by: Mike G <180722391+mikekg@users.noreply.github.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-31 14:48:27 -07:00
bnellnmClaudeTaneem IbrahimAgata DobrzyniewiczTushar JainTushar JainWentao YeJiangyun Zhu
 		454ea5b526 [MoE Refactor] Combine CompressedTensorsWNA16MarlinMoEMethod with CompressedTensorsWNA16MoEMethod (#44570) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Dobrzyniewicz, Agata <agata.dobrzyniewicz@intel.com>
Signed-off-by: Tushar Jain <tushar00jain@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Agata Dobrzyniewicz <160237065+adobrzyn@users.noreply.github.com>
Co-authored-by: Tushar Jain <8455015+tushar00jain@users.noreply.github.com>
Co-authored-by: Tushar Jain <tushar00jain@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-07-31 14:13:03 -07:00
Ning Xie 0c4dc7c488 [graceful shutdown] fix http server start firstly before app signal handler register (#49668) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-07-31 20:51:33 +00:00
Michael GoinandOpenAI Codex 90329913e9 [UX] Reduce startup log noise (#50590) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-31 13:04:34 -07:00
yzong-rhandBen Browning e8b358b4c0 [Bugfix][Responses] Add tests for Chat Completions Responses API Render Parity (#50334) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-07-31 15:45:40 -04:00
AdrianandClaude 4fdd3e0b74 [ROCm][ViT] Detect Triton-AMD kernels at their new aiter location (#40289) 
Signed-off-by: Adrian Kurt <info@zzit.ch>
Signed-off-by: Adrian <info@zipfuzapfu.ch>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-31 15:30:09 -04:00
Ning Xie 9a7ae4b808 [chore] log process manager shutdown with more details (#49437) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-07-31 11:55:09 -07:00
Ning Xie 726ef437a1 [chore] delete useless code (#49424) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-07-31 11:54:08 -07:00
Shangdi Yu 963a658674 Bump Helion to 1.4.0 (#50307) 
Signed-off-by: Shangdi Yu <shangdiy@meta.com>
 2026-07-31 11:06:49 -07:00
Soila KavulyaandClaude aef85aed5d [Bugfix][TurboQuant] Add KV quant mode for turboquant (#50533) 
Signed-off-by: Soila Kavulya <soila.p.kavulya@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-31 17:42:32 +00:00
fxmarty-amd e67a2e0a56 [ROCm][Quark][7/N] Use MXFP4 linear kernel abstraction for emulation backend (#48949) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-07-31 10:07:15 -07:00
Kyle Sayers d87d2ca747 [Compressed-Tensors] Support Kimi-K3 quantized models (#50500) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-07-31 17:06:30 +00:00
Andreas Karatzas 8d8a4e0f2b [ROCm][CI] Restore Mistral tool-parser compatibility after unification (#50515) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-07-31 16:32:43 +00:00
Dennis Yeh 7c08664f5c Upgrade tpu-inference to v0.26.0 (#50522) 2026-07-31 16:22:42 +00:00
ylangtsouandYlang Tsou 10ad649d6c Update torch version to 2.13.0+cpu (#50412) 
Signed-off-by: Ylang Tsou <ylangt@google.com>
Co-authored-by: Ylang Tsou <ylangt@google.com>
 2026-07-31 15:57:16 +00:00
LG c036cb257f [Doc] Add BgeM3EmbeddingModel to embedding supported models (#50571) 2026-07-31 15:56:01 +00:00
Matthew Bonanni a0cd2b69b3 [Bugfix] Universally align block table width to 128 tokens (#50302) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-31 08:52:17 -07:00
Harry MellorClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		94e9ef0768 [Bugfix] Don't transpose fused MoE quantization scales in RoutedExperts.load_weights (#50137) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-31 08:46:52 -07:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> df71917cf1 [DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. (#49236) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-31 08:45:58 -07:00
Andreas Karatzas 5233368dac [ROCm][CI] Fall back to lossless Kimi K3 MXFP4 emulation on gfx942 (#50516) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-07-31 11:33:16 -04:00
Matthew BonanniandOpenAI Codex 82ae4164ee [2/N][Attention] Enable masked MHA for sparse MLA prefills (#48770) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-31 06:01:24 -07:00
Nicolò Lucchesi 864a87febb [Kernel][CI] --jit-monitor-mode error e2e tests for kernel warmup infra (#50109) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-07-31 08:53:16 -04:00
Oğuzhan KIR 7fdc1ab29e [CI] Remove default_torch_num_threads workaround from llava-onevision-transformers test (#50560) 
Signed-off-by: oguz <oguzhankir17@gmail.com>
 2026-07-31 12:29:31 +00:00
Recoordinateandlatent-9 0b5b49dde8 [Bugfix][Frontend] Raise VLLMValidationError for user-facing errors in chat_utils.py (#50491) 
Signed-off-by: latent-9 <296084221+latent-9@users.noreply.github.com>
Co-authored-by: latent-9 <296084221+latent-9@users.noreply.github.com>
 2026-07-31 11:54:43 +00:00
wang.yuqi f5ffc59b6a [Renderer] Warm up the renderer properly. (#50408) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-07-31 11:46:53 +00:00
Itay EtelisItay EtelisOr OzeriItay Etelis
 		2c4d348848 [KV Connector] Add per-layer canonical KV page mappings for parallelism-agnostic offload (#48408) 
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Itay Etelis <Itay.etelis@gmail.com>
 2026-07-31 14:42:54 +03:00
Martin Hickey b2fb83e7ff [Attention]: Use KVCacheSpec for AttentionMetadataBuilder type hints (#50148) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-07-31 11:20:18 +00:00
Jee Jee Li 92643d68f5 K3 DSpark AR fusion (#50242) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-07-31 11:12:03 +00:00
Nicolò Lucchesi 17beffd55f [Misc] Clarify mono audio requirement (#50141) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-07-31 10:46:53 +00:00
bnellnmandHarry Mellor 6e311c6e20 [MoE Refactor] Rename FusedMoE to FusedMoEFactory (#44941) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-31 11:28:18 +01:00
Andreas Karatzas c911120a6f [ROCm][CI] Update Transformers AR+RMS fusion expectation (#50517) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-07-31 11:16:52 +01:00
Fadi Arafeh 0e9b50074e [chore] clean-up weight prepack for INT8 MoE (#50116) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-07-31 10:15:38 +00:00
Rehan KhanandLi, Jiang <jiang1.li@intel.com> 88bc8fb56f [CPU][s390x] Optimize inference perf and add oneDNN INT8 GEMM for s390x (#50219) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-07-31 17:49:31 +08:00
Chaojun ZhangandKunshang Ji 482cfc20f0 [XPU] Unify XPU RMSNorm kernels with vllm_c and drop redundant XPU-specific implementation (#46981) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-07-31 17:28:18 +08:00
Qiming Zhang 5d7647a109 [UT] add skipif for rocm aiter sampler UT (#50530) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
 2026-07-31 16:32:49 +08:00
Thien TranandCodex 34bb795ff3 [CI] Add M3 MSA tests to CI (#49143) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <noreply@openai.com>
 2026-07-31 01:01:48 -07:00
Taneem IbrahimOpenAI CodexIsotr0py
 		f727951d3f [Bugfix] Re-land MiniMax M3 default video processor (#50305) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-07-31 07:46:18 +00:00
Fadi Arafehjiang1.liClaude
 		10e6b40015 [CPU][BugFix] Remove redundant kv cache write (#50437) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-31 15:28:44 +08:00
Mi JiazhiandJiazhi Mi 3ee2bd1321 Fix duplicate HunyuanVL image boundary tokens (#49691) 
Co-authored-by: Jiazhi Mi <jiazhimi@tencent.com>
 2026-07-31 07:20:06 +00:00
xiangdong 0351e9aa1f [XPU][CI]Adjust source_file_dependencies for NixlConnector PD accuracy (4 GPUs) (#50373) 
Signed-off-by: zengxian <xiangdong.zeng@intel.com>
 2026-07-31 14:06:09 +08:00
Isotr0py 1180b604f8 [Multimodal] Expose mm hash algothrim selection to cli args (#49686) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-07-31 05:25:19 +00:00
Taneem Ibrahim 0f17394564 [Model Runner V2] Enable encoder token classification (#50293) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-07-31 04:15:36 +00:00
Sebastian WooandClaude bebf918044 [Bugfix][Model] Reject encoder-backbone jina-embeddings-v5 checkpoints with a clear error (fixes #50337) (#50352) 
Signed-off-by: seewoo <seewoo@ucsc.edu>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-31 11:43:31 +08:00
Jeff (Junze) MaClaude Opus 4.8mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Yongye Zhu
 		b49eaf205a [DSv4] Remove sparse-MLA q-head padding for FlashInfer >=0.6.14 (#48047) 
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-07-31 03:30:21 +00:00