iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,744 Commits 3 Branches 0 Tags
18c19eaf239f09bc2543637c73f3b2d05f9bf5c5
Commit Graph
22744 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Taewoong KimandChauncey 18c19eaf23 [Frontend] Add cache salt and tracing to structured decisions (#60786) 
Signed-off-by: Taewoong Kim <ktw2172@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-10-09 08:53:50 +00:00
aoshen02Claude Fable 5.1indianspeedster
 		a6f43e57d6 [ROCm] Enable the cuMem CUDA-graph pool offload on ROCm (#59523) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: indianspeedster <cspandey016@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: indianspeedster <cspandey016@gmail.com>
 2026-10-09 08:29:55 +00:00
The Anh Nguyen bacbbe1878 [Bugfix][NIXL] Guard record_transfer against None telemetry (#57734) 
Signed-off-by: The Anh Nguyen <ntheanh201@gmail.com>
 2026-10-09 08:22:18 +00:00
Sriram Kumar 560fae64cd [ROCm][Bugfix] Pass per-sequence context lengths to the AITER paged MQA-logits kernels (#59420) 
Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com>
 2026-10-09 08:16:20 +00:00
Maroon AyoubandClaude 8a03ccf7b3 [Frontend] Add return_mm_kwargs to render requests (#60728) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-09 08:14:13 +00:00
Denis ErmakovClaude Opus 5Artem Perevedentsev
 		f43e4c39e3 [Structured Output] Allow propertyNames with additionalProperties on xgrammar (#57594) 
Signed-off-by: errmakov <ide404@gmail.com>
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-10-09 10:49:47 +03:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		efaca9547d [Perf][DSv4.1] Exchange packed FP8 Engram rows across DP and feed wkv MXFP8 (#59923) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-09 15:30:18 +08:00
Zheng CaizigzagcaiClaude Code
 		82e89e9944 [Perf][Qwen4Exp] Retune H200 M=4 merged QSA LL-GEMM plan (#60776) 
Signed-off-by: zigzagcai <zigzagcai@users.noreply.github.com>
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
 2026-10-09 07:18:50 +00:00
Jakub ZakrzewskiCodexMisha Goin
 		19c0bed584 [Fusion] Support 3D activation quantization fusion and enable it for Pixtral (#59612) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-09 15:12:03 +08:00
Chinmay-Kulkarni-AMDCursorMicah Williamson
 		dd30786fef [CI] Enable Zen CPU image build on AMD CI (#60016) 
Signed-off-by: Chinmay Kulkarni <Chinmay.Kulkarni@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
 2026-10-09 01:40:21 -05:00
Paweł MańczakandKunshang Ji f0a5f111f2 [XPU] Add tuned Mamba SSU configs for Intel Arc Pro B60 (#56765) 
Signed-off-by: pmanczak <pawel.manczak@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-09 14:29:29 +08:00
Canlin GuoClaude Opus 5.5Juntian Liu
 		ad73a4740d [Perf][DSv4.1] Trim the last KV-source layer in decoder replay (#59894) 
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Juntian Liu <Juntianl777@gmail.com>
 2026-10-09 06:07:49 +00:00
Matt MastracciDeepSeekClaude Opus 5.5Misha Goin
 		8cbd5d0300 [Frontend] structured decisions: noul and score question types (#60696) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: DeepSeek <noreply@deepseek.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-09 04:48:16 +00:00
Rohan PotdarandClaude Opus 5.5 88a419649e [ROCm][Kimi-K3] Use vLLM's FP8 max in the fused AttnRes quantization (#60717) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 04:45:13 +00:00
Aaron Haoaoshen02Claude Opus 5.5
 		fa312090cb [Frontend] Accept each connection on the least loaded API worker (#60723) 
Signed-off-by: hao-aaron <aaronhao@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 04:40:13 +00:00
Rajeev Warrier 81d9c45e4b [Docs] [Multimodal] Note that per-request media_io_kwargs needs --trust-request-mm… (#60598) 
Signed-off-by: Rajeev Warrier <warrier.rajeev.95@gmail.com>
 2026-10-09 04:23:07 +00:00
Kai HoangClaude Opus 5.5Flora Feng
 		e31b823bf5 [Bugfix][Tool Parser] Use LongCat tags in the LongCat structural tag (#60747) 
Signed-off-by: hpdkhoa <hpdkhoa2311@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-08 21:20:54 -07:00
Vadim GimpelsonClaudeFlora Feng
 		9824159d94 [Bugfix] Fix InternLM2 tool parser dropping characters from streamed arguments (#60709) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-08 21:20:49 -07:00
Vineeth Sai VarikuntlaandFlora Feng 112a282915 [Bugfix][Frontend] Keep streamed completion text_offset in step with tokens (#60668) 
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-08 21:20:43 -07:00
AlecBugen ZhaoClaude Opus 5.5
 		51ef239768 [Rust Frontend] Carry reasoning controls over gRPC (#59836) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 11:11:19 +08:00
Rémi Delhaye a1cd0ec650 [Model] Add zerank-2 reranker (#60415) 
Signed-off-by: rdlhio <contact@rdlh.io>
 2026-10-09 03:01:51 +00:00
zzaebokandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 702ce313f4 [Perf][Multimodal] Sort multimodal encoder inputs by modality (#60196) 
Signed-off-by: Jaebok Lee <jaebok9541@naver.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-09 10:55:36 +08:00
Tiezhen WANGandKevin H. Luu 295c062db8 Remove sudo from dockerfile command (#60488) 
Signed-off-by: Tiezhen WANG <38108242+xianbaoqian@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-08 19:24:56 -07:00
Mia OjedaandLucas Wilkinson 4611c2c3d2 [BugFix][DiffusionGemma][CPU] Copy async output snapshots instead of aliasing reused buffers (#59829) 
Signed-off-by: ojeda-e <eojeda@league.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-08 19:16:51 -07:00
Brian Weston, PhDCodexJee Jee Lilinitra24
 		201e537d98 [Bugfix][LoRA] Validate incompatible MoE adapter formats (#59924) 
Signed-off-by: Brian Weston <brianweston@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io>
 2026-10-09 09:49:13 +08:00
Nishant RevurandClaude 581692cc0f [Bugfix][Spec Decode] Fix multimodal embedding merge in Llama 4 and Mistral Large 3 EAGLE drafters (#57633) 
Signed-off-by: Nishant Revur <kumrfni@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-09 01:44:50 +00:00
DGlan 4274ae956c [Bugfix][EC Connector] Release shared memory on NIXL setup failure (#59875) 
Signed-off-by: DGlan <162426117+DGlan@users.noreply.github.com>
 2026-10-08 17:38:16 -07:00
Misha GoinClaude Opus 5.5Kevin H. Luu
 		ab905a885d [CI] Run GH200 test on the shared arm64 CI image (#60253) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-08 15:22:20 -07:00
KamranandCodex 8d10c38b73 [Perf] Reduce fallback decode CPU by 69% at 32K (#60497) 
Signed-off-by: spa5k <79936503+spa5k@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
 2026-10-08 21:42:56 +00:00
Yifan QiaoandClaude Opus 5.5 240785b82c [Bugfix] Keep the GLM-5.3 kpool tail and Qwen4 QSA ring out of the null block (#59528) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 20:44:04 +00:00
sungbin1015andFlora Feng 2ddde19e8e [Frontend] Support min_p in the Responses API (#48084) 
Signed-off-by: sungbin1015 <sbin@solbox.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-08 20:38:17 +00:00
Divakar Verma c5c1161382 [ROCm][Build] Fail fast when setup.py develop deps aren't pip-preinstalled (#59447) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-10-08 15:28:38 -05:00
Francisco Javier ArceoCodexCyrus Leung
 		ae9adec905 [Frontend] Expose /v1/systemone without an opt-in flag (#60677) 
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-08 20:21:05 +00:00
Alberto MassiddaAlberto MassiddaClaude Fable 5.1Artem Perevedentsev
 		72d59adc2c [Bugfix][KV Connector][NIXL] Transfer the whole PLE short-conv page in disaggregated P/D (#59997) 
Signed-off-by: Alberto Massidda <amassidda@nvidia.com>
Co-authored-by: Alberto Massidda <amassidda@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-10-08 23:58:13 +04:00
Matthew BonanniandClaude Opus 5.5 1afb029839 [Bugfix][HiSparse] Route by swap-row capacity, not the index-group workspace (#60682) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 19:45:44 +00:00
simon-veitner-redhatClaude Opus 5.5Misha GoinYifan Zong
 		c23ca06b6d [Bugfix][Structured Output] Feed the token that implicitly ends reasoning to the grammar (#59627) 
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Yifan Zong <263738792+yzong-rh@users.noreply.github.com>
 2026-10-08 19:38:33 +00:00
dingdangmaoandFlora Feng bab0ac61f8 [Bugfix][Frontend] Forward the reasoning wiring to the engine for batch chat completions (#59263) 
Signed-off-by: apex-mochen <2756823972@qq.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-08 19:32:46 +00:00
Misha GoinandClaude e3a29aa9b3 [Bugfix][MoE] Size the permute scratch from its inputs (#60447) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-08 15:32:07 -04:00
SeongJun LeeandMisha Goin fff10d2072 [Attention] Allow an NVFP4 KV cache on SM90 with fa2 (#60620) 
Signed-off-by: lesj0610 <lesj0610@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 12:16:22 -07:00
Kevin LiandMisha Goin fc494822ea [Test][Quantization] Gate CUTLASS W4A8 kernel tests to SM90 (#58878) 
Signed-off-by: Kevin Li <kevinbean99@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 19:14:35 +00:00
Divakar VermaandAndreas Karatzas 18751bc6a7 [ROCm][CI] Add coverage for AiterExperts token padding: inf/nan garbage rows (#59334) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-08 19:11:45 +00:00
Aarushi JainandAndreas Karatzas 9bbc581560 [Bugfix] Close MessageQueue zmq resources via finalizer to avoid exit (#60472) 
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-08 13:53:00 -05:00
Yeonwoo SungandFlora Feng 56e6da2c47 [Bugfix][Frontend] Honor stop_token_ids on the Responses API (#56592) 
Signed-off-by: YeonwooSung <neos960518@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-08 18:47:30 +00:00
Ning XieandSimon Mo c4a5ff5c18 [Frontend] shutdown engine core and its sub processes asap when initializing (#52299) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2026-10-08 11:46:01 -07:00
Shijin Zhang 2274e86842 [Spec Decode][Model] Support DFlash2 draft models with GLM-5.3-Flash (#56983) 
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
 2026-10-08 14:41:10 -04:00
Misha GoinandClaude 30c7405890 [MoE] MXFP4 oracle: add CUTLASS W4A4 backend and BF16 activation fallback (#60079) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-08 11:39:09 -07:00
Wei ZhaoWei ZhaoCodexzjy0516Claude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		d8fa463e2b [Bugfix][Prefix caching] Fix inconsistent hybrid Mamba prefix cache boundaries (#60533) 
Signed-off-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-08 18:33:48 +00:00
kyleliang-nvKyle LiangOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		1664c4f444 [Bugfix][MiniMax-M3] Guard sparse decode sentinel indices (#54597) 
Signed-off-by: Kyle Liang <kylliang@nvidia.com>
Signed-off-by: Kyle Liang <kyleliang-nv@users.noreply.github.com>
Co-authored-by: Kyle Liang <kyleliang-nv@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-08 18:27:57 +00:00
TialoandMisha Goin 972195f4f5 [CI][Build] build macos wheels using abi3 (#60312) 
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 14:19:20 -04:00
Francisco Javier ArceoandMisha Goin 705e124c05 [Bugfix] Followup to #59299: preserve Unicode in structured decision state prompts (#60651) 
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 11:00:03 -07:00