iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,866 Commits 3 Branches 0 Tags
83773ecdf3ce6da9a2f2bd14cb87e6bbf58d4229
Commit Graph
22866 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Shijin ZhangandChauncey 83773ecdf3 [Frontend] Support CustomToolCall in the non-Harmony Responses API (#57322) 
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-10-12 17:22:55 +08:00
zhangchengzhucufe-devandKunshang Ji a33d46b471 [Bugfix][XPU] Forward softcap and alibi_slopes in xpu_ops.flash_attn_varlen_func (#58936) 
Signed-off-by: zhangchengzhucufe-dev <zhangchengzhucufe@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-12 17:10:55 +08:00
Yan Ma e05201aa2e [XPU] refine memory release for sleep mode (#61158) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-10-12 16:59:38 +08:00
Thuongvu HoandNicolò Lucchesi 99b44c7a42 [Metrics] Add admission rejection metrics (#55812) 
Signed-off-by: Thuongvu Ho <thuongvuho@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
 2026-10-12 08:31:07 +00:00
Cyrus Leung 5bed289707 [DX] Update CODEOWNERS for entrypoints (#60984) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-10-12 08:06:47 +00:00
Shuolei Wangaoshen02Bugen ZhaoClaude Opus 5.5
 		4163a80142 [Rust Frontend] Return weight-version spans on final outputs (#61070) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 15:56:59 +08:00
Ting SUN f4114e068e [Bugfix][Frontend] Preserve token IDs in chat derender responses (#61065) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-10-12 07:55:06 +00:00
AMD-yanfeiwangandaoshen02 e3bbaf2369 [ROCm][DSv4] Fuse the shared expert into AITER MegaMoE (#61004) 
Signed-off-by: Yanfei Wang <yanfei.wang@amd.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-12 15:53:16 +08:00
Giancarlo DelfinandRishi Puri 36f3972812 [Spec Decode] Skip metadata build for dflash/dspark when all model backends support it (#60692) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Rishi Puri <20074092+puririshi98@users.noreply.github.com>
 2026-10-12 00:09:10 -07:00
Zhenzhong XuandKunshang Ji 6a70913590 [XPU] bump up auto-round-lib to 0.16.0 (#60976) 
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-12 14:47:59 +08:00
HubertCodexAo Shen
 		74f4404f58 [Bugfix][Distributed] Own SHM out-of-band buffers before releasing read slots (#61000) 
Signed-off-by: Hubert Marek <163992334+hubert-marek@users.noreply.github.com>
Signed-off-by: Ao Shen <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Ao Shen <aoshen@inferact.ai>
 2026-10-12 01:44:12 -04:00
Canlin GuoandClaude Opus 5.5 fe4df3df0e [Bugfix][GLM-5.3-Flash] Restore MTP index sharing across draft iterations (#61079) 
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 05:41:52 +00:00
shaohuaxiCodexCyrus Leung
 		d4e92ccc37 [Bugfix][Frontend] Honor endpoint plugin route overrides (#58839) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-12 03:58:15 +00:00
liuzhenweiandKunshang Ji bfa7e5b7af [XPU][CI] Enable XPU graph in graph tests (#60784) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-12 10:39:40 +08:00
Yan Ma f147949be8 [XPU] limit FA kernel block size to multipleOf 64 (#57297) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-10-12 10:25:32 +08:00
dingdangmaoandCodex 124b89d5b7 [Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries (#59962) 
Signed-off-by: apex-mochen <2756823972@qq.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-12 02:08:14 +00:00
Michał GanczarenkoandClaude Opus 5.5 03091e28b8 [Bugfix] Read Ling quantization_config from top-level config for Bailing V3 VL (#59026) 
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 09:25:29 +08:00
Laith Sakka cc9b0f097d [Fix] MiniMax: fix CUDA IPC handle exchange on cuda-bindings >= 13.4 (#60917) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2026-10-11 20:38:56 -04:00
Nick Hill 3fe917abb1 [Bugfix][MRV2] Fix FlashInfer attention and ReplaySSM warmups on MRV2 (#61013) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-11 16:04:41 -07:00
Jared WenClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f26d9f432c [Bugfix][GLM-5.3-Flash] Honor --mamba-ssm-cache-dtype for the KDA recurrent state (#59147) 
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-11 17:55:00 +00:00
Nick Hill c059bee812 [MRV2] Switch CI tests still running on MRV1 unnecessarily (#61014) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-11 09:57:07 -07:00
Ting SUNandCyrus Leung b99fdbe1f8 [Bugfix] Respect the configured offline chat template (#61061) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-11 16:27:21 +00:00
Juan Pérez de AlgabaRobert ShawItay Etelis
 		93d2cab35e [Security] Fix P2P ZMQ socket exhaustion via unbounded peer sessions (#51504) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
 2026-10-11 11:39:11 -04:00
aoshen02andCodex 239a3fa3a8 [Sleep] Retain draft-private weights with existing parameter selector (#61046) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-11 23:30:56 +08:00
Zheng CairootOpenAI CodexzigzagcaiClaude CodeWentao Ye
 		3e8e1364ff [Perf][GLM-5.3-Flash] Shard long-context indexer prefill rows across TP (#54951) 
Signed-off-by: Zheng Cai <8370601+zigzagcai@users.noreply.github.com>
Signed-off-by: Zheng Cai <caizheng1993@gmail.com>
Signed-off-by: Codex <noreply@openai.com>
Co-authored-by: root <root@rjob-fb29567b88fec0a9-88c9a4c797eb2ec1-0.caizheng.ailab-sys.svc.pjlab.local>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-11 10:43:37 -04:00
aoshen02 bd5bbce6e4 [RL] Release weight-transfer communicators without stopping inference (#61060) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
 2026-10-11 21:15:14 +08:00
tc-mbandCyrus Leung c85616efc6 [Model] Support MiniCPM-V 4.7 (#58674) 
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-11 10:21:10 +00:00
Subhash Polisetti f425a21208 [Bugfix][Frontend] Support scalar priority and wire priority into LLM.chat() (#57243) 
Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
 2026-10-11 10:03:33 +00:00
Stu CaoClaude Fable 5.1Yifan QiaoCodex
 		2339bd39d6 [Bugfix][Mooncake] Use the EAGLE replay boundary when retaining sliding-window tails (#61012) 
Signed-off-by: Stu Cao <stu@primeintellect.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-11 09:15:18 +00:00
aoshen02andOpenAI Codex 1789f09588 [Frontend] Explain disabled scale-out endpoints with opt-in hints (#61042) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-11 08:58:07 +00:00
Juan Pérez de AlgabaandCyrus Leung fcae5c9887 [Security] Restrict default CORS origins (#61009) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-11 08:26:02 +00:00
Xun Sun 26a488ad93 [Fast Start][Bugfix] Snapshot GPU memory after external weights are ready (#60966) 2026-10-11 07:10:04 +00:00
debanshdIsotr0pyKimi Code
 		ef42b4385f [Bugfix] Reclaim cached GPU memory during weight loading (#57516) 
Signed-off-by: Debanshu Das <debanshu@google.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-10-11 07:03:06 +00:00
Bruno VolpatoandCodex 61f084b5f3 [Bugfix][Multimodal] Fix per-video cache option reuse (#55203) 
Signed-off-by: bvolpato <brunocvcunha@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-11 06:21:55 +00:00
Yongye ZhuandClaude Opus 5.5 2eb491e18b [Perf][MiniMax-M3] MSA prefill attends with the E4M3 query; bump fmha_sm100 to 968061a (#60684) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 13:40:45 +08:00
Nick Hill f8471164bf [MRV2] Detatch weight-reload logic from MRV1 (#60937) 2026-10-10 21:15:05 -07:00
bzsuni 01abe975b5 [Bugfix][KV Connector] Fix speculative block reservations across KV cache groups (#59880) 
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
 2026-10-10 20:55:41 -07:00
Shuolei Wangandaoshen02 b89852aad0 [RL][Frontend] Return token-level weight-version spans on final outputs (#59582) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-11 11:33:29 +08:00
Subhash Polisettiaoshen02Codex
 		3c062d391d [Bugfix] Support 0-dim tensors in packed weight transfer (#52235) 
Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-11 01:41:06 +00:00
Wentao Ye 276fbcff27 [Deprecation] Remove scheduled deprecated items (#60923) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-10 15:45:33 -04:00
Yufeng He 187a0eb98a [Bugfix][Frontend] Forward --enable-log-outputs to the Anthropic handler (#60982) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
 2026-10-10 16:36:03 +00:00
siyuClaude Opus 5Isotr0py
 		f2376cdc4d [Fast Start] Preload the FlashInfer autotune table on the weight cache daemon (#60085) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-10 15:48:06 +00:00
Wentao Ye 3709632ff2 [Refactor] Remove dead code multiple places (#60120) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-10 15:41:10 +00:00
Wentao Ye 838526c7e5 [Compile] Fix sampler compile warning (#60680) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-10 08:09:45 -07:00
Mook bf16c9c139 [Bugfix][Model] K-EXAONE: restore loading of pre-2.0 configs and apply RoPE to the MTP layer (#60187) 
Signed-off-by: godmook <cmoh4135@naver.com>
Signed-off-by: Mook <68294499+Godmook@users.noreply.github.com>
 2026-10-10 15:05:55 +00:00
Tony Lin 2bea22002d [BUG] Fix KV offload shared-memory slot selection using worker rank (#58497) 
Signed-off-by: Tony Lin <tony.lin@intel.com>
 2026-10-10 20:25:11 +08:00
Kay YanandCodex c41b2639e2 [Bugfix][Responses API] Complete cache-write token accounting (#60973) 
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-10 09:08:36 +00:00
SeongJun Leeandlesj0610 a98247ab4d [Model] Drop the 4x M-RoPE cache headroom for Qwen3-VL, Qwen3.5 and Qwen3.8 (#59853) 
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
Signed-off-by: lesj0610 <lesj0610@gmail.com>
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com>
 2026-10-10 06:51:52 +00:00
wang.yuqi c1692c85f1 [Misc] mv chat_utils.py to renderers. (#60522) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-10-10 06:50:34 +00:00
SeongJun LeeandIsotr0py 699c892dd5 [Perf] Skip the M-RoPE warmup when every holder of the RoPE fuses it (#60246) 
Signed-off-by: lesj0610 <lesj0610@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-10 06:47:39 +00:00