iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,878 Commits 3 Branches 0 Tags
258b97ff6bc1dbbc7b76d2d5a5a87591f9de107f
Commit Graph
22878 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Tianyu Guo 258b97ff6b [Bugfix][EC Connector] Size transfers by measured encoder output widths (#57904) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-10-12 12:32:39 +00:00
Liana KolevaandBugen Zhao aa1d44df16 [Bugfix][Rust Frontend] Alias qwen3_xml to qwen3_coder and add qwen3_json for the Qwen JSON format (#60136) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-12 20:20:53 +08:00
Shimi BandielHyeonki HongMimirClaudeBugen Zhao
 		8d7b93fec2 [Frontend] Report stop_reason on /inference/v1/generate and pass it through derender (#60251) 
Signed-off-by: Hyeonki Hong <hhk7734@gmail.com>
Signed-off-by: Shimi Bandiel <shimib@google.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Hyeonki Hong <hhk7734@gmail.com>
Co-authored-by: Mimir <mimir@nousresearch.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-12 19:37:53 +08:00
ReidBugen ZhaoClaude Opus 5.5
 		32b05addc1 [Bugfix][Rust Frontend] Restore DP wave delivery after XPUB reconnects (#60249) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 19:12:43 +08:00
ReidandBugen Zhao f5cda48c90 [Bugfix][Rust Frontend] Export corrupted request metric (#59937) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-12 19:12:39 +08:00
0xsenseiBugen ZhaoClaude Opus 5.5
 		3b9e462774 [Bugfix][Rust Frontend] Correct DP routing after request removal (#59631) 
Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 19:12:35 +08:00
Zupeng WangOpenAI CodexBugen ZhaoClaude Opus 5.5
 		21445806bd [Rust Frontend] Support logprob_token_ids on OpenAI endpoints (#59549) 
Signed-off-by: Wang Zupeng <zupenwang@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 19:12:30 +08:00
Thang NguyenandClaude Opus 5.5 ee8f7ea449 [CI] Enroll a fourth batch of five steps in automatic sharding (#60858) 
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 03:46:07 -07:00
MatthewCYM f7a3a7265f [Bugfix][Multimodal] Fix Qwen3-ASR audio preprocessing parity (#57146) 
Signed-off-by: MatthewCYM <chenyiming0v0@gmail.com>
 2026-10-12 10:42:51 +00:00
Jimmy WangandJuan Pérez de Algaba 2534a96fcf [Docs] List additional unauthenticated endpoints (#61081) 
Signed-off-by: Jimmy Wang <53066820+JimmyWang0417@users.noreply.github.com>
Co-authored-by: Juan Pérez de Algaba <124347725+jperezdealgaba@users.noreply.github.com>
 2026-10-12 11:47:52 +02:00
Bugen ZhaoandClaude Opus 5.5 5a09062632 [Rust Frontend] Compose answer constraints into parser-owned output grammars (#60781) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 09:43:48 +00:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0466bd36ea [Rust Frontend] Build the Kimi K3 full-output grammar from its parser mode (#59190) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-12 09:43:47 +00:00
Shijin ZhangandChauncey 83773ecdf3 [Frontend] Support CustomToolCall in the non-Harmony Responses API (#57322) 
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-10-12 17:22:55 +08:00
zhangchengzhucufe-devandKunshang Ji a33d46b471 [Bugfix][XPU] Forward softcap and alibi_slopes in xpu_ops.flash_attn_varlen_func (#58936) 
Signed-off-by: zhangchengzhucufe-dev <zhangchengzhucufe@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-12 17:10:55 +08:00
Yan Ma e05201aa2e [XPU] refine memory release for sleep mode (#61158) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-10-12 16:59:38 +08:00
Thuongvu HoandNicolò Lucchesi 99b44c7a42 [Metrics] Add admission rejection metrics (#55812) 
Signed-off-by: Thuongvu Ho <thuongvuho@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
 2026-10-12 08:31:07 +00:00
Cyrus Leung 5bed289707 [DX] Update CODEOWNERS for entrypoints (#60984) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-10-12 08:06:47 +00:00
Shuolei Wangaoshen02Bugen ZhaoClaude Opus 5.5
 		4163a80142 [Rust Frontend] Return weight-version spans on final outputs (#61070) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 15:56:59 +08:00
Ting SUN f4114e068e [Bugfix][Frontend] Preserve token IDs in chat derender responses (#61065) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-10-12 07:55:06 +00:00
AMD-yanfeiwangandaoshen02 e3bbaf2369 [ROCm][DSv4] Fuse the shared expert into AITER MegaMoE (#61004) 
Signed-off-by: Yanfei Wang <yanfei.wang@amd.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-12 15:53:16 +08:00
Giancarlo DelfinandRishi Puri 36f3972812 [Spec Decode] Skip metadata build for dflash/dspark when all model backends support it (#60692) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Rishi Puri <20074092+puririshi98@users.noreply.github.com>
 2026-10-12 00:09:10 -07:00
Zhenzhong XuandKunshang Ji 6a70913590 [XPU] bump up auto-round-lib to 0.16.0 (#60976) 
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-12 14:47:59 +08:00
HubertCodexAo Shen
 		74f4404f58 [Bugfix][Distributed] Own SHM out-of-band buffers before releasing read slots (#61000) 
Signed-off-by: Hubert Marek <163992334+hubert-marek@users.noreply.github.com>
Signed-off-by: Ao Shen <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Ao Shen <aoshen@inferact.ai>
 2026-10-12 01:44:12 -04:00
Canlin GuoandClaude Opus 5.5 fe4df3df0e [Bugfix][GLM-5.3-Flash] Restore MTP index sharing across draft iterations (#61079) 
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 05:41:52 +00:00
shaohuaxiCodexCyrus Leung
 		d4e92ccc37 [Bugfix][Frontend] Honor endpoint plugin route overrides (#58839) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-12 03:58:15 +00:00
liuzhenweiandKunshang Ji bfa7e5b7af [XPU][CI] Enable XPU graph in graph tests (#60784) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-12 10:39:40 +08:00
Yan Ma f147949be8 [XPU] limit FA kernel block size to multipleOf 64 (#57297) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-10-12 10:25:32 +08:00
dingdangmaoandCodex 124b89d5b7 [Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries (#59962) 
Signed-off-by: apex-mochen <2756823972@qq.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-12 02:08:14 +00:00
Michał GanczarenkoandClaude Opus 5.5 03091e28b8 [Bugfix] Read Ling quantization_config from top-level config for Bailing V3 VL (#59026) 
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 09:25:29 +08:00
Laith Sakka cc9b0f097d [Fix] MiniMax: fix CUDA IPC handle exchange on cuda-bindings >= 13.4 (#60917) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2026-10-11 20:38:56 -04:00
Nick Hill 3fe917abb1 [Bugfix][MRV2] Fix FlashInfer attention and ReplaySSM warmups on MRV2 (#61013) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-11 16:04:41 -07:00
Jared WenClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f26d9f432c [Bugfix][GLM-5.3-Flash] Honor --mamba-ssm-cache-dtype for the KDA recurrent state (#59147) 
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-11 17:55:00 +00:00
Nick Hill c059bee812 [MRV2] Switch CI tests still running on MRV1 unnecessarily (#61014) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-11 09:57:07 -07:00
Ting SUNandCyrus Leung b99fdbe1f8 [Bugfix] Respect the configured offline chat template (#61061) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-11 16:27:21 +00:00
Juan Pérez de AlgabaRobert ShawItay Etelis
 		93d2cab35e [Security] Fix P2P ZMQ socket exhaustion via unbounded peer sessions (#51504) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
 2026-10-11 11:39:11 -04:00
aoshen02andCodex 239a3fa3a8 [Sleep] Retain draft-private weights with existing parameter selector (#61046) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-11 23:30:56 +08:00
Zheng CairootOpenAI CodexzigzagcaiClaude CodeWentao Ye
 		3e8e1364ff [Perf][GLM-5.3-Flash] Shard long-context indexer prefill rows across TP (#54951) 
Signed-off-by: Zheng Cai <8370601+zigzagcai@users.noreply.github.com>
Signed-off-by: Zheng Cai <caizheng1993@gmail.com>
Signed-off-by: Codex <noreply@openai.com>
Co-authored-by: root <root@rjob-fb29567b88fec0a9-88c9a4c797eb2ec1-0.caizheng.ailab-sys.svc.pjlab.local>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-11 10:43:37 -04:00
aoshen02 bd5bbce6e4 [RL] Release weight-transfer communicators without stopping inference (#61060) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
 2026-10-11 21:15:14 +08:00
tc-mbandCyrus Leung c85616efc6 [Model] Support MiniCPM-V 4.7 (#58674) 
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-11 10:21:10 +00:00
Subhash Polisetti f425a21208 [Bugfix][Frontend] Support scalar priority and wire priority into LLM.chat() (#57243) 
Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
 2026-10-11 10:03:33 +00:00
Stu CaoClaude Fable 5.1Yifan QiaoCodex
 		2339bd39d6 [Bugfix][Mooncake] Use the EAGLE replay boundary when retaining sliding-window tails (#61012) 
Signed-off-by: Stu Cao <stu@primeintellect.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-11 09:15:18 +00:00
aoshen02andOpenAI Codex 1789f09588 [Frontend] Explain disabled scale-out endpoints with opt-in hints (#61042) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-11 08:58:07 +00:00
Juan Pérez de AlgabaandCyrus Leung fcae5c9887 [Security] Restrict default CORS origins (#61009) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-11 08:26:02 +00:00
Xun Sun 26a488ad93 [Fast Start][Bugfix] Snapshot GPU memory after external weights are ready (#60966) 2026-10-11 07:10:04 +00:00
debanshdIsotr0pyKimi Code
 		ef42b4385f [Bugfix] Reclaim cached GPU memory during weight loading (#57516) 
Signed-off-by: Debanshu Das <debanshu@google.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-10-11 07:03:06 +00:00
Bruno VolpatoandCodex 61f084b5f3 [Bugfix][Multimodal] Fix per-video cache option reuse (#55203) 
Signed-off-by: bvolpato <brunocvcunha@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-11 06:21:55 +00:00
Yongye ZhuandClaude Opus 5.5 2eb491e18b [Perf][MiniMax-M3] MSA prefill attends with the E4M3 query; bump fmha_sm100 to 968061a (#60684) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 13:40:45 +08:00
Nick Hill f8471164bf [MRV2] Detatch weight-reload logic from MRV1 (#60937) 2026-10-10 21:15:05 -07:00
bzsuni 01abe975b5 [Bugfix][KV Connector] Fix speculative block reservations across KV cache groups (#59880) 
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
 2026-10-10 20:55:41 -07:00
Shuolei Wangandaoshen02 b89852aad0 [RL][Frontend] Return token-level weight-version spans on final outputs (#59582) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-11 11:33:29 +08:00