iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,893 Commits 3 Branches 0 Tags
a686c8e1b7e5870eadfc9f47447de2b8a26211fc
Commit Graph
22893 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Riasanovsky a686c8e1b7 [Triton] Migrate Triton kernels off tl.make_block_ptr for triton-main compat (#60502) 
Signed-off-by: Nick Riasanovsky <njriasan@meta.com>
 2026-10-12 11:38:43 -04:00
Shang WangandClaude Opus 5.5 ffec221276 [Bugfix][DSv4.1] Match decoder-replay input_ids dtype for FlashInfer MegaMoE (#61041) 
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 14:23:00 +00:00
Hoai VoandClaude Opus 5.5 f3cb10b690 [Model][LoRA] Enable LoRA support for BERT and RoBERTa embedding models (#61033) 
Signed-off-by: hoaivoke <vokehoai@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 14:09:53 +00:00
Shang WangandClaude Opus 5.5 716961cf1f [Bugfix] Drop enable_in_kernel_fc2_reduce from FlashInfer CuteDSL MegaMoE config (#59900) 
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 09:05:09 -05:00
Ilya Markov a281ad32f0 [EPLB] Add EPLB support to dspark draft models (DeepSeekV4) (#57549) 
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
 2026-10-12 10:03:09 -04:00
Ilya Markov b3ee560ae8 [Bugfix][MoE] Apply expert_map remapping for CUTLASS FP4 MOE expert parallel (#59042) 
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
 2026-10-12 10:02:56 -04:00
bzsuni f022975964 [Bugfix] Reject beam search in completion render (#61212) 
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
 2026-10-12 13:49:46 +00:00
Suraj SharanVadim Gimpelsonlinitra24
 		4c36d21280 [Bugfix][LoRA] Apply PEFT rank_pattern / alpha_pattern scaling per module (#59801) 
Signed-off-by: suraj <surajsharan85@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io>
 2026-10-12 21:49:30 +08:00
Harry MellorClaude Opus 5.5Kevin H. Luu
 		3facf822b7 [Performance] Load HF tokenizers and processors from a local snapshot (#60825) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-12 13:45:54 +00:00
Jiangyun ZhuClaude Opus 5.5Isotr0py
 		092c35dd15 [Model] Support Cloudflare/clef-omni on /v1/systemone (#61064) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-12 21:27:34 +08:00
DimensionSTPandArtem Perevedentsev 314ab8cffd [Bugfix][Structured Output] Drain compilation tasks before releasing the backend (#61047) 
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-10-12 16:25:36 +03:00
Chauncey ac28877f0e [Bugfix] Fix out-of-bounds lookahead in adaptive block verification (#60945) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-10-12 09:21:35 -04:00
stefankoncarevic 8c175f403a [CI][ROCm] Compare greedy output only in the AITER attn+quant test (#61185) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-10-12 08:20:35 -05:00
Rohan PotdarandClaude Opus 5.5 7900d2eabc [Test] Accept tied tokens at the k-th value in the Triton top-k test (#60726) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 08:18:44 -05:00
Harry Huangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> c75cb63cd2 [Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next (#56322) 
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-12 20:56:44 +08:00
Tianyu Guo 258b97ff6b [Bugfix][EC Connector] Size transfers by measured encoder output widths (#57904) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-10-12 12:32:39 +00:00
Liana KolevaandBugen Zhao aa1d44df16 [Bugfix][Rust Frontend] Alias qwen3_xml to qwen3_coder and add qwen3_json for the Qwen JSON format (#60136) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-12 20:20:53 +08:00
Shimi BandielHyeonki HongMimirClaudeBugen Zhao
 		8d7b93fec2 [Frontend] Report stop_reason on /inference/v1/generate and pass it through derender (#60251) 
Signed-off-by: Hyeonki Hong <hhk7734@gmail.com>
Signed-off-by: Shimi Bandiel <shimib@google.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Hyeonki Hong <hhk7734@gmail.com>
Co-authored-by: Mimir <mimir@nousresearch.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-12 19:37:53 +08:00
ReidBugen ZhaoClaude Opus 5.5
 		32b05addc1 [Bugfix][Rust Frontend] Restore DP wave delivery after XPUB reconnects (#60249) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 19:12:43 +08:00
ReidandBugen Zhao f5cda48c90 [Bugfix][Rust Frontend] Export corrupted request metric (#59937) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-12 19:12:39 +08:00
0xsenseiBugen ZhaoClaude Opus 5.5
 		3b9e462774 [Bugfix][Rust Frontend] Correct DP routing after request removal (#59631) 
Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 19:12:35 +08:00
Zupeng WangOpenAI CodexBugen ZhaoClaude Opus 5.5
 		21445806bd [Rust Frontend] Support logprob_token_ids on OpenAI endpoints (#59549) 
Signed-off-by: Wang Zupeng <zupenwang@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 19:12:30 +08:00
Thang NguyenandClaude Opus 5.5 ee8f7ea449 [CI] Enroll a fourth batch of five steps in automatic sharding (#60858) 
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 03:46:07 -07:00
MatthewCYM f7a3a7265f [Bugfix][Multimodal] Fix Qwen3-ASR audio preprocessing parity (#57146) 
Signed-off-by: MatthewCYM <chenyiming0v0@gmail.com>
 2026-10-12 10:42:51 +00:00
Jimmy WangandJuan Pérez de Algaba 2534a96fcf [Docs] List additional unauthenticated endpoints (#61081) 
Signed-off-by: Jimmy Wang <53066820+JimmyWang0417@users.noreply.github.com>
Co-authored-by: Juan Pérez de Algaba <124347725+jperezdealgaba@users.noreply.github.com>
 2026-10-12 11:47:52 +02:00
Bugen ZhaoandClaude Opus 5.5 5a09062632 [Rust Frontend] Compose answer constraints into parser-owned output grammars (#60781) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 09:43:48 +00:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0466bd36ea [Rust Frontend] Build the Kimi K3 full-output grammar from its parser mode (#59190) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-12 09:43:47 +00:00
Shijin ZhangandChauncey 83773ecdf3 [Frontend] Support CustomToolCall in the non-Harmony Responses API (#57322) 
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-10-12 17:22:55 +08:00
zhangchengzhucufe-devandKunshang Ji a33d46b471 [Bugfix][XPU] Forward softcap and alibi_slopes in xpu_ops.flash_attn_varlen_func (#58936) 
Signed-off-by: zhangchengzhucufe-dev <zhangchengzhucufe@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-12 17:10:55 +08:00
Yan Ma e05201aa2e [XPU] refine memory release for sleep mode (#61158) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-10-12 16:59:38 +08:00
Thuongvu HoandNicolò Lucchesi 99b44c7a42 [Metrics] Add admission rejection metrics (#55812) 
Signed-off-by: Thuongvu Ho <thuongvuho@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
 2026-10-12 08:31:07 +00:00
Cyrus Leung 5bed289707 [DX] Update CODEOWNERS for entrypoints (#60984) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-10-12 08:06:47 +00:00
Shuolei Wangaoshen02Bugen ZhaoClaude Opus 5.5
 		4163a80142 [Rust Frontend] Return weight-version spans on final outputs (#61070) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 15:56:59 +08:00
Ting SUN f4114e068e [Bugfix][Frontend] Preserve token IDs in chat derender responses (#61065) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-10-12 07:55:06 +00:00
AMD-yanfeiwangandaoshen02 e3bbaf2369 [ROCm][DSv4] Fuse the shared expert into AITER MegaMoE (#61004) 
Signed-off-by: Yanfei Wang <yanfei.wang@amd.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-12 15:53:16 +08:00
Giancarlo DelfinandRishi Puri 36f3972812 [Spec Decode] Skip metadata build for dflash/dspark when all model backends support it (#60692) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Rishi Puri <20074092+puririshi98@users.noreply.github.com>
 2026-10-12 00:09:10 -07:00
Zhenzhong XuandKunshang Ji 6a70913590 [XPU] bump up auto-round-lib to 0.16.0 (#60976) 
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-12 14:47:59 +08:00
HubertCodexAo Shen
 		74f4404f58 [Bugfix][Distributed] Own SHM out-of-band buffers before releasing read slots (#61000) 
Signed-off-by: Hubert Marek <163992334+hubert-marek@users.noreply.github.com>
Signed-off-by: Ao Shen <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Ao Shen <aoshen@inferact.ai>
 2026-10-12 01:44:12 -04:00
Canlin GuoandClaude Opus 5.5 fe4df3df0e [Bugfix][GLM-5.3-Flash] Restore MTP index sharing across draft iterations (#61079) 
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 05:41:52 +00:00
shaohuaxiCodexCyrus Leung
 		d4e92ccc37 [Bugfix][Frontend] Honor endpoint plugin route overrides (#58839) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-12 03:58:15 +00:00
liuzhenweiandKunshang Ji bfa7e5b7af [XPU][CI] Enable XPU graph in graph tests (#60784) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-12 10:39:40 +08:00
Yan Ma f147949be8 [XPU] limit FA kernel block size to multipleOf 64 (#57297) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-10-12 10:25:32 +08:00
dingdangmaoandCodex 124b89d5b7 [Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries (#59962) 
Signed-off-by: apex-mochen <2756823972@qq.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-12 02:08:14 +00:00
Michał GanczarenkoandClaude Opus 5.5 03091e28b8 [Bugfix] Read Ling quantization_config from top-level config for Bailing V3 VL (#59026) 
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 09:25:29 +08:00
Laith Sakka cc9b0f097d [Fix] MiniMax: fix CUDA IPC handle exchange on cuda-bindings >= 13.4 (#60917) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2026-10-11 20:38:56 -04:00
Nick Hill 3fe917abb1 [Bugfix][MRV2] Fix FlashInfer attention and ReplaySSM warmups on MRV2 (#61013) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-11 16:04:41 -07:00
Jared WenClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f26d9f432c [Bugfix][GLM-5.3-Flash] Honor --mamba-ssm-cache-dtype for the KDA recurrent state (#59147) 
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-11 17:55:00 +00:00
Nick Hill c059bee812 [MRV2] Switch CI tests still running on MRV1 unnecessarily (#61014) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-11 09:57:07 -07:00
Ting SUNandCyrus Leung b99fdbe1f8 [Bugfix] Respect the configured offline chat template (#61061) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-11 16:27:21 +00:00
Juan Pérez de AlgabaRobert ShawItay Etelis
 		93d2cab35e [Security] Fix P2P ZMQ socket exhaustion via unbounded peer sessions (#51504) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
 2026-10-11 11:39:11 -04:00