iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
16,743 Commits 3 Branches 0 Tags
afd7b1dce94fed484351fafd5bf5ea6601ac621e
Commit Graph
1550 Commits
Author SHA1 Message Date
Michael GoinandClaude 239b5ff30c [Frontend] Add --spec-method/--spec-model/--spec-tokens CLI aliases (#42476) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-18 17:22:27 -07:00
Jee Jee Ligemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Roger Wang
 		7d5b033782 [LoRA] Support 2D and 3D MoE LoRA adapter at the same time (#42242) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-18 15:22:26 +08:00
DustHunterQwen-Coderwang.yuqi
 		39c67d714e fix: add API key authorization to /v2 endpoints (#42594) 
Signed-off-by: DustHunter <dusthunter@126.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-16 01:29:27 +00:00
wang.yuqi 75fd68c7a5 [Entrypoints] Split the pooling offline API into PoolingOfflineMixin. (#42267) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-15 08:05:57 +00:00
Cyrus Leung 2676ab1e0b [Deprecation] Remove old locations of get_tokenizer and resolve_hf_chat_template (#35024) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-05-15 00:13:32 -07:00
Hongxin XuClaude Opus 4.6arlenxuJunjie Zhang
 		c7560af424 [RFC] Replace shared-memory routed experts with ModelRunnerOutput transfer and HTTP support (#39568) 
Signed-off-by: xhx1022 <1737006628@qq.com>
Signed-off-by: arlenxu <arlenxu@tencent.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: arlenxu <arlenxu@tencent.com>
Co-authored-by: Junjie Zhang <junj.jay.zhang@gmail.com>
 2026-05-14 14:12:30 +00:00
aoshen02 8c79ad6580 Revert "[Core] Replace routing replay with device cache and async D2H pipeline" (#39917) (#42434) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-05-13 23:49:01 -07:00
Krish Gupta 70c00163ff [Feature] Add instruction support for score/rerank chat templates (#42412) 
Signed-off-by: KrxGu <krishom70@gmail.com>
 2026-05-14 09:41:22 +08:00
Michael Goin 8efd508204 [Quantization] Rework quantization_config to use QuantKey and allow for activation override (#41566) 2026-05-13 16:58:32 -04:00
Walter Beller-Morales 873910d608 [Frontend] add support for thinking_token_budget in completions (#42116) 2026-05-13 16:01:52 -04:00
Ben Browning 0f69128a37 [Bugfix] Handle real-world gpt-oss tool call output in Harmony parsing (#42454) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
 2026-05-13 17:54:46 +00:00
halleriteandClaude Opus 4.7 97c4317bf5 [Bugfix][Frontend] Default max_tokens server-side on /inference/v1/generate (#42329) 
Signed-off-by: hallerite <git@hallerite.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 11:16:46 +02:00
Taneem Ibrahimgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Wentao Ye
 		503697c9ce [chore] Refactor pooling metadata token ID accessors (#42368) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-05-13 06:08:01 +00:00
yzong-rh 6ff7405b81 [Bugfix] [Frontend] Responses API, fix merging of messages (#42189) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Signed-off-by: Yifan <yzong@redhat.com>
 2026-05-12 16:09:59 +00:00
Wentao Ye a1b2d87498 [Refactor] Clean up pooling models build_tok_params logic (#42341) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-12 15:05:05 +00:00
chazenandChauncey 7f65f84428 [Bugfix] Fix empty channel/recipient in harmony for /v1/responses (#35540) 
Signed-off-by: kg6-sleipnir <christopherhazen42@gmail.com>
Signed-off-by: chazen <45186108+kg6-sleipnir@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-05-12 08:45:51 +00:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> d37e25ffbe [Frontend] Consolidate Speech to Text entrypoints. (#42370) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-12 07:06:57 +00:00
Chauncey 617239b70c [Frontend]Responses API supports chat_template_kwargs (#42272) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-05-11 11:59:39 +00:00
汪志鹏Wang, Zhipeng | RASIA <zhipeng.wang@rakuten.com>CursorChauncey
 		581b5e9afc [Frontend] Return rendered prompt text in chat completion response (#42052) 
Signed-off-by: Wang, Zhipeng | RASIA <zhipeng.wang@rakuten.com>
Co-authored-by: Wang, Zhipeng | RASIA <zhipeng.wang@rakuten.com>
Co-authored-by: Cursor <cursor@cursor.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-05-11 13:53:39 +08:00
abdulrahman-cohereCursor AgentNicolò Lucchesi
 		a5d0a5afba [Frontend][Bugfix] Abort ASR engine requests on cancellation (#41266) 
Signed-off-by: abdulrahman-cohere <abdulrahman.abdulrazzag@cohere.com>
Signed-off-by: <>
Co-authored-by: Cursor Agent <cursor-agent@cursor.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-05-09 23:51:11 -07:00
Dao007forever 3f5bd482f5 [Bugfix][KV Transfer][NIXL] Notify P node on pre-admission rejection to free stranded KV blocks (#41269) 2026-05-09 22:52:09 -07:00
Sumanth R Hegde e3b65a5ba0 [feat] Add explicit /start_weight_update and /finish_weight_update APIs for weight transfer (#39212) 2026-05-08 18:03:33 -07:00
Shrinav LokaClaude Opus 4.6wang.yuqimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		cd58e30872 [Perf] Use numpy zero-copy path for embedding float response serialization (#41681) 
Signed-off-by: Shrinav Loka <lokashrinav@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-07 19:42:21 -07:00
TomerBN-Nvidia 8189a15914 [Core] Replace routing replay with device cache and async D2H pipeline (#39917) 
Signed-off-by: Tomer Barnatan <tbarnatan@nvidia.com>
 2026-05-07 11:24:56 -07:00
Flora Feng 8eb401134e [Refactor] Consolidate required/named tool_choice streaming into DelegatingParser (#41876) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-05-07 09:50:59 -07:00
Viktor Pus d5b31c954d [Bugfix] Account for truncate_prompt_tokens when computing max_tokens (#41800) 
Signed-off-by: Viktor Pus <viktorpus@tenstorrent.com>
 2026-05-06 16:10:17 +00:00
jackQwertyJackchaunceyjiang
 		6467213a9f fix(openai): tolerate empty content in forced tool choice (#40148) 
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-05-06 07:16:03 -07:00
Chauncey c7aa186d67 [Frontend] Supports resubmitting output items with missing fields in Responses API (#41355) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-05-05 22:21:33 -04:00
Linzhang LiMichael GoinUbospicasfeng33
 		844df54269 feat: update xgrammar==0.2.0 to use structural tags for strict tool calling + reasoning for more models (#40894) 
Signed-off-by: Yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Ubospica <ubospica@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Ubospica <ubospica@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-05-04 12:45:24 -07:00
Bugen Zhaoandwang.yuqi a07642667d [Bugfix] Pass reasoning parser kwargs to structured output (#41199) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-30 23:38:02 -07:00
Luis 🚀andAndrew Sansom 14043dfecd feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API (#40720) 
Signed-off-by: Luis Robaina <luis@protopia.ai>
Signed-off-by: Luis Robaina 🚀 <luisfabian1545@gmail.com>
Signed-off-by: LuisRobaina <luis@protopia.ai>
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com>
 2026-05-01 10:05:55 +08:00
Dhruv SingalClaude Opus 4.6Dhruv SingalYour NameOpenCodeSimon Mo
 		cb1b02d0e8 [Frontend] Add VLLM_SKIP_MODEL_NAME_VALIDATION environment variable (#34676) 
Signed-off-by: Dhruv Singal <dhruvsingalabc@gmail.com>
Signed-off-by: Dhruv Singal <dsingal@Dhruvs-MacBook-Pro.local>
Signed-off-by: Your Name <you@example.com>
Signed-off-by: vLLM Assistant <assistant@vllm.ai>
Signed-off-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Dhruv Singal <dsingal@Dhruvs-MacBook-Pro.local>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: OpenCode <noreply@openai.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2026-04-29 23:19:09 -07:00
Jared Wensfeng33Chauncey
 		11b69129e2 [Frontend] Add defer_loading and tool_reference support for Anthropic and OpenAI APIs (#40190) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-04-29 04:35:50 -07:00
Bugen Zhao 33f36d4260 [DSV4] Support max reasoning effort (#40982) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-04-29 11:03:47 +00:00
Chauncey 3885d340a4 [Frontend]Responses API supports Tool/Function calling with streaming with named tool/function (#41110) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-29 09:11:27 +00:00
haosdent 4b95e9cec4 [CI] Return HTTP 400 for unsupported chat content part type (#41121) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-04-29 10:23:26 +08:00
wang.yuqi a8208e6a81 [Examples] Resettle features examples. (#40995) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-28 00:33:41 -07:00
Chauncey 146f44b77d [Frontend]Responses API supports Tool/Function calling with streaming with required (#40700) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-27 19:36:58 -07:00
Simon MoandClaude ebf862c351 Add system_fingerprint field to OpenAI-compatible API responses (#40537) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-27 16:17:52 +08:00
+7
Yifan QiaoYongye ZhuYongye ZhuSimon MoBugen ZhaoGiancarlo DelfinJee Jee LiNick HillRoger WangRoy WangWoosuk KwonyoukaichaoZhewen LiZijing LiukhluuqizixiZhewen Li
 		4d51588e23 [Feat] DeepSeek V4 Rebased (#40860) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
 2026-04-26 18:31:08 -07:00
Chenguang ZhengandOpenAI Codex 07351e0883 [Feature] Warm up readonly multimodal processor during renderer startup (#40797) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-04-25 03:57:41 +00:00
Alex Brooks 5e11b40365 [Frontend] Delegate to vLLM Omni When --omni Passed (#40744) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-04-24 12:30:00 -04:00
wang.yuqigemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
 		9744b699ba [Deprecate] Deprecate LLM.reward offline api, use LLM.encode instead. (#40688) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-04-24 05:37:50 +00:00
Neil Schemenauer 56bdf85e10 [Feature] Avoid eager import of the "mistral_common" package. (#40043) 
Signed-off-by: Neil Schemenauer <nas@arctrix.com>
 2026-04-24 02:49:16 +00:00
Flora Fengandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 8317cedc77 [Responses] Add tool_choice/tools validation to match OpenAI behavior (#40399) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-22 22:46:10 -07:00
Honglin Cao 9c271f9403 [gRPC] Add standard gRPC health checking (grpc.health.v1) for Kubernetes native probes (#38016) 
Signed-off-by: Honglin Cao <Caohonglin317@hotmail.com>
 2026-04-22 21:31:00 +00:00
AllenDouandzixiao 9047288b68 support hotwords for FunASR model (#39674) 
Signed-off-by: zixiao <shunli.dsl@alibaba-inc.com>
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
 2026-04-22 02:25:06 -07:00
Bugen Zhao 0210024ae7 [Bugfix] Pass effective chat template kwargs to reasoning parsers (#40460) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-04-21 22:17:51 -07:00
Ekagra Ranjangemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
 		4eafc72928 [Audio] Bundle get_generation_prompt() params into SpeechToTextParams (#36268) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-04-22 12:24:18 +08:00
Matthew BonanniandTyler Michael Smith 96a85c5750 [Startup][UX] Enable CUDAGraph memory profiling by default (#38284) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-04-21 18:16:59 -04:00