iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,242 Commits 3 Branches 0 Tags
be95edb7f6820ff6b341e6e48ba7bfa15d628209
Commit Graph
454 Commits
Author SHA1 Message Date
MOVZX be95edb7f6 Merge branch 'main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-16 17:35:42 +07:00
Stefano CastagnettaandKevin H. Luu 784d1fff0c [MRV2][Spec Decode] Quantized draft lm_head for drafters that share the target head (#59973) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-15 22:06:40 +00:00
Shern Shiou TanClaude Opus 5.5Simon VeitnerRaphael RiallandLucas Wilkinson
 		d92a4f25f2 [watermarking] Add DFlash support to dual-key gumbel-max watermarking (#59554) 
Signed-off-by: Shern Shiou Tan <shernshiou@gmail.com>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-15 22:06:06 +00:00
MOVZX 76df77c04e Merge branch 'main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-15 21:16:39 +07:00
Misha GoinandClaude Opus 5.5 1c4c319792 [Frontend] Advertise --enable-x/--disable-x for boolean CLI flags (#61479) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 01:08:02 -07:00
Bugen ZhaoandClaude Opus 5.5 8fa5ae844a [Frontend] Let an unset --tool-strict-level fall back to the tool parser's default (#61328) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 15:48:38 +08:00
liranschourandKevin H. Luu c5f8adc0b1 [KV Offload] Expose OffloadingManager.lock()/unlock() and service the tiering control plane off the scheduler thread (#58168) 
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-15 08:50:56 +03:00
Woosuk KwonandClaude Opus 5.5 8530aed2b7 [Spec Decode] Remove Medusa and MLP speculator support (#61701) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 17:29:23 -07:00
Juan Pérez de AlgabaandOpenAI Codex 5d88c034f6 [Security] Reject local verifier references in Speculators metadata (#61587) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-14 15:30:20 +00:00
Amir FriedmanClaude Opus 5Or Ozeri
 		775fefa804 [Metrics][KV Offload] Add vllm:kv_offload_config_info - generic info metrics for offloading the offloading connector (#56867) 
Signed-off-by: Amir Friedman <Amir.Friedman1@ibm.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-10-14 12:34:37 +00:00
fxmarty-amdvllmellmTan Pin Siang
 		c9bfa574ce [Online quantization] Extend re-quantization API to MOE (showcase NVFP4 modelopt/CT/quark to any online MOE method) (#60276) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com>
 2026-10-14 15:49:37 +08:00
aoshen02Claude Fable 5.1Dakai AnDakai An
 		3ccc0a90b6 [KV Connector] Support sleep mode with MooncakeConnector over RDMA (#59625) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Dakai An <77474977+andakai@users.noreply.github.com>
 2026-10-14 04:18:47 +00:00
Andrii SkliarAndrii SkliarLuke Alonso
 		6849ca696a [Quantization] Add ModelOpt IQ2_XS, IQ2_XXS, and Q8_0 support via b12x (#59604) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Luke Alonso <lalonso@gmail.com>
 2026-10-13 09:06:33 -07:00
Jiangyun ZhuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		2cc2e68d1c [Core][Mamba] Remove leftovers of mamba_cache_mode="all" (#60043) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-13 00:58:59 -07:00
simon-veitner-redhatClaudeLucas Wilkinson
 		a7b7bc1fbe [Metrics] Count finished requests by watermark status (#58715) 
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-12 17:21:22 +00:00
MattClaude Opus 5.5Andreas Karatzas
 		e0317b28ce [Feature] Support batch invariance on ROCm (#52231) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-12 10:53:26 -05:00
Tianyu Guo 258b97ff6b [Bugfix][EC Connector] Size transfers by measured encoder output widths (#57904) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-10-12 12:32:39 +00:00
aoshen02andCodex 239a3fa3a8 [Sleep] Retain draft-private weights with existing parameter selector (#61046) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-11 23:30:56 +08:00
siyuClaude Opus 5Isotr0py
 		f2376cdc4d [Fast Start] Preload the FlashInfer autotune table on the weight cache daemon (#60085) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-10 15:48:06 +00:00
Giancarlo Delfin ffdde24f54 [Docs] Document adaptive verification without a confidence head (#60913) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-10-09 13:54:40 -07:00
MOVZX ac305c44cb Merge branch 'main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-09 17:41:13 +00:00
yuvalluriaClaude Opus 5Harry Mellor
 		5cadabfafd [Docs] Add Qwen2.5-Coder-7B-Instruct to batch invariance tested models (#46368) 
Signed-off-by: Yuval Luria <yluria@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-10-09 10:41:29 +00:00
Chao Lei bc713cf8cc [Bugfix]MooncakeConnect add query timeout and retry (#55923) 2026-10-09 09:19:33 +00:00
aoshen02Claude Fable 5.1indianspeedster
 		a6f43e57d6 [ROCm] Enable the cuMem CUDA-graph pool offload on ROCm (#59523) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: indianspeedster <cspandey016@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: indianspeedster <cspandey016@gmail.com>
 2026-10-09 08:29:55 +00:00
Rajeev Warrier 81d9c45e4b [Docs] [Multimodal] Note that per-request media_io_kwargs needs --trust-request-mm… (#60598) 
Signed-off-by: Rajeev Warrier <warrier.rajeev.95@gmail.com>
 2026-10-09 04:23:07 +00:00
Brian Weston, PhDCodexJee Jee Lilinitra24
 		201e537d98 [Bugfix][LoRA] Validate incompatible MoE adapter formats (#59924) 
Signed-off-by: Brian Weston <brianweston@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io>
 2026-10-09 09:49:13 +08:00
Woosuk Kwon 194bd72be7 [Compile] Remove torch.compile-based sequence parallelism and AsyncTP (#60517) 2026-10-08 08:28:23 -07:00
YueshenZandsfeng33 ed825aafc0 [Bugfix][Frontend] Fix Responses namespace aliases in named tool calling (#59723) 
Signed-off-by: Shen <312058264+YueshenZ@users.noreply.github.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-10-08 00:07:52 +00:00
fxmarty-amdvllmellmTan Pin SiangMisha Goin
 		f8d92cae41 [Quantization] Layer re-quantization for linear layers through online quantization API (MXFP8 -> FP8 PTPC showcase) (#55684) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-06 11:38:18 -05:00
Raphaël RiallandandSimon Veitner c32495d58f [watermark] compatibility validation (#56801) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
 2026-10-05 14:37:13 -04:00
MOVZX f83ecfac16 Merge remote-tracking branch 'upstream/main' into patches 2026-10-05 08:52:31 +07:00
Shenglei FuClaude Opus 5.5Wentao Ye
 		1388100560 [Kernel][LoRA] Add deterministic split-K=8 LoRA shrink for batch invariance (#59377) 
Signed-off-by: Shenglei Fu <117230642+ShengleiFu@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-04 18:46:16 -04:00
Lio EinaudiYifan DaCodeBuddyClaude Fable 5.1Wentao Ye
 		89439db727 [CI][Docs] Fix decode/prefill consistency test prefix construction; add GLM-4-9B and Phi-4 to batch-invariant tested models (#53692) 
Signed-off-by: Yifan Da <1363818765@qq.com>
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Yifan Da <1363818765@qq.com>
Co-authored-by: CodeBuddy <noreply@codebuddy.dev>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-04 11:55:18 -04:00
aoshen02andClaude Opus 5.5 5e56e9af2a [Feature] Release the CUDA graph pool on sleep (#59160) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 01:13:10 +00:00
MOVZX 85c31d47ad Merge remote-tracking branch 'upstream/main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-03 22:06:10 +07:00
MOVZX 2877121b2d Merge remote-tracking branch 'upstream/main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-03 21:59:03 +07:00
Martin Hickey bc21cba967 [Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) (#58588) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-03 19:38:25 +08:00
Nils Matteson 9fdb174678 [Bugfix] Avoid InfiniBand state in TP1 snapshots (#59699) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
 2026-10-03 03:59:17 +00:00
yifanFenggClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Wentao Ye
 		6e517b15c1 [Docs] Add ERNIE 4.5 to batch invariance tested models (#58476) 
Signed-off-by: yifanFengg <evanfengff@gmail.com>
Signed-off-by: yifanFengg <yifanfen@amazon.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-02 20:37:32 +00:00
Rajeev Warrierandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 30e956fad3 [Docs] Fix stale W4A16 NVFP4 default kernel in ModelOpt docs (#59135) 
Signed-off-by: Rajeev Warrier <warrier.rajeev.95@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-02 13:21:43 +00:00
Teng MaandTeng Ma 1a9eaa3a68 [Doc] Document ECMooncakeConnector for EPD disaggregation (#59196) 
Signed-off-by: Teng Ma <stmatengss@gmail.com>
Co-authored-by: Teng Ma <stmatengss@users.noreply.github.com>
 2026-10-02 12:10:33 +08:00
Yoni GozlanCursorBugen ZhaoClaude Opus 5.5
 		9bd993564f [Frontend] Parse tool calls and reasoning from checkpoint response templates (#58604) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 02:33:00 +00:00
Flora Feng 785b990116 [Distributed] Enable custom all-reduce under VLLM_BATCH_INVARIANT (#58623) 
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-10-02 02:25:48 +00:00
Flora Feng a5105dba05 [Frontend] Port Step-3.5 parsers to the streaming parser engine (#59321) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-10-01 22:18:37 +00:00
Zijing Liu ea84051819 [KVConnector][NIXL] Count completion notifications that arrive after KV expiry (#58875) 
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
 2026-10-01 21:30:58 +00:00
Sundri Lai bcee730b1a [Docs] Remove references to removed env vars (#59530) 
Signed-off-by: Sundri Lai <laiyanting.neu@gmail.com>
 2026-10-01 08:26:14 +00:00
Nils Matteson 51048b13ce [Docs] Clarify snapshot runtime image support (#59127) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
 2026-09-30 20:31:21 +00:00
Raphaël RiallandSimon VeitnerLucas Wilkinson
 		0f8b398158 [watermarking] add context deduplication support to speculative decoding (#56807) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-09-30 12:22:10 -04:00
yuvalluriaandClaude Sonnet 4.6 863475eb9c docs: add gemma-2, SmolLM2, Qwen2.5-Coder, Llama-3.2-1B to batch invariance tested models (#54441) 
Signed-off-by: Yuval Luria <yluria@redhat.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-09-30 11:13:23 +00:00
Isotr0py 31876d8ea1 [Bugfix] Fix TorchCodec audio IO correctness (#58364) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-29 11:34:48 -05:00