iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,910 Commits 3 Branches 0 Tags
08582942d8af7fc58339f3a5fedaffbbc8ea9f02
Commit Graph
9041 Commits
Author SHA1 Message Date
Martin Hickey 08582942d8 [Bugfix][Frontend] Honor request_id and X-Request-Id in chat and completions render (#61236) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-12 19:17:12 +00:00
Wentao Ye d0948f754d [Bug] Honor spec decode draft model config (#61240) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-12 15:13:23 -04:00
fululi12Claude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>lifulu
 		8055293353 [ROCm][MLA] Cut ~21x host dispatch from the MLA metadata build (#58381) 
Signed-off-by: lifulu <fululi12@amd.com>
Signed-off-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: lifulu <lifulu@users.noreply.github.com>
 2026-10-12 13:11:20 -05:00
Woosuk KwonandClaude Opus 5.5 e1a5086256 [Attention] Remove cascade attention (#61126) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 10:41:07 -07:00
Maroon AyoubRobert ShawMatthew BonanniClaude Opus 5.5
 		5b0b8f251e [Bugfix][HiSparse] Fix TP group hang on steps without scheduled tokens (#60941) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 17:38:12 +00:00
simon-veitner-redhatClaudeLucas Wilkinson
 		a7b7bc1fbe [Metrics] Count finished requests by watermark status (#58715) 
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-12 17:21:22 +00:00
Misha GoinandClaude Opus 5.5 45bb710eca [CI] Add nightly Python 3.14 smoke test (#60456) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 10:11:26 -07:00
Chester c599b5971f [Bugfix][Frontend] Return all logprobs when completions requests logprobs=-1 (#55717) 
Signed-off-by: Chester <42577861+superdiaodiao@users.noreply.github.com>
 2026-10-12 17:05:55 +00:00
haotiaa 6e6e86078a [Bugfix][Platform] Don't resolve current_platform during import vllm (#60766) 
Signed-off-by: Haotian An <haotiaa@amazon.com>
 2026-10-12 17:05:34 +00:00
Misha GoinSimon VeitnerClaude
 		fa39582756 [Frontend] Serve LiquidAI/d1-3B on /v1/systemone (#60676) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-12 13:00:47 -04:00
Misha GoinandClaude d79b52b182 [Misc] Consolidate csrc shared headers and prune dead code (#60721) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-12 09:46:22 -07:00
MattClaude Opus 5.5Andreas Karatzas
 		e0317b28ce [Feature] Support batch invariance on ROCm (#52231) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-12 10:53:26 -05:00
TJian 50b21a2667 [Perf] [Feat] [ROCm] Add densemha support to ROCm AITER Sparse MLA (#49263) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-10-12 15:50:28 +00:00
Shang WangandClaude Opus 5.5 ffec221276 [Bugfix][DSv4.1] Match decoder-replay input_ids dtype for FlashInfer MegaMoE (#61041) 
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 14:23:00 +00:00
Shang WangandClaude Opus 5.5 716961cf1f [Bugfix] Drop enable_in_kernel_fc2_reduce from FlashInfer CuteDSL MegaMoE config (#59900) 
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 09:05:09 -05:00
Ilya Markov a281ad32f0 [EPLB] Add EPLB support to dspark draft models (DeepSeekV4) (#57549) 
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
 2026-10-12 10:03:09 -04:00
Ilya Markov b3ee560ae8 [Bugfix][MoE] Apply expert_map remapping for CUTLASS FP4 MOE expert parallel (#59042) 
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
 2026-10-12 10:02:56 -04:00
bzsuni f022975964 [Bugfix] Reject beam search in completion render (#61212) 
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
 2026-10-12 13:49:46 +00:00
Suraj SharanVadim Gimpelsonlinitra24
 		4c36d21280 [Bugfix][LoRA] Apply PEFT rank_pattern / alpha_pattern scaling per module (#59801) 
Signed-off-by: suraj <surajsharan85@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io>
 2026-10-12 21:49:30 +08:00
Harry MellorClaude Opus 5.5Kevin H. Luu
 		3facf822b7 [Performance] Load HF tokenizers and processors from a local snapshot (#60825) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-12 13:45:54 +00:00
Jiangyun ZhuClaude Opus 5.5Isotr0py
 		092c35dd15 [Model] Support Cloudflare/clef-omni on /v1/systemone (#61064) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-12 21:27:34 +08:00
DimensionSTPandArtem Perevedentsev 314ab8cffd [Bugfix][Structured Output] Drain compilation tasks before releasing the backend (#61047) 
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-10-12 16:25:36 +03:00
stefankoncarevic 8c175f403a [CI][ROCm] Compare greedy output only in the AITER attn+quant test (#61185) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-10-12 08:20:35 -05:00
Rohan PotdarandClaude Opus 5.5 7900d2eabc [Test] Accept tied tokens at the k-th value in the Triton top-k test (#60726) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 08:18:44 -05:00
Harry Huangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> c75cb63cd2 [Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next (#56322) 
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-12 20:56:44 +08:00
Tianyu Guo 258b97ff6b [Bugfix][EC Connector] Size transfers by measured encoder output widths (#57904) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-10-12 12:32:39 +00:00
Shimi BandielHyeonki HongMimirClaudeBugen Zhao
 		8d7b93fec2 [Frontend] Report stop_reason on /inference/v1/generate and pass it through derender (#60251) 
Signed-off-by: Hyeonki Hong <hhk7734@gmail.com>
Signed-off-by: Shimi Bandiel <shimib@google.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Hyeonki Hong <hhk7734@gmail.com>
Co-authored-by: Mimir <mimir@nousresearch.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-12 19:37:53 +08:00
MatthewCYM f7a3a7265f [Bugfix][Multimodal] Fix Qwen3-ASR audio preprocessing parity (#57146) 
Signed-off-by: MatthewCYM <chenyiming0v0@gmail.com>
 2026-10-12 10:42:51 +00:00
Shijin ZhangandChauncey 83773ecdf3 [Frontend] Support CustomToolCall in the non-Harmony Responses API (#57322) 
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-10-12 17:22:55 +08:00
Thuongvu HoandNicolò Lucchesi 99b44c7a42 [Metrics] Add admission rejection metrics (#55812) 
Signed-off-by: Thuongvu Ho <thuongvuho@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
 2026-10-12 08:31:07 +00:00
Ting SUN f4114e068e [Bugfix][Frontend] Preserve token IDs in chat derender responses (#61065) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-10-12 07:55:06 +00:00
AMD-yanfeiwangandaoshen02 e3bbaf2369 [ROCm][DSv4] Fuse the shared expert into AITER MegaMoE (#61004) 
Signed-off-by: Yanfei Wang <yanfei.wang@amd.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-12 15:53:16 +08:00
Giancarlo DelfinandRishi Puri 36f3972812 [Spec Decode] Skip metadata build for dflash/dspark when all model backends support it (#60692) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Rishi Puri <20074092+puririshi98@users.noreply.github.com>
 2026-10-12 00:09:10 -07:00
Zhenzhong XuandKunshang Ji 6a70913590 [XPU] bump up auto-round-lib to 0.16.0 (#60976) 
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-12 14:47:59 +08:00
HubertCodexAo Shen
 		74f4404f58 [Bugfix][Distributed] Own SHM out-of-band buffers before releasing read slots (#61000) 
Signed-off-by: Hubert Marek <163992334+hubert-marek@users.noreply.github.com>
Signed-off-by: Ao Shen <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Ao Shen <aoshen@inferact.ai>
 2026-10-12 01:44:12 -04:00
Canlin GuoandClaude Opus 5.5 fe4df3df0e [Bugfix][GLM-5.3-Flash] Restore MTP index sharing across draft iterations (#61079) 
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 05:41:52 +00:00
shaohuaxiCodexCyrus Leung
 		d4e92ccc37 [Bugfix][Frontend] Honor endpoint plugin route overrides (#58839) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-12 03:58:15 +00:00
liuzhenweiandKunshang Ji bfa7e5b7af [XPU][CI] Enable XPU graph in graph tests (#60784) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-12 10:39:40 +08:00
Yan Ma f147949be8 [XPU] limit FA kernel block size to multipleOf 64 (#57297) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-10-12 10:25:32 +08:00
dingdangmaoandCodex 124b89d5b7 [Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries (#59962) 
Signed-off-by: apex-mochen <2756823972@qq.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-12 02:08:14 +00:00
Nick Hill 3fe917abb1 [Bugfix][MRV2] Fix FlashInfer attention and ReplaySSM warmups on MRV2 (#61013) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-11 16:04:41 -07:00
Jared WenClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f26d9f432c [Bugfix][GLM-5.3-Flash] Honor --mamba-ssm-cache-dtype for the KDA recurrent state (#59147) 
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-11 17:55:00 +00:00
Nick Hill c059bee812 [MRV2] Switch CI tests still running on MRV1 unnecessarily (#61014) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-11 09:57:07 -07:00
Ting SUNandCyrus Leung b99fdbe1f8 [Bugfix] Respect the configured offline chat template (#61061) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-11 16:27:21 +00:00
Juan Pérez de AlgabaRobert ShawItay Etelis
 		93d2cab35e [Security] Fix P2P ZMQ socket exhaustion via unbounded peer sessions (#51504) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
 2026-10-11 11:39:11 -04:00
aoshen02andCodex 239a3fa3a8 [Sleep] Retain draft-private weights with existing parameter selector (#61046) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-11 23:30:56 +08:00
Zheng CairootOpenAI CodexzigzagcaiClaude CodeWentao Ye
 		3e8e1364ff [Perf][GLM-5.3-Flash] Shard long-context indexer prefill rows across TP (#54951) 
Signed-off-by: Zheng Cai <8370601+zigzagcai@users.noreply.github.com>
Signed-off-by: Zheng Cai <caizheng1993@gmail.com>
Signed-off-by: Codex <noreply@openai.com>
Co-authored-by: root <root@rjob-fb29567b88fec0a9-88c9a4c797eb2ec1-0.caizheng.ailab-sys.svc.pjlab.local>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-11 10:43:37 -04:00
aoshen02 bd5bbce6e4 [RL] Release weight-transfer communicators without stopping inference (#61060) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
 2026-10-11 21:15:14 +08:00
tc-mbandCyrus Leung c85616efc6 [Model] Support MiniCPM-V 4.7 (#58674) 
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-11 10:21:10 +00:00
Subhash Polisetti f425a21208 [Bugfix][Frontend] Support scalar priority and wire priority into LLM.chat() (#57243) 
Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
 2026-10-11 10:03:33 +00:00