iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,299 Commits 3 Branches 0 Tags
040700aaa6cae3ef54dad8ba75db1abfb79db449
Commit Graph
2618 Commits
Author SHA1 Message Date
Cyrus Leung 0b19ebcacd [MM] Simplify _apply_hf_processor_main (#53275) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-08-22 01:49:37 -07:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00
Maroon Ayoub a556f3fccb Forward Anthropic vllm_xargs to sampling params (#53308) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
 2026-08-21 15:23:21 -07:00
Kevin H. LuuandAndreas Karatzas e3f6026503 Revert "Remove native Hunyuan V1 and VL implementations" (#53296) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-21 13:39:58 -07:00
Tiezhen WANGandOpenAI Codex d53b1c2efc Remove native Hunyuan V1 and VL implementations (#53272) 
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-21 09:07:10 -07:00
Russell Bryant 6d8cd88e8d [Docs] document cache salting for prefix cache timing side-channel mitigation (#39082) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2026-08-21 09:06:49 -07:00
Luke Alonsoandmgoin 2740c817ff [Kernel] Add b12x FP4 MoE backend (#52018) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-08-21 08:04:36 -07:00
Cyrus Leung b00f475f09 [MM] Remove text components from ProcessorInputs (#53093) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-08-20 20:42:53 -07:00
Kevin H. LuuandOpenAI Codex 0a5a55136f [CI][Docker] Pin remaining manylinux builder images (#53172) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-20 23:17:02 +00:00
Matthew KotilaReed MeyersonClaude Opus 4.8
 		7cfb97e337 [Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915) 
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-20 15:42:49 -07:00
SeonjinandMisha Goin bd8865a299 [Kernel] Add FlashInfer TRTLLM MXFP8 linear backend (#52204) 
Signed-off-by: seonjinn <sna@nvidia.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-08-20 10:07:22 -04:00
Prudhvi Vuda de216b6e64 [Bugfix] Skip MM processor cache inserts larger than capacity (#53016) 
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
 2026-08-20 13:42:50 +00:00
Turner JabbourandClaude Sonnet 5 1eab6fef01 [CI] replace shellcheck script with shellcheck-py hook (#52572) 
Signed-off-by: Turner <doubleujabbour@gmail.com>
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-08-20 10:09:59 +00:00
Zhixia LiuianzxliuClaudeaoshen02CodexNick Hill
 		a1c5b1fd9f [Core][V1] Support trace_decode_token_ids for deterministic decode replay (#46701) 
Signed-off-by: ianzxliu <ianzxliu@tencent.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Zhixia Liu <liuzhixia.nju@gmail.com>
Co-authored-by: ianzxliu <ianzxliu@tencent.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-20 15:12:43 +08:00
pmanczakandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> f85e060ec2 [Doc] Update Gaudi HPU committers (#52726) 
Signed-off-by: pmanczak <pawel.manczak@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-20 10:59:22 +08:00
Guanyi Chenandz-zanez bf2866f8bf [KV Connector] Add decode offloading to Mooncake Store consumers (#52466) 
Signed-off-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: Guanyi Chen <939416532@qq.com>
Co-authored-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
 2026-08-19 17:40:35 -07:00
jcotant-inferactClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c205726108 [CI] Fix and extend PR/issue auto-labeling (#51459) 
Signed-off-by: Joe Cotant <joe@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-19 21:45:44 +00:00
Harry Mellor 2f54100a59 [CI] Fix docs build (#52937) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-19 05:25:46 -07:00
QWERQWERQWE86 58302b4591 [Doc] Fix group numbering in Case 3 of hybrid_kv_cache_manager.md (#52160) 
Signed-off-by: qwerqwerqwe8688-jpg <xuuuuu2021@163.com>
 2026-08-19 11:38:34 +00:00
wang.yuqi eac636a7fa [Frontend] Move api_server.py out openai folder (#52131) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-19 09:43:46 +00:00
Davis WertheimerClaudeHarry Mellor
 		3130573630 [Model] Add GraniteSWA and GraniteMoeSWA via existing Granite (#52706) 
Signed-off-by: Davis Wertheimer <davis.wertheimer@ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-18 20:32:46 -07:00
Russell BryantandClaude Opus 4.8 ef47a897e2 [Core] Make prefix-cache NONE_HASH deterministic by default (#51875) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-18 23:14:39 +00:00
Andreas Karatzas 203926c477 [ROCm][CI] Add AMD CI Pull-Request Commands (#52822) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-18 15:16:25 -07:00
karthik 241ff8c443 [Model] Enable LoRA support for tower and connector in LlavaNextForConditionalGeneration (#49788) 
Signed-off-by: gangula-karthik <gkarthik923@gmail.com>
 2026-08-18 15:53:49 +00:00
Bo Liandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> c296851a7d [MoE] Refine FlashInfer one-sided All2All integration (#51924) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-18 00:18:14 +00:00
Roberto L. CastroandCodex 49905ad94d [3/N][Feat][Perf] Add new warmup infrastructure for JITs. Add provider registry and orchestration for JIT warmup (#50174) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-08-17 10:11:14 -07:00
OlaandOpenAI Codex 95901ce70a fix(pooling): validate BGE-M3 combined task ownership (#51823) 
Signed-off-by: Zhe Li <2843409461@qq.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-17 10:30:35 +00:00
TJian c05d923f18 [Doc] [ROCm] Update installation documentation (#52303) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-08-17 01:56:44 -07:00
Amal Sebastian 0ff370b51c docs: fix incorrect --custom-skip-chat-template flag reference (#52588) 2026-08-17 01:48:11 -07:00
Yiting Jiang 502af5ed00 [Doc] Add MatrixHub as a model loading source (#50492) 
Signed-off-by: yiting.jiang <yiting.jiang@daocloud.io>
 2026-08-17 02:32:23 +00:00
Shanshan ShenandAndreas Karatzas 0ad04cff1b [ROCm][CI] Enable ViT CUDA graph tests on AMD gfx950 GPUs (#52256) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-17 09:52:02 +08:00
Shane Widanagama d4801990a4 [CI/Build] Add warning for unsupported global PTX architecture requests in... (#51901) 2026-08-15 07:01:52 -07:00
Connor CarpenterandNick Hill f473870ecf [Rust Frontend][gRPC] Add RL lifecycle control (#51316) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-14 11:32:39 -07:00
Jee Jee Li 69e0e58da1 [Doc] Update model support information (#52289) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-08-14 09:47:30 +00:00
Kunshang Ji 8e6d8e4f6a [XPU][CI/Release][3/N] Add xpu wheel release to release pipeline (#52108) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-08-14 14:00:17 +08:00
Tzu-Ling KanandClaude Opus 4.8 653cc6faca [Bugfix][NIXL] Include transfer mode (push/pull) in the compatibility hash (#50620) 
Signed-off-by: Tzu-Ling <tzulingk@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-14 12:11:22 +08:00
Luke Alonso 3c79b1a8bf [Kernel] Add B12X dense linear backends (#52016) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
 2026-08-13 19:48:33 -07:00
fxmarty-amdAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c5b7c069a4 [ROCm] Defer tilelang import through its import from vllm.tilelang_utils import tilelang and relaxed has_tilelang (#51159) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 13:12:40 -05:00
linitra24Jee Jee Lilinitra24
 		95c9144424 [LoRA][Gemma4] Support vision tower LoRA (#42662) 
Signed-off-by: bk-201 <joy25810@foxmail.com>
Signed-off-by: linitra24 <Joy25810@foxmail.com>
Signed-off-by: linitra24 <joy25810@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: linitra24 <joy25810@gmail.com>
 2026-08-13 22:07:43 +08:00
QWERQWERQWE86 443fa5aca0 [Doc] Fix stale rejection_sample_method and synthetic_acceptance_rate (#51611) 
Signed-off-by: qwerqwerqwe8688-jpg <xuuuuu2021@163.com>
 2026-08-13 10:56:27 +00:00
vx120aoshen02CodexClaude Opus 4.6OpenAI Codex
 		50ba4bc6b2 [Feature] Mask Replay (#49577) 
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-13 08:44:31 +00:00
Lucas WilkinsonOpenAI CodexClaude Opus 5Benjamin Chislettmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick HillOpenAI Codex
 		7f7a32cfec [Spec Decode] DSpark confidence-scheduled verification (#47808) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-12 12:27:48 -07:00
Russell BryantandClaude Opus 4.8 8151f2ad43 [Docs] Warn that --api-key does not gate all endpoints (#51999) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-12 08:16:35 -07:00
Aaron HaoandClaude Opus 5 793ca6998a [Docs][RL] Rewrite weight-transfer docs; standardize examples (#51729) 
Signed-off-by: haoaaron <ahao@anyscale.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-11 18:53:15 -07:00
鐘天楽 a367cdbca0 [Profiler] Add minimal Triton Proton profiling backend (#48789) 
Signed-off-by: Luossu <zhluosuu@outlook.com>
 2026-08-11 14:51:57 -07:00
Louie Tsai 5b184f775a connects vLLM Recipes with vLLM's native config-based deployment and benchmark (#51308) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-08-11 13:53:51 +00:00
Kyungmin LeeandHarry Mellor 529d010351 [Doc] Fix typos in speculative decoding docs (#51500) 
Signed-off-by: lkm2835 <lkm2835@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-11 09:17:05 +01:00
Yizhou a311916a29 [MRV2][Spec] Fuse AR speculator multi-step decodes back into one CUDA graph (#46849) 
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
 2026-08-11 15:06:22 +08:00
Khushali Desaiandwang.yuqi dc5101fb1b replace batch_norm to numerically identical without cudnn (#51734) 
Signed-off-by: khushali9 <khushali.desai9@gmail.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-11 12:43:20 +08:00
Michael GoinOpenAI CodexOpenAI Codex
 		21c667aa64 [Docker] Cache test dependencies before vLLM install (#51184) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-10 10:34:59 -07:00