iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
23,331 Commits 3 Branches 0 Tags
6a95f62e2fbfebca86ec1e476e3b2e6fd8a574ab
Commit Graph
485 Commits
Author SHA1 Message Date
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
Haiyu ZuoandKevin H. Luu ff5a91a456 [Feature][Model Loader] Apply model-owned MTP shard selection to RunAI ModelStreamer (#60759) 
Signed-off-by: 浩韵 <zuohaiyu.zhy@alibaba-inc.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-15 12:23:11 +08:00
Itay EtelisRobert ShawMisha Goin
 		13dbf96093 [Model Loader] Apply EP weight filtering in InstantTensor (#61167) 
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-14 09:10:10 -07:00
Ronaldandaoshen02 19aba3e4d5 [Bugfix][RL] Load attention sinks through weight_loader for layerwise reload (#57797) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-14 10:17:58 +00:00
Woosuk KwonandClaude Opus 5.5 f47792af7f [Model Loader] Make dummy weight init dtype-aware (#61134) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 18:33:26 -07:00
Ilya BoytsovIlya BoytsovCodexMisha GoinClaude Opus 5.5
 		189bbfb9f4 Add MoNe experts pruning support (#50415) 
Signed-off-by: Ilya Boytsov <ieboytsov@nebius.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Ilya Boytsov <ieboytsov@nebius.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 23:18:43 +00:00
Jiangyun ZhuClaude Opus 5.5Isotr0py
 		092c35dd15 [Model] Support Cloudflare/clef-omni on /v1/systemone (#61064) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-12 21:27:34 +08:00
Yan Ma e05201aa2e [XPU] refine memory release for sleep mode (#61158) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-10-12 16:59:38 +08:00
debanshdIsotr0pyKimi Code
 		ef42b4385f [Bugfix] Reclaim cached GPU memory during weight loading (#57516) 
Signed-off-by: Debanshu Das <debanshu@google.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-10-11 07:03:06 +00:00
Nick Hill f8471164bf [MRV2] Detatch weight-reload logic from MRV1 (#60937) 2026-10-10 21:15:05 -07:00
siyuClaude Opus 5Isotr0py
 		f2376cdc4d [Fast Start] Preload the FlashInfer autotune table on the weight cache daemon (#60085) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-10 15:48:06 +00:00
siliangchen-amdCursorShanshan Shen
 		02aa10539f [Bugfix][ModelLoader] Make enable_weights_track check quantized layers (#57108) 
Signed-off-by: siliangchen-amd <SiLiang.Chen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-10-10 10:44:06 +08:00
Ning XieandSimon Mo a0f28d73a8 [sharded state loader] support pp in sharded state loader (#52110) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2026-10-09 11:08:31 -07:00
Harry MellorandClaude Opus 5.5 21d9758459 Remove Python 3.10 support now that it is EOL (#60402) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 02:30:52 -07:00
Xuanteng HuangClaude Fable 5.1OpenAI Codex
 		f38c6778ce [quantization] Add per-token NVFP4 quantization MoE support for ReLU2 (#56740) 
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-07 22:01:09 +00:00
Robert ShawRobert ShawClaude Opus 5.5
 		e572e02d21 [UX] Skip checkpoint shards an MTP head does not load (#60152) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 15:16:19 -04:00
viktorkouClaude Fable 5.1Isotr0py
 		c83935b300 [Bugfix][Model Loader] Support reload_weights with runai_streamer load format (#58124) 
Signed-off-by: Viktor Koukouliev <vkoukouliev@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-07 14:04:35 +00:00
fxmarty-amdvllmellmTan Pin SiangMisha Goin
 		f8d92cae41 [Quantization] Layer re-quantization for linear layers through online quantization API (MXFP8 -> FP8 PTPC showcase) (#55684) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-06 11:38:18 -05:00
Jee Jee Li f9c9e8ac24 [LoRA] Remove tensorizer (#60024) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-10-05 07:01:56 -07:00
Ning Xie bfbcde26d5 [Observability] add model initializing duration log (#52141) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-10-01 02:54:33 +00:00
shiyuan680shiyuan680Ronald1995DeepSeek Harnessaoshen02Claude Opus 5.5aoshen02
 		42d9f8ccdb support rl feature : weight checker (#51350) 
Signed-off-by: shiyuan680 <yangcheng104@huawei.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: shiyuan680 <yangcheng104@huawei.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: DeepSeek Harness <noreply@deepseek.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 03:13:43 +00:00
siyuIsotr0pyXun Sun
 		f5ed3d9c03 [Fast Start] Charge daemon-held weights against gpu_memory_utilization (#57298) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Xun Sun <UNIDY2002@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
 2026-09-29 17:58:35 +00:00
siyuIsotr0pyXun Sun
 		af5b4857e1 [Fast Start] Support PP (#55477) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
 2026-09-29 09:37:43 +00:00
Ashraf Bhuiyan b9e2903b89 [Mypy] Fix mypy typing for Ultravox and Unlimited-OCR models (#58239) 
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com>
 2026-09-27 21:19:04 -04:00
Isotr0py 24928a4d62 [UX][Frontend] Introduce vllm preload cli for fast restart (#56680) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 06:56:39 +00:00
Juntian LiuKimi CodeClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		6936e77ba4 [Perf][Engram] Serialize offloaded lookups and pack host tables into huge pages (#56926) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 10:52:02 +08:00
fxmarty-amdmgoinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		90a9515006 [Cleanup] Remove online quantization support in fp8.py in favor of online shorthands (#53585) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 12:48:14 -07:00
Xun Sun e773247f8a [Fast Start] Wait for weight cache daemon readiness (#58370) 2026-09-24 19:18:04 +00:00
LucainClaude Opus 5Harry Mellormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		344fcc252a [Bugfix] Resolve the Hub revision once per repo (#56092) 
Signed-off-by: Wauplin <lucainp@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 08:48:24 -07:00
Harry MellorandClaude Opus 5.5 d881812500 [Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 14:14:10 +00:00
GuorongYegrYe99Cursor Agent
 		1291bdb85e [EPD][Model Loader] Skip language-model checkpoint shards for --mm-encoder-only (#58086) 
Signed-off-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-09-24 00:41:15 -07:00
Wentao Ye fae5cbfc8b [Refactor] Remove dead code multiple places (#58002) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-22 16:36:03 -04:00
siyuandIsotr0py c024cfdb02 [Fast Start] Support data parallelism in the weight cache daemon (#57386) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-22 18:19:40 +00:00
siyuandIsotr0py c723a831a8 [Fast Start] Cache the MTP draft model in a separate daemon group (#57312) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-22 03:13:38 +00:00
Yuchen WangDakai AnOpenAI Codexaoshen02
 		78fdf4efb8 [Bugfix][MoE] Convert FlashInfer BF16 weights in place (#54699) 
Signed-off-by: Yuchen Wang <93700456+yuchenwang3@users.noreply.github.com>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-17 22:00:04 +08:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
AlexHuangOpenAI CodexItay Etelis
 		0f8fa53acf [Bugfix][KV Offload] Check cgroup memory before SHM allocation (#54014) 
Signed-off-by: Alex <jihui.huang@daocloud.io>
Signed-off-by: AlexHuang <alex.tech.lab@outlook.com>
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
 2026-09-16 09:17:50 +03:00
aoshen02 eb61707851 [Bugfix] Fix stale HPC QK-norm weights after weight refit (#56706) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-09-14 21:34:41 -07:00
shaohuaxiCodexIsotr0py
 		567f745c1b [Bugfix] Export weight-cache IPC tensors separately for each client (#56472) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-15 02:43:42 +00:00
Isotr0py dbf49dad11 [Fast Start] Use GPU uuid as socket folder identifier (#56669) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-14 06:28:13 +00:00
siyuandIsotr0py 52dd0d7562 [Fast Start] Fast loader support nnode>1 (#55468) 
Signed-off-by: siyu <mg21330037@smail.nju.edu.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2026-09-14 03:05:54 +00:00
Woosuk KwonandOpenAI Codex 71888f507a [Perf] Avoid redundant conversions in FP8 dummy initialization (#56688) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-13 12:06:13 -07:00
Woosuk Kwon e7a3963339 [Bugfix] Avoid repeated dummy initialization and random CPU Engram fills (#56682) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-09-13 02:23:20 -07:00
Isotr0py 533dbe424b [Misc] Clean up fast loader daemon quant method verification (#55656) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-12 20:16:39 -07:00
siyuandIsotr0py 73fb19151f [Fast Start] Support fp4 (#55465) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-10 13:22:17 +00:00
Qiming Zhang 49632fb3ac [CI][XPU] Reject CUDA-IPC weight cache on non-CUDA/ROCm platforms (#56010) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
 2026-09-10 14:05:07 +08:00
Harry Huangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 3116c5d06b [Qwen4Exp] Support UVA PLE-offload and Engram tensor parallelism (#54371) 
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-09 07:32:34 -07:00
Roderick Wuandmgoin 2c9d68f80b [Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering (#54809) 
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com>
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-08 12:45:12 -07:00
Wentao Ye 6b15bea080 [Refactor] Remove utils dead code (#53941) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-08 11:22:09 -04:00