iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
23,342 Commits 3 Branches 0 Tags
sm86
Commit Graph
103 Commits
Author SHA1 Message Date
MOVZXandHinata 5f3c7bd8ab [Mini-VLLM] Remove the ROCm device path, debug routes and arch-gated tests 
- Delete csrc/rocm, hipify, the _rocm_C extension and the setup.py rocm
  target path; drop the rocm requirement files
- Delete the aiter, rdna3, skinny-gemm and ROCm fused-MoE kernel modules
  and retarget the linear kernel registries away from them
- Reduce platforms/rocm.py to the GCN architecture predicates that live
  code still imports, and delete the RocmPlatform class and plugin
- Drop the VLLM_SERVER_DEV_MODE debug surface: reset, collective_rpc,
  server_info and sleep routes plus the log_error_stack flag
- Delete lm-format-enforcer, lark and otel-semantic-conventions-ai pins;
  the live server process maps none of them
- Delete tests that only build architectures this fork no longer
  registers, and test directories left without any test file
- Restore the tests/v1/kv_connector mock helpers that scheduler tests
  import at module level

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 19:31:07 +07:00
MOVZXandHinata 2066d3143f [Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack 
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
  rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
  select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
  quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
  requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
  fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
  dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
  mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
  cut deleted while keeping the model tree

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 17:10:56 +07:00
MOVZXandHinata 54a396586e [Mini-VLLM] Repair test suite after SM86 cut 
- Restore example connector, test helpers, and dropped patch tests
- Guard MLA metadata import in the example connector
- Point the PLE test at common.ops.ple after upstream #60021 move
- Cut tests for removed features: tools, DRY, epd proxy, STT batch,
  diffusion scheduler, INC/Quark PLE cases, mamba-2 mixer

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 01:06:30 +07:00
MOVZXandHinata 81074e94a3 [Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests 
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 00:20:48 +07:00
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
Haiyu ZuoandKevin H. Luu ff5a91a456 [Feature][Model Loader] Apply model-owned MTP shard selection to RunAI ModelStreamer (#60759) 
Signed-off-by: 浩韵 <zuohaiyu.zhy@alibaba-inc.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-15 12:23:11 +08:00
amd-xavierwangandJartX 06265beb25 [ROCm] Enable fp8 query for Triton attention (#60296) 
Signed-off-by: xawang <xawang@amd.com>
Co-authored-by: JartX <sagformas@epdcenter.es>
 2026-10-14 15:03:20 -04:00
Itay EtelisRobert ShawMisha Goin
 		13dbf96093 [Model Loader] Apply EP weight filtering in InstantTensor (#61167) 
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-14 09:10:10 -07:00
Ronaldandaoshen02 19aba3e4d5 [Bugfix][RL] Load attention sinks through weight_loader for layerwise reload (#57797) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-14 10:17:58 +00:00
Ilya BoytsovIlya BoytsovCodexMisha GoinClaude Opus 5.5
 		189bbfb9f4 Add MoNe experts pruning support (#50415) 
Signed-off-by: Ilya Boytsov <ieboytsov@nebius.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Ilya Boytsov <ieboytsov@nebius.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 23:18:43 +00:00
Jiangyun ZhuClaude Opus 5.5Isotr0py
 		092c35dd15 [Model] Support Cloudflare/clef-omni on /v1/systemone (#61064) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-12 21:27:34 +08:00
siyuClaude Opus 5Isotr0py
 		f2376cdc4d [Fast Start] Preload the FlashInfer autotune table on the weight cache daemon (#60085) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-10 15:48:06 +00:00
siliangchen-amdCursorShanshan Shen
 		02aa10539f [Bugfix][ModelLoader] Make enable_weights_track check quantized layers (#57108) 
Signed-off-by: siliangchen-amd <SiLiang.Chen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-10-10 10:44:06 +08:00
Ning XieandSimon Mo a0f28d73a8 [sharded state loader] support pp in sharded state loader (#52110) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2026-10-09 11:08:31 -07:00
fxmarty-amdOpenAI CodexMisha Goin
 		9f3d70e081 [Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) (#55686) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 12:04:46 -04:00
Harry MellorandClaude Opus 5.5 21d9758459 Remove Python 3.10 support now that it is EOL (#60402) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 02:30:52 -07:00
MattandClaude Opus 5.5 9dba339d07 [Bugfix][ROCm] Keep MLA derived weights in place on weight reload (#60389) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 20:38:06 -05:00
Xuanteng HuangClaude Fable 5.1OpenAI Codex
 		f38c6778ce [quantization] Add per-token NVFP4 quantization MoE support for ReLU2 (#56740) 
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-07 22:01:09 +00:00
Robert ShawRobert ShawClaude Opus 5.5
 		e572e02d21 [UX] Skip checkpoint shards an MTP head does not load (#60152) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 15:16:19 -04:00
viktorkouClaude Fable 5.1Isotr0py
 		c83935b300 [Bugfix][Model Loader] Support reload_weights with runai_streamer load format (#58124) 
Signed-off-by: Viktor Koukouliev <vkoukouliev@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-07 14:04:35 +00:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
shiyuan680shiyuan680Ronald1995DeepSeek Harnessaoshen02Claude Opus 5.5aoshen02
 		42d9f8ccdb support rl feature : weight checker (#51350) 
Signed-off-by: shiyuan680 <yangcheng104@huawei.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: shiyuan680 <yangcheng104@huawei.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: DeepSeek Harness <noreply@deepseek.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 03:13:43 +00:00
siyuIsotr0pyXun Sun
 		f5ed3d9c03 [Fast Start] Charge daemon-held weights against gpu_memory_utilization (#57298) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Xun Sun <UNIDY2002@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
 2026-09-29 17:58:35 +00:00
siyuIsotr0pyXun Sun
 		af5b4857e1 [Fast Start] Support PP (#55477) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
 2026-09-29 09:37:43 +00:00
Xun SunandIsotr0py e7900156e1 [Fast Start] Add /health endpoint for the weight cache daemon (#58552) 
Signed-off-by: Xun Sun <UNIDY2002@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-27 17:48:54 +00:00
Isotr0py 24928a4d62 [UX][Frontend] Introduce vllm preload cli for fast restart (#56680) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 06:56:39 +00:00
fxmarty-amdmgoinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		90a9515006 [Cleanup] Remove online quantization support in fp8.py in favor of online shorthands (#53585) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 12:48:14 -07:00
Xun Sun e773247f8a [Fast Start] Wait for weight cache daemon readiness (#58370) 2026-09-24 19:18:04 +00:00
GuorongYegrYe99Cursor Agent
 		1291bdb85e [EPD][Model Loader] Skip language-model checkpoint shards for --mm-encoder-only (#58086) 
Signed-off-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-09-24 00:41:15 -07:00
siyuandIsotr0py c024cfdb02 [Fast Start] Support data parallelism in the weight cache daemon (#57386) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-22 18:19:40 +00:00
siyuandIsotr0py c723a831a8 [Fast Start] Cache the MTP draft model in a separate daemon group (#57312) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-22 03:13:38 +00:00
Misha GoinandErinYin e5fce7b56b [Core][Kernel] Share persistent workspaces for Marlin and Humming (#57421) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: ErinYin <liang.yin@daocloud.io>
 2026-09-20 17:28:33 +08:00
vllm-agentkimi-no-na-waMisha Goin
 		b2f2bd71ba [CI][Bugfix] Add tp_shard_with_padding to padded MoE reload test mock (#57402) 
Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com>
Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-17 13:02:24 -04:00
Yuchen WangDakai AnOpenAI Codexaoshen02
 		78fdf4efb8 [Bugfix][MoE] Convert FlashInfer BF16 weights in place (#54699) 
Signed-off-by: Yuchen Wang <93700456+yuchenwang3@users.noreply.github.com>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-17 22:00:04 +08:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
aoshen02 eb61707851 [Bugfix] Fix stale HPC QK-norm weights after weight refit (#56706) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-09-14 21:34:41 -07:00
Isotr0py dbf49dad11 [Fast Start] Use GPU uuid as socket folder identifier (#56669) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-14 06:28:13 +00:00
siyuandIsotr0py 73fb19151f [Fast Start] Support fp4 (#55465) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-10 13:22:17 +00:00
Qiming Zhang 49632fb3ac [CI][XPU] Reject CUDA-IPC weight cache on non-CUDA/ROCm platforms (#56010) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
 2026-09-10 14:05:07 +08:00
Roderick Wuandmgoin 2c9d68f80b [Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering (#54809) 
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com>
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-08 12:45:12 -07:00
siyuliusy58Isotr0py
 		a69e75b9b6 Fast Start (#54921) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: liusy58 <liusy58@smail.nju.edu.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-04 06:32:06 -07:00
Stellar鱼 5182f2705b [CI/Build] Use file rendezvous for UniProc loader fixtures (#52367) 
Signed-off-by: yu-xin-c <2182712990@qq.com>
 2026-08-29 11:53:04 +08:00
Wyett b26039b09f Add routed expert loading for gpt-oss (#52209) 
Signed-off-by: Wyett <wyettzeng@gmail.com>
Signed-off-by: wyett <wyettzeng@gmail.com>
 2026-08-23 17:59:08 +08:00
Shuolei Wangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 236f78cc5c [Core][RL] Support sparse checkpoint updates through native weight loaders (#50723) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-22 17:39:58 +00:00
Harry Mellor f0c14b4f77 Fix weight tying (#51665) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-20 10:31:55 +00:00
aoshen02andCodex f2906b492e [Bugfix] Avoid repeated layerwise reload warning scans (#49505) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-08-12 13:53:50 +08:00
Richard Zou cb30f6f8dd [Testing] Fix test_sharded_state_loader (#51736) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-08-11 21:16:09 +00:00
aoshen02andRoger Wang 99e62b802c [Bugfix][Model Loader] Defer post-load attention weight processing (#49519) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-08-10 19:16:57 -07:00
aoshen02andaoshen02 7153fd70f6 [Bugfix][MoE] Filter packed expert weights during EP loading (#49558) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-08-04 16:32:03 +00:00