iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
23,342 Commits 3 Branches 0 Tags
sm86
Commit Graph
403 Commits
Author SHA1 Message Date
MOVZXandHinata 54a396586e [Mini-VLLM] Repair test suite after SM86 cut 
- Restore example connector, test helpers, and dropped patch tests
- Guard MLA metadata import in the example connector
- Point the PLE test at common.ops.ple after upstream #60021 move
- Cut tests for removed features: tools, DRY, epd proxy, STT batch,
  diffusion scheduler, INC/Quark PLE cases, mamba-2 mixer

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 01:06:30 +07:00
MOVZXandHinata 0152af1336 [Mini-VLLM] Remove upstream CI, docs, and lint surfaces 
- Delete .buildkite: fork runs no Buildkite pipelines
- Delete mkdocs.yaml, .readthedocs.yaml: docs/ is gone
- Delete rust-toolchain.toml: Rust frontend is gone
- Delete .pre-commit-config.yaml: all hooks call deleted tools/pre_commit
- Delete .dockerignore: docker/ is gone
- Drop three tests importing deleted quark and mooncake modules

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 00:43:46 +07:00
MOVZXandHinata 81074e94a3 [Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests 
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 00:20:48 +07:00
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
Misha GoinandClaude Opus 5.5 4d9a8d6a41 [Bugfix] Make Humming DeviceInfo lookups traceable under torch.compile (#61858) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 11:12:21 -07:00
Chaojun ZhangKevin H. LuuCyrus Leung
 		85c94c1706 [Tests][XPU] Skip modelopt_fp4 online MoE re-quantization test on XPU (#61723) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-15 00:23:45 -07:00
Misha GoinClaudeKevin H. Luu
 		984c242ede [Quantization] Support compressed-tensors MXFP4 MoE with FP8 activations via Mxfp4MoEMethod (#59815) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-14 13:33:35 -07:00
Stefano CastagnettaandMisha Goin 222e00fc44 [CI/Build] Fix test_deepseek_v41_mxfp8_scale_loading after #57797 (#61678) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-14 12:17:59 -07:00
fxmarty-amdvllmellmTan Pin Siang
 		c9bfa574ce [Online quantization] Extend re-quantization API to MOE (showcase NVFP4 modelopt/CT/quark to any online MOE method) (#60276) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com>
 2026-10-14 15:49:37 +08:00
Andrii SkliarAndrii SkliarLuke Alonso
 		6849ca696a [Quantization] Add ModelOpt IQ2_XS, IQ2_XXS, and Q8_0 support via b12x (#59604) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Luke Alonso <lalonso@gmail.com>
 2026-10-13 09:06:33 -07:00
Zhenzhong XuandKunshang Ji 6a70913590 [XPU] bump up auto-round-lib to 0.16.0 (#60976) 
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-12 14:47:59 +08:00
Yan Ma 4e318ac563 [XPU] fix online fp8_per_channel quantization (#56027) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-10-09 18:38:01 +08:00
fxmarty-amdOpenAI CodexMisha Goin
 		9f3d70e081 [Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) (#55686) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 12:04:46 -04:00
stefankoncarevicandAndreas Karatzas 814301b268 [Bugfix][Quant] Read safetensors headers of checkpoints with other file names (#60411) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-08 01:38:41 -05:00
v-shobhitClaude Opus 5.5Benjamin Chislett
 		8faac3dd2a [Bugfix] [Spec Decode] DeepSeek V4/V4.1 NVFP4: keep ignored MTP/DSpark experts on MXFP4 (#59655) 
Signed-off-by: Shobhit Verma <shobhitv@nvidia.com>
Signed-off-by: v-shobhit <161510941+v-shobhit@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2026-10-08 00:10:32 +00:00
Xuanteng HuangClaude Fable 5.1OpenAI Codex
 		f38c6778ce [quantization] Add per-token NVFP4 quantization MoE support for ReLU2 (#56740) 
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-07 22:01:09 +00:00
Jinzhen LinCodexmgoinClaude Opus 5.5
 		d1f3d8b870 [Quantization] Prefer Humming before Marlin backends on SM90 (#56997) 
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 12:31:35 -07:00
fxmarty-amdvllmellmTan Pin SiangMisha Goin
 		f8d92cae41 [Quantization] Layer re-quantization for linear layers through online quantization API (MXFP8 -> FP8 PTPC showcase) (#55684) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-06 11:38:18 -05:00
Markus Hartikainen b0e21b3083 [ROCm][Kimi-K3] Add opt-in gfx942 MXFP4-to-int4 conversion (#51274) 
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Signed-off-by: Hartikainen <mahartik@amd.com>
 2026-10-03 14:28:02 +08:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
Rukhaiya2004andLi, Jiang <jiang1.li@intel.com> 72e7874fa6 [Hardware][Power] Enable W4A16 (AWQ & GPTQ) quantization on POWER10 using VSX (#59149) 
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-30 19:44:48 +08:00
TaimysCodexLuyi XiaoTaimysHarry MellorClaude Opus 5.5
 		5225f1bacb [MyPy][3/N] Fix MyPy errors in test groups (part 3) (#55939) 
Signed-off-by: Taimys <ffy2281693949@gmail.com>
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Co-authored-by: Taimys <ffy2281693949@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 11:06:07 +00:00
Matej Sirovatkaaoshen02OpenAI Codexaoshen02
 		0103a9b96f [Quantization] Add per-token NVFP4 CuTe-DSL MoE backend (#50030) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-09-30 10:42:03 +00:00
Harshal AdhavCursorLi, Jiang <jiang1.li@intel.com>
 		5faf81a429 [CPU][Whisper] Support W4A16 quantized Whisper on the CPU WNA16 kernel (#58268) 
Signed-off-by: Harshal Adhav <harshal.adhav@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-30 13:52:53 +08:00
Spandan TiwariClaudeAndreas Karatzas
 		9b29370524 [Bugfix] Fix moe_wna16 w13 zero-point shard split for 8-bit asym GPTQ MoE (#58950) 
Signed-off-by: Spandan Tiwari <sptiwari@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-28 23:38:28 +00:00
HDCharlesClaude Opus 4.6HDCharles <{"message":"Not Found","documentation_url":"https://docs.github.com/rest/users/emails#list-email-addresses-for-the-authenticated-user","status":"404"}>OpenAI CodexOpenAI Codex
 		c01368fd64 [Quant] Use canonical N-first weight format for CT WNA16 MoE (#52798) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: HDCharles <{"message":"Not Found","documentation_url":"https://docs.github.com/rest/users/emails#list-email-addresses-for-the-authenticated-user","status":"404"}>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-28 11:22:44 -06:00
Wentao Ye bf93e1f92c [Refactor] Remove dead tests utils (#58803) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-26 09:16:53 -04:00
xiao-llmandCursor 5840d95284 [Bugfix][ROCm] AMD-Quark mixed-precision DeepSeek-V4.1 support (#57071) 
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-25 20:51:18 -05:00
Thang NguyenClaude Opus 5.5Andreas KaratzasCodex
 		da1aaec31f [CI] Split (H200 MIG/MI300) Basic Correctness into named jobs (#57054) 
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-25 17:05:33 -07:00
stefankoncarevicAndreas KaratzasAndreas Karatzas
 		1b922ffcbe [ROCm][CI] Add quantized MoE serving test for gfx950 (#58748) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 18:02:15 +00:00
Samuel Kimandmgoin f55419c43c [Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxfp8 shape limits (#54223) 
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-25 02:54:11 +00:00
Matej Sirovatkaaoshen02Claude Opus 5aoshen02
 		3652f35e7c [Bugfix][Quantization] Refresh online NVFP4 scales before reload post-processing (#57954) 
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-24 18:36:08 -07:00
Misha Goin 4f72bd0152 [Bugfix][Quantization] Give LM heads standard linear metadata (#58444) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-24 16:39:49 -07:00
fxmarty-amdmgoinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		90a9515006 [Cleanup] Remove online quantization support in fp8.py in favor of online shorthands (#53585) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-24 12:48:14 -07:00
Wentao Ye 3c42385d7d [Refactor] Remove dead or duplicate tests (#58446) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-24 10:29:56 -04:00
liuzhenwei dcfc17e0b1 [XPU] enable XPU GRAPH by default (#51600) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-09-24 19:17:25 +08:00
fxmarty-amdandmgoin 0549e8d0ab [Quark] Remove quark-specific silent online quantization (#51800) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-23 11:35:30 -04:00
Matej Sirovatkaaoshen02OpenAI Codexaoshen02Claude Opus 5.5
 		9127170b88 [Quantization] Select per-token NVFP4 MoE backends explicitly (#57176) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-23 07:30:58 +00:00
Jinzhen LinCodexmgoin
 		7d99c2c4fd [Feature][Humming] Humming feature integration (#56685) 
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-20 08:11:01 -07:00
siyuandIsotr0py 4cc15f2121 [Quantization] Let ModelOpt MXFP8 layers load pre-processed weights Purpose (#57316) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2026-09-19 09:16:39 +00:00
Artur FierkaClaudeKunshang Ji
 		1cdf1689e5 [Quantization][XPU] Enable int8_w8a8 MoE on the Triton backend for XPU (#53162) 
Signed-off-by: Artur Fierka <artur.fierka@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-18 22:38:15 +08:00
Asaf GardinHarry MellorClaude Opus 5
 		44dd18fe0b [Model][Gemma4] Load Weights with AutoWeightsLoader (#55911) 
Signed-off-by: Josephasafg <ajgard7@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-18 12:07:30 +00:00
meng-amd 6df2b1a8c6 [Quantization] Support native Quark W4A16 INT4/UINT4 exports in vLLM (#48606) 
Signed-off-by: HongWei Meng <HongWei.Meng@amd.com>
 2026-09-18 17:52:46 +08:00
andrewor14 62f9482583 [Tests] Delete deprecate torchao tests for v1 configs (#52956) 
Signed-off-by: andrewor14 <andrewor14@gmail.com>
 2026-09-16 23:11:27 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Rebecca LeeandCursor 6ca2b23e22 [ROCm] Expose kFp8DynamicTokenSym on AITER PTPC linears (#54248) 
Signed-off-by: Rebecca Lee <rebecca.lee@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-16 11:48:30 -05:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Colin ZandAndreas Karatzas c8d1cf077a [ROCm] [Bugfix] Enable Load and Inference of GLM-5.3-Flash Quark MXFP4 Checkpoint (#56176) 
Signed-off-by: Colin Zeng <Colin.Zeng@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-16 01:25:29 +00:00
Bugen Zhao cf1584f373 [Refactor] Normalize DeepSeek V4.1 model package naming (#56741) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-13 23:17:50 -07:00
JartX 986e2f870a [Refactor][ROCm] Migrate the RDNA3 W4A16 MoE to the oracle/experts pa… (#55522) 
Signed-off-by: JartX <sagformas@epdcenter.es>
 2026-09-12 21:20:26 +08:00