iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,342 Commits 3 Branches 0 Tags
sm86
Commit Graph
192 Commits
Author SHA1 Message Date
MOVZXandHinata 61fbe0b865 [Mini-VLLM] Remove unreferenced definitions found by static scan 
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.

- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
  reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
  unused BailingLinearAttentionBackend, and the sparse flash-attn entry
  points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
  cpu_mla_extend, cpu_mla_get_scheduler_metadata,
  gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
  gptq_quantize_weights, get_and_maybe_dequant_weights,
  marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
  ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
  input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
  run_dp_sharded_vision_model, fast_topk, get_layer_index,
  is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
  selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
  split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
  is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
  create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
  vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
  neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.

Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 21:39:56 +07:00
MOVZXandHinata 5f3c7bd8ab [Mini-VLLM] Remove the ROCm device path, debug routes and arch-gated tests 
- Delete csrc/rocm, hipify, the _rocm_C extension and the setup.py rocm
  target path; drop the rocm requirement files
- Delete the aiter, rdna3, skinny-gemm and ROCm fused-MoE kernel modules
  and retarget the linear kernel registries away from them
- Reduce platforms/rocm.py to the GCN architecture predicates that live
  code still imports, and delete the RocmPlatform class and plugin
- Drop the VLLM_SERVER_DEV_MODE debug surface: reset, collective_rpc,
  server_info and sleep routes plus the log_error_stack flag
- Delete lm-format-enforcer, lark and otel-semantic-conventions-ai pins;
  the live server process maps none of them
- Delete tests that only build architectures this fork no longer
  registers, and test directories left without any test file
- Restore the tests/v1/kv_connector mock helpers that scheduler tests
  import at module level

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 19:31:07 +07:00
MOVZXandHinata 2b544b7ba0 [Mini-VLLM] Cut dead subsystems across serving, connectors, parsers 
- Remove P/D transfer connectors: nixl, nixl_chat, mooncake x2, hf3fs,
  moriio, hisparse, example and hidden-states connectors
- Reduce weight_transfer to the base API used by model loading
- Remove kv_offload tiering, simple_kv_offload, ec cpu/example,
  snapshot command, engine snapshot server
- Remove transformers model fallback, dsv4 tokenizers, dead platforms
  cpu/tpu/xpu, flashmla third_party, dsa, megamoe/flydsl, dead attn
  ops, dead layers (short_conv, fused_embed_norm, indexer_topk,
  relu2 gemma_ar, lightning_attn, minimax_rms_norm), gemm_rs_ar
- Keep MoE LoRA op (imported eagerly by punica_gpu); drop fp8 variant
- Rebuild parser hubs around Qwen3 adapters only (36 dead adapters)
- Remove dead tests with the code they covered

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 02:19:03 +07:00
MOVZXandHinata 81074e94a3 [Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests 
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 00:20:48 +07:00
afriedriSimon DanielssonKevin H. LuuAndreas Karatzas
 		2067297b2f [ROCm][Perf] Use AITER tuned configs to select a4w4 gemm kernel instead of env var (#61278) 
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Signed-off-by: afriedri <afriedri@amd.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-17 04:00:28 -05:00
Woosuk KwonLainCodexClaude Opus 5.5vllm-agent
 		596e530675 [Kernel] Add router GEMM kernels for (5120, 256) and BF16 output to ll_bf16_gemm (#62042) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Lain <fusiyuan2000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-16 15:39:46 -07:00
zofia 07f3857fb9 [XPU][Linear][MXFP4] enable torch as mxfp4 linear backend on xpu (#51965) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Signed-off-by: zofia <110436990+zufangzhu@users.noreply.github.com>
 2026-10-16 17:07:05 +08:00
Juan VillamizarJaden MathiasjpvillamOmar MuhammadDanielClaude Opus 5Daniel Chan
 		e711723352 [ROCm] gfx1250 optimizations and model enablement (#58800) 
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: Juan Villamizar <jpvillam@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
 2026-10-15 14:51:57 -07:00
KaiandKunshang Ji de2b869ee9 [Bugfix][Kernel] Register int8 scaled_mm as a custom op (#57331) 
Signed-off-by: Kai <limkc.reply@gmail.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-15 09:27:49 +08:00
Misha GoinClaudeKevin H. Luu
 		984c242ede [Quantization] Support compressed-tensors MXFP4 MoE with FP8 activations via Mxfp4MoEMethod (#59815) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-14 13:33:35 -07:00
Jinzhen LinCodexMisha GoinKevin H. Luu
 		fe98d90a9d [Dependency] Update Humming and remove NVFP4 scale workaround (#61556) 
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-14 13:32:36 -07:00
zofia 4dc5add6ea [XPU] [Linear] [MXFP8] enable torch as mxfp8 linear backend (#51933) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Signed-off-by: zofia <110436990+zufangzhu@users.noreply.github.com>
 2026-10-14 00:48:38 -07:00
Robert EsclapezClaude Opus 4.7Douglas Lehr
 		e83222396f [ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches (#57199) 
Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-10-13 17:21:30 +00:00
Roberto L. CastroandClaude Opus 5.5 7fad7db330 [Perf][Feat] Add generic cuteDSL LL FP32 router (GEMM) (#44343) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 12:38:46 -04:00
Andrii SkliarAndrii SkliarLuke Alonso
 		6849ca696a [Quantization] Add ModelOpt IQ2_XS, IQ2_XXS, and Q8_0 support via b12x (#59604) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Luke Alonso <lalonso@gmail.com>
 2026-10-13 09:06:33 -07:00
Misha GoinandClaude d79b52b182 [Misc] Consolidate csrc shared headers and prune dead code (#60721) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-12 09:46:22 -07:00
MattClaude Opus 5.5Andreas Karatzas
 		e0317b28ce [Feature] Support batch invariance on ROCm (#52231) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-12 10:53:26 -05:00
Guancheng FuandKunshang Ji acec1b1d84 [Bugfix][XPU] Remove stale W4A8 weight after repacking (#57511) 
Signed-off-by: Guancheng Fu <guancheng.fu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-10 04:27:35 +00:00
Jakub ZakrzewskiCodexMisha Goin
 		19c0bed584 [Fusion] Support 3D activation quantization fusion and enable it for Pixtral (#59612) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-09 15:12:03 +08:00
Harry MellorandClaude Opus 5.5 21d9758459 Remove Python 3.10 support now that it is EOL (#60402) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 02:30:52 -07:00
00CC 5281e49908 [ROCm][Perf] Enable medium-skinny dispatch in RDNAHybridW4A16LinearKernel (#52619) 
Signed-off-by: tangzzycc <3081129260@qq.com>
 2026-10-07 13:21:43 +00:00
Jinzhen LinCodexmgoinClaude Opus 5.5
 		d1f3d8b870 [Quantization] Prefer Humming before Marlin backends on SM90 (#56997) 
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 12:31:35 -07:00
Matthias Gehre b267fe4881 [ROCm][Perf] W4A16: pad gfx11 weight and activation strides (#56301) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-10-06 08:22:33 -05:00
Stefano CastagnettaandThien Tran 55b80221ed [Perf][Qwen4Exp] Keep the HC up projection on the skinny GEMM path (#60027) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-05 20:09:36 +08:00
Mehmet Cagri 08e03df940 [Bugfix][ROCm] Use a zero default for masked scales in the MXFP8 GEMM (#59454) 
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com>
 2026-10-01 11:32:25 -05:00
Ganesh RCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e12291d733 [CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers (#54024) 
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 15:18:25 +08:00
Rukhaiya2004andLi, Jiang <jiang1.li@intel.com> 72e7874fa6 [Hardware][Power] Enable W4A16 (AWQ & GPTQ) quantization on POWER10 using VSX (#59149) 
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-30 19:44:48 +08:00
Rohan PotdarClaude Fable 5jpvillam
 		e440b75bb6 [ROCm] Bump torch 2.13, triton 3.8, torchaudio, torchvision (#50605) 
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: jpvillam <juan.villamizar@amd.com>
 2026-09-25 14:55:52 -05:00
Lingpeng JinandAndreas Karatzas 35f92d2abc [ROCm] Credit ROCm/aiter for the block32 GEMM's packed kernel and in-launch split-K (#58659) 
Signed-off-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 21:37:41 -07:00
Samuel Kimandmgoin f55419c43c [Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxfp8 shape limits (#54223) 
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-25 02:54:11 +00:00
Fangzhou AiandClaude Opus 5.5 3059155b47 [ROCm][Perf] MXFP8 GEMM on native 32x32 block scales for gfx950 (#58510) 
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 21:26:30 -05:00
HDCharles 9f07d023d0 [Quantization] Enable humming wNaM asymmetric quant (zero_point) with compressed-tensors (#46528) 2026-09-23 10:23:11 -04:00
Misha Goin 25965ee583 [Kernel] Remove AllSpark INT8 W8A16 GEMM backend (#58001) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-22 14:07:34 -07:00
Canlin GuoThien TranOpenAI CodexClaude Fable 5.1Jee Jee Li
 		f92b78f6ef [Kernel][DSV4.1] Fuse MXFP8 wo_b GEMM with sequence-parallel reduce-scatter (#57428) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-09-22 01:07:17 -07:00
yinfengLiuandShanshan Shen 2af05512cf [Kernel] Add opt-in load-time MXFP4 dequantization (#50814) 
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-22 14:51:12 +08:00
Jinzhen LinCodexmgoin
 		7d99c2c4fd [Feature][Humming] Humming feature integration (#56685) 
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-20 08:11:01 -07:00
Misha GoinandErinYin e5fce7b56b [Core][Kernel] Share persistent workspaces for Marlin and Humming (#57421) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: ErinYin <liang.yin@daocloud.io>
 2026-09-20 17:28:33 +08:00
siyuandIsotr0py 4cc15f2121 [Quantization] Let ModelOpt MXFP8 layers load pre-processed weights Purpose (#57316) 
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2026-09-19 09:16:39 +00:00
Yuwen ZhouandLi, Jiang <jiang1.li@intel.com> b346479065 [CPU] Add CPU FP8 W8A8 linear/MoE support (#49942) 
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-18 17:59:42 +08:00
meng-amd 6df2b1a8c6 [Quantization] Support native Quark W4A16 INT4/UINT4 exports in vLLM (#48606) 
Signed-off-by: HongWei Meng <HongWei.Meng@amd.com>
 2026-09-18 17:52:46 +08:00
rasmithandShanshan Shen acc2ed2a5f [ROCm][CI][The Rock 10] Fix (MI355) Quantized Models failure on The Rock 10 with Triton 3.8.x (#54849) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-17 15:20:35 -05:00
danisereb 2bdbbc8080 [BugFix] Fix is_supported of cutlass FP8 linear (selected and fails on A100) (#55884) 
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
 2026-09-16 12:31:43 -07:00
Shanshan Shen bc0f47cd03 [ROCm][Bugfix] Revert #56433 + #51692 to fix accuracy breakdown for DeepSeek-V4 (#57132) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-09-16 18:58:23 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Rebecca LeeandCursor 6ca2b23e22 [ROCm] Expose kFp8DynamicTokenSym on AITER PTPC linears (#54248) 
Signed-off-by: Rebecca Lee <rebecca.lee@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-16 11:48:30 -05:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Li, Jiang b68408043d [Bugfix][CPU] Add per-tensor FP8 W8A16 kernel to fix Ministral crash on CPU (#56985) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-09-16 15:49:17 +08:00
JohnQinAMDandClaude Opus 5 ef5f7cd119 [ROCm][DSv4.1][Perf] Dequantize the MXFP8 weight once when dot_scaled cannot be used (#56560) 
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-15 09:07:26 +08:00
Netanel Haber d4ee7fe7a9 [Quantization] Select linear backends per quantization (#51204) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2026-09-14 16:24:18 -04:00
Matthias Gehre dc2e8f1157 [ROCm][Perf] W4A16: keep skinny GEMM zero-points packed 4-bit (#54965) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-09-14 17:40:25 +08:00