iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,331 Commits 3 Branches 0 Tags
6a95f62e2fbfebca86ec1e476e3b2e6fd8a574ab
Commit Graph
121 Commits
Author SHA1 Message Date
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
Wentao Ye 8e25bbca19 [Refactor] Remove dead code multiple places (#61843) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-16 15:01:07 +00:00
Harry Huangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> c75cb63cd2 [Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next (#56322) 
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-12 20:56:44 +08:00
Miłosz GrunwaldandKunshang Ji be0823357f [TEST][XPU][CI] Fix ci uva kernel test instability 2 (#60404) 
Signed-off-by: microslaw <milosz.grunwald@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-10 12:09:13 +08:00
Cheese CakeWilliam JoynerClaude Opus 5.5
 		956960f85f [Perf][MRV2] Avoid stream syncs in NonUvaBuffer host-to-device copies (#60215) 
Signed-off-by: cheesecake <farzanaman99@gmail.com>
Co-authored-by: William Joyner <133391209+joynerwk03@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 01:51:12 +00:00
Miłosz GrunwaldYour NameKunshang Ji
 		4e198cef83 [TEST][XPU][CI] Fix ci uva kernel test instability (#60041) 
Signed-off-by: Your Name <you@example.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-06 09:19:01 +08:00
Chaojun Zhang 7230dfea50 [XPU] Preserve non-contiguous strides when pinning CPU tensors for UVA view (#54874) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
 2026-09-29 09:43:07 +08:00
stefankoncarevicandAndreas Karatzas 6c300ddb9e [CI][Bugfix] Relax packed_qk_rope_ correctness test to one ULP (#59008) 
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-28 17:29:59 +00:00
Thien Tran 39d49539de [KimiViT][Perf] Fuse per-layer QK RoPE into one in-place kernel (#58651) 2026-09-27 18:18:06 -07:00
Samuel Kimandmgoin f55419c43c [Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxfp8 shape limits (#54223) 
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-25 02:54:11 +00:00
Thien TranKimiIsotr0py
 		16070ed7e9 [Minimax-M3][Perf] Use triton_mrope for vision tower + int64 offset fix for triton_mrope (#58526) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi <noreply@moonshot.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-24 18:30:32 -07:00
vllm-agentandkimi-no-na-wa 955bd6abef [CI][Bugfix] Extend groupwise rms_norm scale tolerance to CUDA (#58252) 
Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com>
Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com>
 2026-09-23 01:29:11 -07:00
Michał GanczarenkoandClaude 73c59d365b [XPU] Wire up SYCL apply_rotary_emb kernel in ApplyRotaryEmb (#55721) 
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
Signed-off-by: Michał Ganczarenko <michal.ganczarenko@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-22 20:43:24 -07:00
TresCursorShanshan ShenClaude Sonnet 5Andreas Karatzas
 		d110c2f19c [ROCm] Use silu_and_mul_with_clamp's torch._C op (#52052) 
Signed-off-by: Tres Popp <tres.popp@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-22 18:30:14 -05:00
Jinzhen LinCodexmgoin
 		7d99c2c4fd [Feature][Humming] Humming feature integration (#56685) 
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-20 08:11:01 -07:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
MattandAndreas Karatzas b3b13c1292 [ROCm][CI] Fix MLA RoPE fused-kernel tests for TheRock image (#57112) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-16 05:57:40 +00:00
Bugen Zhao cf1584f373 [Refactor] Normalize DeepSeek V4.1 model package naming (#56741) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-13 23:17:50 -07:00
Yongye ZhuJee Jee Liandyluo7OpenAI Codexzjy0516Claude Opus 5
 		e77daef89e [Model] Support DeepSeek-V4.1-Flash (#56214) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-11 02:11:19 -07:00
bjf-frzandNick Hill 2e0ee66cab [Perf] Use UVA-backed contents for MRV2 apply_write (#55819) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-10 07:27:16 -07:00
Harry MellorandClaude Opus 5 719284fe15 [Core][Model] Unify XD-RoPE into M-RoPE and derive the channel count (#56078) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-09 09:23:30 -07:00
Roderick Wuandmgoin 2c9d68f80b [Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering (#54809) 
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com>
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-08 12:45:12 -07:00
Roberto L. CastroandCodex 6ddbab03de [4/N][warmup][DSv4] Migrate common attention kernels (#50176) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-09-08 10:25:48 -04:00
Andreas Karatzas 096d8e8ce6 [ROCm][CI] Add MiniMax reduce RMS kernel coverage (#55057) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-03 04:07:59 -05:00
Divakar VermaandAndreas Karatzas c0adee9231 [ROCm][CI] Add ROCm misc ops and env tests (#53279) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-01 19:03:37 +00:00
Misha GoinandOpenAI Codex 82b7d49a6e [MoE] Generalize masked activation for padded layouts (#51217) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-01 11:34:13 -04:00
Jee Jee Li e2c8eeac40 [kernel] Fused embedding kernel (#53677) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-08-31 11:34:14 +00:00
canlahlahOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Yongye Zhu
 		06cccf8730 [Bugfix] Fix int32 token offset overflow in fused SiLU block quant (#53409) 
Signed-off-by: Anni <dranox2222@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-08-28 16:23:36 -04:00
Roy WangandClaude Fable 5 b74ac0bc61 [Bugfix] Allocate packed outputs in fused_q_kv_rmsnorm so q_b_proj keeps its low-latency GEMM path at decode (#53109) 
Signed-off-by: yasong.wang <yasong.wang@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-28 02:08:27 +00:00
Misha GoinandOpenAI Codex 5bbc58c0ad [Bugfix] Remove race in fused groupwise RMSNorm quantization (#54111) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-27 19:01:22 -07:00
Chauncey 7d5769b2d9 [Perf][DSv4] Use native CUDA SwiGLU clamp kernel for Humming MoE (throughput +1.40%) (#53685) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-08-27 13:52:45 -07:00
Shanshan Shen 32ad1400d7 [ROCm][Perf] Fuse SWA q/kv RMSNorm and q FP8 group quant for DeepSeek-V4 (#53540) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-08-27 09:37:54 -07:00
Divakar Verma 29b7c2f7d4 [ROCm][CI] aiter kernel ops - enable rope test (#52854) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-08-21 15:17:34 -05:00
Jee Jee Li 2687fec6ef Replicated embedding and norm fusion for DSV3 flat model (#48484) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-08-18 14:53:21 +08:00
Divakar Verma 49fb2ee348 [ROCm][CI] add Aiter ops tests (#52208) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-08-17 18:33:22 -05:00
bastefaniak 71b0da7c4e [Bugfix] Fix .../mrope.py::apply_interleaved_rope() when torch.compile is used in torch==2.13 (#52005) 
Signed-off-by: Bartosz Stefaniak <bstefaniak@nvidia.com>
 2026-08-13 22:08:40 +00:00
pmanczakandpmanczak 6b5bec7bed [Misc] Add and enable Triton kernel unit tests on XPU (#45694) 
Signed-off-by: pmanczak <pmanczak@users.noreply.github.com>
Signed-off-by: pmanczak <pawel.manczak@intel.com>
Co-authored-by: pmanczak <pmanczak@users.noreply.github.com>
 2026-08-07 02:53:03 -07:00
Rohan PotdarandClaude Opus 4.8 38ebd97bca [ROCm] Relax MLA rope+cache test tolerances for bf16 (#51083) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-05 17:50:52 -05:00
Andreas Karatzas bdc98bf506 [CI] Initialize fused gated RMSNorm weights (#50377) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 15:34:08 -04:00
Martin HickeyandHarry Mellor 38a267cdd5 [MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent (#49570) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-30 05:02:51 -07:00
Andreas KaratzasandOpenAI Codex 0c64be8873 [Test][ROCm] Account for gfx950 FP8 RMSNorm rounding (#49839) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-30 18:46:37 +08:00
yinfengLiu f1e8fd27ae [ROCm] Add AITER FP8 ViT encoder attention (#49937) 
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
 2026-07-30 06:22:10 +00:00
Wentao Ye 82642d7d6c [Perf] RMSNorm uncontiguous support, 1.2~3.1x kernel performance improvement (#49750) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-07-29 13:45:38 -07:00
+12
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodex
 		7c6729b769 [Model] Add Kimi K3 support: model files and kernels [1/N] (#50089) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
 2026-07-29 14:10:58 +08:00
Andreas Karatzas 05a0814863 [ROCm] Fix and optimize GPT-J-style MRoPE (#49906) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-28 10:50:10 -06:00
Andreas Karatzas e09900436c [CI][ROCm] Reduce kernel test runtime (#49915) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-27 06:44:21 +00:00
Wentao Ye 626c90b2d5 [Refactor] Move fla to third party (#48500) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-16 19:22:36 +01:00
Tanish MalekarandClaude Sonnet 4.6 e26264f3ef [Kernel] Implement CUDA kernel for ReLUSquaredActivation (relu^2) (#39058) 
Signed-off-by: Tanish Malekar <tanishmalekar32@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-07-12 19:18:03 -07:00
almayneandLi, Jiang <jiang1.li@intel.com> 89e99202f2 [CPU][Perf]Added tanh AOR for faster gelu activations. (#44639) 
Signed-off-by: Anna Mayne <anna.mayne@arm.com>
Signed-off-by: almayne <anna.mayne@arm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-06-30 23:24:40 -07:00