MOVZX and Hinata
5f3c7bd8ab
[Mini-VLLM] Remove the ROCm device path, debug routes and arch-gated tests
...
- Delete csrc/rocm, hipify, the _rocm_C extension and the setup.py rocm
target path; drop the rocm requirement files
- Delete the aiter, rdna3, skinny-gemm and ROCm fused-MoE kernel modules
and retarget the linear kernel registries away from them
- Reduce platforms/rocm.py to the GCN architecture predicates that live
code still imports, and delete the RocmPlatform class and plugin
- Drop the VLLM_SERVER_DEV_MODE debug surface: reset, collective_rpc,
server_info and sleep routes plus the log_error_stack flag
- Delete lm-format-enforcer, lark and otel-semantic-conventions-ai pins;
the live server process maps none of them
- Delete tests that only build architectures this fork no longer
registers, and test directories left without any test file
- Restore the tests/v1/kv_connector mock helpers that scheduler tests
import at module level
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 19:31:07 +07:00
MOVZX and Hinata
bd70a61c20
[Mini-VLLM] Remove orphan subsystems: snapshot, CPU workers, dead decoders
...
- Delete the snapshot package, CLI command, and its launch tests
- Delete CPU workers, cpu_communicator, and the hw_agnostic layer package
- Delete the EC CPU connector, keeping the EC base glue intact
- Delete dead composite/triton_flashinfer backends and third_party/flashmla
- Delete dead video decoders, video-bound tests, and orphan utils
- Remove the deep_gemm_mega_moe option and dead allreduce fusion pair
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 09:27:06 +07:00
MOVZX and Hinata
8dfaffb700
[Bugfix] Restore xpu_communicator for CUDA graph capture
...
parallel_state.graph_capture() imports xpu_communicator on CUDA too.
The SM86 cut removed the file and capture crashed at startup.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:33:19 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
2b85d9cb65
[Perf][Bugfix] Prefer MNNVL for single-node FlashInfer allreduce again ( #61989 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-17 02:55:28 -07:00
Woosuk Kwon and Claude Opus 5.5
e57bb4ace7
[Core] Remove the legacy Ray compiled-graph executor ( #61712 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 18:13:27 -07:00
Misha Goin and Claude
d79b52b182
[Misc] Consolidate csrc shared headers and prune dead code ( #60721 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 09:46:22 -07:00
e0317b28ce
[Feature] Support batch invariance on ROCm ( #52231 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-12 10:53:26 -05:00
AMD-yanfeiwang and aoshen02
e3bbaf2369
[ROCm][DSv4] Fuse the shared expert into AITER MegaMoE ( #61004 )
...
Signed-off-by: Yanfei Wang <yanfei.wang@amd.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-12 15:53:16 +08:00
74f4404f58
[Bugfix][Distributed] Own SHM out-of-band buffers before releasing read slots ( #61000 )
...
Signed-off-by: Hubert Marek <163992334+hubert-marek@users.noreply.github.com >
Signed-off-by: Ao Shen <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Ao Shen <aoshen@inferact.ai >
2026-10-12 01:44:12 -04:00
aoshen02 and Claude Opus 5.5
1dfe9fbdae
[Core] Release FlashInfer allreduce workspace with --enable-nccl-comm-suspend ( #59360 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 02:26:40 +00:00
04b02fc8a5
[Core] shm tensor arena for efficient cpu->gpu worker broadcast ( #51207 )
...
Signed-off-by: Bolin Sun <bolins@nvidia.com >
Signed-off-by: <bolins@nvidia.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-10-09 16:13:24 -07:00
Misha Goin and Claude
3766a67a32
[WideEP] Explain how to fix a too-old NCCL for DeepEP v2 ( #60449 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-09 12:34:58 -07:00
a6f43e57d6
[ROCm] Enable the cuMem CUDA-graph pool offload on ROCm ( #59523 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: indianspeedster <cspandey016@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: indianspeedster <cspandey016@gmail.com >
2026-10-09 08:29:55 +00:00
Aarushi Jain and Andreas Karatzas
9bbc581560
[Bugfix] Close MessageQueue zmq resources via finalizer to avoid exit ( #60472 )
...
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-08 13:53:00 -05:00
Micah Williamson and Fangzhou Ai
f6e3944e31
[ROCm][DSv4] AITER MegaMoEV2 Integration For DeepSeek V4 ( #59685 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
2026-10-07 09:35:53 -05:00
Samuel Nordmann and Misha Goin
4a99ca40d6
[Perf][MoE] Eliminate staging from NCCL symmetric reduce-scatter ( #49194 )
...
Signed-off-by: samnordmann <snordmann@nvidia.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-06 07:37:01 -07:00
Elvir Crnčević
ce49174247
[Bugfix] Fix FlashInfer all_reduce backend selection ( #56891 )
2026-10-03 22:18:58 -04:00
aoshen02 and Claude Opus 5.5
5e56e9af2a
[Feature] Release the CUDA graph pool on sleep ( #59160 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-04 01:13:10 +00:00
Ayaan Nadamal
84593955fa
[Perf][MoE] Support fp8 combine in FlashInfer one-sided MoE all2all ( #57995 )
...
Signed-off-by: Ayaan Nadamal <anadamal@nvidia.com >
2026-10-02 16:29:16 -04:00
Julien Debache
111f71a6db
[feat] FlashInfer CuteDSL MegaMoE integration ( #54049 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
2026-10-02 15:43:33 -04:00
tenotek and Claude Fable 5.1
443dccd3c9
[Bugfix][Multimodal] Fix reference counting of the SHM processor cache ( #59695 )
...
Signed-off-by: tenotek <50898317+hdq66666@users.noreply.github.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-02 22:14:13 +08:00
Flora Feng
785b990116
[Distributed] Enable custom all-reduce under VLLM_BATCH_INVARIANT ( #58623 )
...
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-10-02 02:25:48 +00:00
3d5625cc3c
[Security] Authenticate shared-memory multimodal cache handles ( #59357 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-30 05:16:26 -07:00
Tyler Michael Smith and Codex
50239dfc4d
[WideEP] Change DeepEPv2 to auto select hybrid mode by default ( #57991 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-29 18:36:30 -04:00
764413559a
[Bugfix] Don't sync-police or retry FlashInfer all-reduce workspace creation in eager mode ( #58498 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-28 12:14:21 -07:00
Mark McLoughlin and AI Assistant
45d56c0baf
[Core] Model console logging as CLI configuration ( #57205 )
...
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: AI Assistant <noreply@openai.com >
2026-09-25 11:36:59 +01:00
Yongye Zhu and Summer Yang
442bd00d9f
[Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 ( #55072 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
2026-09-25 03:27:21 +00:00
e6dc16cebd
[PCP] Support prefill context parallelism with data parallelism ( #57075 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 22:32:36 -04:00
Tony Lin and Flora Feng
e4340e41c9
[Feat][XPU] VLLM_BATCH_INVARIANT support for Dense/MoE models ( #55881 )
...
Signed-off-by: Tony Lin <tony.lin@intel.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-22 11:06:54 -04:00
YiSheng5 and Kunshang Ji
17e50b9b76
[XPU][UT]Bugfix when the process can't see all the world_size meet accuracy issue. ( #57779 )
...
Signed-off-by: yisheng <yi.sheng@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-21 06:50:25 +08:00
Kevin Li and kevinli
911548247a
[Distributed][MoonEP] BF16 integration of MoonEP balanced EP backend ( #52101 )
...
Signed-off-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com >
Co-authored-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com >
2026-09-18 10:34:01 -04:00
40b40d1d39
[Bugfix][KV Offload] Register the offload region in chunks ( #51081 )
...
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Colton Ottley <coltonottley@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-09-17 13:19:56 +03:00
Robert Shaw and Robert Shaw
c38f92aae7
[A2A] Allow EPLB + SharedExpert Overlap + DeepEPv2 ( #57236 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
2026-09-17 04:02:39 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
0f8fa53acf
[Bugfix][KV Offload] Check cgroup memory before SHM allocation ( #54014 )
...
Signed-off-by: Alex <jihui.huang@daocloud.io >
Signed-off-by: AlexHuang <alex.tech.lab@outlook.com >
Signed-off-by: Alex <alex.tech.lab@outlook.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
2026-09-16 09:17:50 +03:00
Matt and Claude Opus 4.8
72d4d83157
[ROCm][Bugfix] Fix elastic EP scaling deadlock ( #56610 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-09-12 23:09:19 +00:00
Itay Alroy
658c8131c7
[Elastic EP] Reuse CUDA graphs across reconfiguration ( #54985 )
2026-09-12 10:26:39 -04:00
Nicolò Lucchesi and Harry Mellor
8c1d1c2974
[Misc] Log FlashInfer allreduce workspace init failure as error ( #55127 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-11 09:35:56 +00:00
Simon Danielsson
ae48466cf3
[Perf][ROCm] Add AITER custom AG/RS (DP only) ( #48247 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-09-11 13:19:57 +08:00
Harry Huang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
3116c5d06b
[Qwen4Exp] Support UVA PLE-offload and Engram tensor parallelism ( #54371 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-09 07:32:34 -07:00
Taoyu Zhu and Shanshan Shen
ce6c241ddc
[ROCm][Perf][M3] Fused allreduce+GemmaRMSNorm fast path ( #54787 )
...
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-08 07:21:49 +00:00
dharak-cohere and Cursor
d0e695a91b
[Distributed] Support pre-shared ncclUniqueId rendezvous for weight transfer ( #53784 )
...
Signed-off-by: Dharak Kharod <dharak.kharod@cohere.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-01 14:55:57 +08:00
4ac452ad98
[Core] Release NCCL communicator memory in sleep mode ( #51485 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-09-01 14:31:16 +08:00
446c769482
[Distributed] Add opt-in FlashInfer PCIe IPC all-reduce backend ( #53576 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-01 03:34:56 +00:00
+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00
99013d77d3
[Bugfix][Distributed] Gate cross-node MNNVL custom all-reduce by group capability ( #53253 )
...
Signed-off-by: JulianZJN <148066842+JulianZJN@users.noreply.github.com >
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
2026-08-29 08:49:24 +08:00
Xuan-1998 and Vladimir Aerov
026d5af7f4
[Bugfix] Fix ncclCommQueryProperties heap overflow with NCCL >= 2.31 ( #53008 )
...
Signed-off-by: Xuan Jiang <xuanj@amazon.com >
Co-authored-by: Vladimir Aerov <vaerov@amazon.com >
2026-08-29 00:25:00 +00:00
Khushali Desai
c7b0467c25
buffer size insuffient Dspark sd for FlashInfer MNNVL allreduce ( #50932 )
2026-08-27 11:01:08 -04:00