iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,331 Commits 3 Branches 0 Tags
6a95f62e2fbfebca86ec1e476e3b2e6fd8a574ab
Commit Graph
91 Commits
Author SHA1 Message Date
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
Misha GoinandClaude Opus 5.5 1c4c319792 [Frontend] Advertise --enable-x/--disable-x for boolean CLI flags (#61479) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 01:08:02 -07:00
Sherif Waly be4ce2e505 [Bugfix] Share PCP-aware local GPU indexing for NUMA binding (#61237) 2026-10-14 10:54:03 +02:00
Lanze LiuandNing Xie f4b8d43a68 [Bugfix] Keep force-killing the process tree when a child exits mid-walk (#59984) 
Signed-off-by: Lanze Liu <lanzetech@gmail.com>
Co-authored-by: Ning Xie <andy.xning@gmail.com>
 2026-10-13 20:33:18 -07:00
Haokai MaandClaude 0751f5272f [Bugfix] Scope deprecated CLI argument warnings to the parser that defines them (#59917) 
Signed-off-by: Tyler Ma <158780265+TylerMa-debugg@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-08 15:29:04 +00:00
Muhammad Usman MateenandMisha Goin e73895a5df [Bugfix] Make suppress_stdout redirect fd 1 instead of sys.stdout.fileno() (#59336) 
Signed-off-by: usmanmateen <usmanmateen255@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 09:52:50 +00:00
Bugen ZhaoOpenAI CodexClaude Opus 5.5
 		f226af0e11 [Bugfix] Eliminate frontend ZMQ port TOCTOU with inherited listeners (#54113) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 06:25:27 +00:00
aoshen02andClaude Opus 5.5 d09ff77637 [Bugfix] Give each data-parallel engine its own global RNG streams (#59788) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 17:38:18 +00:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
Martin HickeyHarry MellorClaude Opus 5.5
 		5dd693122c [MyPy][2/N] Fix mypy errors in small tests/ dirs (#51043) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 21:27:45 +01:00
Bowen WangandClaude Opus 5.5 1fde8ff7da [Bugfix] Detect the CUDA toolkit the way FlashInfer does in has_flashinfer() (#58360) 
Signed-off-by: Bowen Wang <bowen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 04:14:49 +00:00
stefankoncarevicandAndreas Karatzas 2d7dd0e5c1 [Bugfix][CI] Report subprocess test skips as skips, not passes (#58701) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 14:44:49 -05:00
Harry MellorandClaude Opus 5 0ef8366ba3 [Frontend] Only show the summary line of config docstrings in --help (#57357) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-17 12:53:11 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
AlexHuangOpenAI CodexItay Etelis
 		0f8fa53acf [Bugfix][KV Offload] Check cgroup memory before SHM allocation (#54014) 
Signed-off-by: Alex <jihui.huang@daocloud.io>
Signed-off-by: AlexHuang <alex.tech.lab@outlook.com>
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
 2026-09-16 09:17:50 +03:00
Kevin H. LuuSherlockmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		af1c01499b [Bugfix] Make GPU sync checks safe under torch.compile (#56904) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Sherlock <sherlock@raft.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-15 21:37:08 -07:00
Nick Hill f2aad6aa70 [MRV2] Buffer util simplifications (#56888) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-15 20:36:51 +00:00
Tianyu Guo 479611c0c6 [Frontend][EPD] Use JSON arrays for multimodal metadata (#56090) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-09-10 05:04:14 +00:00
Nick Hill 4aadb0f14d [Core] Enhance cpu<->gpu sync checking to include paged async copies (#53491) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-08 17:26:21 -07:00
Yusuke OdaandChauncey 798b557e06 [Frontend] Add site-packages support for reasoning/tool parser plugins (#45241) 
Signed-off-by: odashi <yusuke.oda@predicate.jp>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-02 07:41:13 +00:00
fxmarty-amdandAndreas Karatzas d1c15e589d [CI] Speed up quantization test group (#53291) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-01 13:15:24 -05:00
Harry MellorXianbao QIANClaude Opus 5
 		7ca336929c [Model] Remove ten deprecated model architectures (#53608) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Xianbao QIAN <xianbao.qian@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-24 17:06:00 -07:00
Prudhvi Vuda de216b6e64 [Bugfix] Skip MM processor cache inserts larger than capacity (#53016) 
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
 2026-08-20 13:42:50 +00:00
aoshen02OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7553aac77b [Frontend] Add routed-experts prompt offset (#51906) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 08:09:00 -07:00
Stellar鱼 3fa89225bc [CI/Build] Use file rendezvous for local distributed tests (#51652) 
Signed-off-by: yu-xin-c <2182712990@qq.com>
 2026-08-12 10:48:12 +08:00
Raj Vijay FirkeandHarry Mellor 3dafaef027 [Bugfix][Core] Emit --no-{key} for false BooleanOptionalAction flags in YAML config (#51573) 
Signed-off-by: Raj Firke <79653531+rajfirke@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-10 11:05:15 +00:00
Nick Hill fbff187d59 [Core] Make the GPU sync check thread-local and fix its suppressors (#51455) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-08 17:13:40 -07:00
Wei-Ming Chen 1c94e8dc7d Add NVFP4 KV 4-over-6 scale search (#45187) 
Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com>
 2026-08-07 13:22:27 -07:00
aoshen02Chenglun Huferkans-amiraoshen02Claude Fable 5
 		448344c0e2 [Bugfix] Fix get_open_port() livelock on DP-reserved ports and cover get_open_ports_list (#50965) 
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Chenglun Hu <chenglunhu@gmail.com>
Co-authored-by: ferkans-amir <amir.rezaei@tu-berlin.de>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-07 22:39:33 +08:00
Nick HillandClaude 7635a9002b [Core] Explicitly manage torch CPU threads in workers (#49919) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-04 10:06:38 -07:00
Matthew BonannihaosdentClaude Opus 4.7Justin Wood
 		58b2012aa2 [copy of #45208] CuMem slept-L1 fragmentation accounting (#49208) 
Signed-off-by: haosdent <haosdent@gmail.com>
Signed-off-by: Justin Wood <justin.m.wood@me.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: haosdent <haosdent@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Justin Wood <jwood@me.com>
 2026-07-20 23:08:11 +00:00
Ting SUNandNick Hill 8b91cd5b20 [Bugfix][Core] Close underlying iterator in merge_async_iterators single-iterator fast path (#44726) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-07 05:13:41 -07:00
Kunshang Ji fb42e5219e [Platform] Replace torch.cuda.mem_get_info with torch.accelerator.get_memory_info (#44825) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-06-30 14:39:52 +08:00
Nick Hill 4e07ca2c92 [Core] Add VLLM_GPU_SYNC_CHECK env var (#44800) 2026-06-26 08:24:33 -07:00
Ting SUNandShengqi Chen c45f681932 [Bugfix][Core] Fall back when numactl --membind is blocked in constrained containers (#45438) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-06-16 09:49:56 -07:00
Nick Hill 6f8b40a23f [BugFix][CI] Fix added _has_module tests (#44248) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-06-01 14:23:12 -07:00
Jeffrey WangesmeetuNick Hill
 		29d69332aa [BugFix] Fix _has_module to verify native deps via trial import (#44035) 
Signed-off-by: esmeetu <jasonailu87@gmail.com>
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: esmeetu <jasonailu87@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-05-31 22:06:33 -07:00
Vadim GimpelsonandCursor bf18d7e0b4 [Misc][NUMA] Auto-bind to PCT priority cores on DGX B300 + widen EngineCore across shard NUMA nodes (#43270) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Cursor <noreply@cursor.com>
 2026-05-29 10:07:44 +08:00
Divakar Verma 35d0141a0b [ROCm][CI] add warmup to mem_util test before measurement (#43236) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-05-22 03:17:54 +00:00
David Zheng ee38750a75 [Bugfix] Fix spawn_new_process_for_each_test silently swallowing test failures (#41423) 
Signed-off-by: dqzhengAP <dqzheng1996@gmail.com>
 2026-05-06 11:17:15 -04:00
Nithin ChalapathiandCyrus Leung 80b18230e0 [Frontend] Add multimodal support to /inference/v1/generate endpoint (#38405) 
Signed-off-by: Nithin Chalapathi <nithin.ch10@gmail.com>
Signed-off-by: Nithin Chalapathi <nithinc@berkeley.edu>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-04-17 20:31:56 -07:00
haosdentandDarkLight1337 995e9a209e [Bugfix] Use is_integrated to detect UMA GPUs for memory reporting (#35356) 
Signed-off-by: haosdent <haosdent@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-04-13 11:07:40 -07:00
Shengqi ChenJason LiRoger Wang
 		75e01a39a1 [Feature] NUMA binding support for GPU workers (#38635) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: Jason Li <jasonlizhengjian@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-04-08 09:55:24 -07:00
Jeffrey Wang de5e6c44c6 [Feat][Executor] Introduce RayExecutorV2 (#36836) 
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
 2026-04-01 14:34:29 -07:00
Wentao Ye bf5eec638d [Refactor] Remove unused utils (#38153) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-26 17:08:19 +08:00
Kunshang Ji 747b068136 [Hardware] Replace memory related torch.cuda APIs (#37031) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-03-16 10:24:48 +00:00
Walter Beller-Morales 43e77e59ab [BugFix] avoid infinite loop with VLLM_PORT and get_open_ports_list (#36191) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
 2026-03-05 22:15:29 -08:00
Cyrus Leung 83449a5ff0 [Refactor] Clean up pooling serial utils (#33665) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-03 10:29:18 +00:00
Michael Goin ca1969186d [UX] Enable nested configs in config yaml files (#33193) 2026-01-28 16:54:25 -05:00