iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,342 Commits 3 Branches 0 Tags
sm86
Commit Graph
165 Commits
Author SHA1 Message Date
MOVZXandHinata 2066d3143f [Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack 
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
  rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
  select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
  quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
  requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
  fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
  dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
  mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
  cut deleted while keeping the model tree

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 17:10:56 +07:00
MOVZXandHinata 54a396586e [Mini-VLLM] Repair test suite after SM86 cut 
- Restore example connector, test helpers, and dropped patch tests
- Guard MLA metadata import in the example connector
- Point the PLE test at common.ops.ple after upstream #60021 move
- Cut tests for removed features: tools, DRY, epd proxy, STT batch,
  diffusion scheduler, INC/Quark PLE cases, mamba-2 mixer

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 01:06:30 +07:00
MOVZXandHinata 81074e94a3 [Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests 
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 00:20:48 +07:00
Nick Hill c059bee812 [MRV2] Switch CI tests still running on MRV1 unnecessarily (#61014) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-11 09:57:07 -07:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
Wentao Ye bf93e1f92c [Refactor] Remove dead tests utils (#58803) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-26 09:16:53 -04:00
stefankoncarevicandAndreas Karatzas 2d7dd0e5c1 [Bugfix][CI] Report subprocess test skips as skips, not passes (#58701) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 14:44:49 -05:00
Sheral KumarAndreas KaratzasCodexAndreas Karatzas
 		01f1f58f10 [ROCm][CI] Query HIP device memory for test GPU teardown waits. (#57450) 
Signed-off-by: Sheral Kumar <shekumar@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-20 15:20:39 -05:00
YiSheng5andKunshang Ji 1c3ef2ad4c [XPU][UT]Fix test_mnnvl_alltoall device and distributed backend (#57591) 
Signed-off-by: yisustc <yi.sheng@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-20 10:53:00 +08:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Simon Danielsson ae48466cf3 [Perf][ROCm] Add AITER custom AG/RS (DP only) (#48247) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
 2026-09-11 13:19:57 +08:00
Taneem Ibrahim e41011129b [CI] Avoid logging test server environment values (#54379) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-03 17:00:29 -04:00
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
Andreas KaratzasandOpenAI Codex cb58bb9c1e [CI] Harden RemoteVLLMServer GPU cleanup checks (#52282) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 14:17:17 -05:00
Misha GoinandOpenAI Codex bca7bea240 Remove VLLM_TEST_FORCE_FP8_MARLIN to replace with linear_backend/moe_backend (#52182) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-18 12:58:04 -04:00
Qiming ZhangandKunshang Ji f4b161d7fc [XPU][UT] Fix OOM and skip graph case (#49287) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-18 09:47:35 +08:00
Andreas KaratzasandOpenAI Codex 1ab2801dde [ROCm] Remove stale SDPA and skinny GEMM workarounds (#50907) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 14:19:16 -04:00
Michael GoinOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		fcdc7c2e9c [CI] Organize speculative decoding E2E tests by coverage (#50330) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-31 18:05:42 -07:00
Harry MellorandAndreas Karatzas ca0defa343 Make bare hugging_face imports forbidden (#49726) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-25 04:18:39 -07:00
Kevin H. LuuSimon MoOpenAI CodexOpenAI Codex
 		1dcbbd9cac [CI] Move compatible 1xL4 jobs to H200 35GB MIG (#43024) 
Signed-off-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-19 19:21:25 -07:00
BadrBasowid 740f379fae [ROCm][AITER] Directly Implement AITER Custom All-reduce in CudaCommunicator (#46065) 
Signed-off-by: BadrBasowid <badr.basowid@gmail.com>
 2026-07-06 12:16:32 +00:00
Chaojun Zhang 8cc242335d [XPU] Optimize XPU worker shutdown logic to prevent resource leak (#46433) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
 2026-06-30 15:27:21 +08:00
Andreas KaratzasandCodex 4eb227992a [ROCm][CI] Make memory sampling less racy in tests and sleep mode (#45490) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Codex <codex@example.invalid>
Co-authored-by: Codex <codex@example.invalid>
 2026-06-29 14:26:41 -07:00
Andreas KaratzasMatthew WongMatt
 		3f5a1e1733 [ROCm][CI] Expand basic correctness target suites (#46573) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Matt <156021403+mawong-amd@users.noreply.github.com>
 2026-06-25 12:18:57 +08:00
Andreas Karatzas 82a42234be [ROCm][CI] Defer AITER sampler import and isolate server test PYTHONPATH (#44823) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-10 08:56:11 +00:00
Andreas Karatzas d9ff7e4e9a [ROCm][CI] Stabilizing teardown and timeout of flaky tests to prevent rare OOMs (#44761) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-08 14:11:17 +08:00
Andreas Karatzas 445ded18c1 [ROCm][CI] Extend ROCm quick reduce coverage (#40990) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-26 21:57:13 +08:00
Wentao Ye 33d7cbe02c [Model Runner v2] Force v1 runner for tests (#43233) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-23 16:37:24 -07:00
Andreas Karatzas 76ea1d5d2f [ROCm][CI] Stabilize Granite tool-use and test URL construction (#43017) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-23 12:21:11 +08:00
haosdent 1fe3303983 [CI] De-flake renderers/test_hf.py::test_resolve_content_format_fallbacks[Qwen/Qwen-VL-string] (#43064) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-05-22 12:15:22 +08:00
David Zhenggemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Andreas KaratzasKunshang Ji
 		df2636a9d8 [Bugfix] Fix LOGITPROC_SOURCE_ENTRYPOINT test to use spawn-compatible dist-info registration for XPU/ROCm (#42040) 
Signed-off-by: dqzhengAP <dqzheng1996@gmail.com>
Signed-off-by: David Zheng <153074367+dzhengAP@users.noreply.github.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-05-09 12:32:04 +08:00
Lucas Wilkinson b1728c1e66 [Attention][Cleanup] Remove tree attention (#42121) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-05-08 18:36:19 -07:00
haosdent 160858cba4 [CI][Bugfix] Surface subprocess output in spawn_new_process_for_each_test (#41943) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-05-08 10:39:37 +02:00
haosdentandRoger Wang 5f6a02812a [CI][Bugfix] Fix failure CI step "PyTorch Fullgraph Smoke Test" (#41953) 
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-07 19:41:56 -07:00
David Zheng ee38750a75 [Bugfix] Fix spawn_new_process_for_each_test silently swallowing test failures (#41423) 
Signed-off-by: dqzhengAP <dqzheng1996@gmail.com>
 2026-05-06 11:17:15 -04:00
Andreas Karatzas 6ec9bbec38 [CI] Stabilize cpu offload compressed tensors test (#41102) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-04 05:22:42 +00:00
Andreas Karatzas 67058ca326 [CI] Clean up remote servers on pytest parent exit (#41570) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-04 03:11:22 +00:00
Nick Hill 18599bfdf2 [Ci][BugFix] Fix slow DP tests due to bad teardown logic (#41166) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-04-29 19:31:00 -04:00
Lucas Kabela b8401a9bf4 [Bugfix] Fix RMS norm + quant fusion on DeepGEMM UE8M0 path for B200 (#40552) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-04-22 22:04:42 +00:00
vllmellm 5f7fab881a [ROCm][FEAT] Integrate aiter gemm w8a8 ptpc (#33773) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2026-04-16 09:55:29 +08:00
Mark McLoughlin 8ad6ff0037 [Test] Fix @create_new_process_for_each_test("fork") in interactive shell pipeline (#29130) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-04-15 12:22:20 -04:00
Maral 2e9034c998 [W8A8 Block Linear Refactor][2/N] Remove W8A8Fp8BlockLinearOp and adopt Fp8 block linear kernel selections. (#33892) 
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: Maral <maralbahari.98@gmail.com>
 2026-04-09 08:50:39 +08:00
wliao2Harry MellorKunshang Ji
 		4dfad17ed1 replace cuda_device_count_stateless() to current_platform.device_count() (#37841) 
Signed-off-by: Liao, Wei <wei.liao@intel.com>
Signed-off-by: wliao2 <wei.liao@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-31 22:32:54 +08:00
Andreas Karatzas 4ed51308c8 [CI] Fix GPU memory leak when RemoteOpenAIServer fails to start in __init__ (#37230) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-17 09:08:08 -07:00
Andreas Karatzas d4c57863f7 [ROCm][CI] Fix engine teardown and text normalization to stabilize voxtral test (#37138) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-16 04:49:31 +00:00
Nick Hill b373b5102a [Tests] Shutdown test RemoteVLLMServer cleanly (#36950) 
Recent PR #33949 changed the teardown logic of the RemoteVLLMServer test utility class to
send SIGTERM to all vllm (sub)processes at once, which breaks the clean/coordinated
shutdown logic that assumes only the top-level process will receive a signal (for example
when running in a container that's shut down).

This caused a bunch of errors and stacktraces in some test logs, even though those tests
still pass. We should still attempt a normal shutdown and only kill other procs if they are
still running after a few seconds.

Example: tests/v1/distributed/test_external_lb_dp.py::test_external_lb_completion_streaming

Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-03-13 07:32:55 +00:00
Sage 802f306cd1 [Tests] Skip model weight download for render-only test server (#36813) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-03-12 06:24:42 +00:00
Andreas Karatzas 1e0f917b34 [ROCm][CI] Fix logprob divergence for TitanML/tiny-mixtral under AITER rms_norm (#36101) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-09 12:07:44 -05:00
Andreas Karatzas 807d680337 [ROCm][CI] Fix tool use test stability - disable skinny GEMM, prefix caching, eliminate batch variance (#35553) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-06 15:15:12 +08:00