iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,342 Commits 3 Branches 0 Tags
sm86
Commit Graph
43 Commits
Author SHA1 Message Date
MOVZXandHinata 2066d3143f [Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack 
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
  rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
  select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
  quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
  requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
  fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
  dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
  mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
  cut deleted while keeping the model tree

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 17:10:56 +07:00
MOVZXandHinata 81074e94a3 [Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests 
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 00:20:48 +07:00
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
wang.yuqi 669f85e686 [Misc] Break chat_utils into more maintainable modules. (#60972) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-10-13 15:08:07 +00:00
Subhash Polisetti f425a21208 [Bugfix][Frontend] Support scalar priority and wire priority into LLM.chat() (#57243) 
Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
 2026-10-11 10:03:33 +00:00
Kay YanandCodex c41b2639e2 [Bugfix][Responses API] Complete cache-write token accounting (#60973) 
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-10 09:08:36 +00:00
wang.yuqi c1692c85f1 [Misc] mv chat_utils.py to renderers. (#60522) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-10-10 06:50:34 +00:00
Misha GoinandClaude Opus 5.5 29e4d2d42e [Frontend][Build] Add vllm download-kernels for FlashInfer precompiled kernels (#58765) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 21:22:35 -07:00
Taewoong KimandCyrus Leung 75e3b17c05 [Frontend] Add cache salt support to generative scoring (#60841) 
Signed-off-by: Taewoong Kim <ktw2172@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-09 16:07:50 +00:00
Taewoong KimandChauncey 18c19eaf23 [Frontend] Add cache salt and tracing to structured decisions (#60786) 
Signed-off-by: Taewoong Kim <ktw2172@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-10-09 08:53:50 +00:00
wang.yuqi 459764104d [Frontend] Move offline_utils.py to entrypoints/common. (#58052) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-10-08 09:56:05 +00:00
wang.yuqiaoshen02Claude Opus 5.5
 		4187ade7b8 [Frontend] Consolidate rl entrypoints. (#57849) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 21:37:11 -07:00
Maroon AyoubClaudeRobert ShawRobert Shaw
 		5d9214d6b2 [Bugfix][Frontend] Strip x-anthropic-billing-header billing header from /v1/chatcompletions (#59419) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-01 13:44:40 +00:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
Ronaldaoshen02Claude Opus 5.5aoshen02
 		713ec07655 [Frontend][RL] Track HTTP weight operation outcomes and concurrency (#55781) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 09:19:43 +00:00
Martin HickeyHarry MellorClaude Opus 5.5
 		5dd693122c [MyPy][2/N] Fix mypy errors in small tests/ dirs (#51043) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 21:27:45 +01:00
aoshen02aoshen02Claude Opus 5.5
 		cd94b21264 [Bugfix][Frontend] Return 400 for malformed RL dev route bodies (#59236) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 16:46:56 +00:00
yzong-rhandFlora Feng 5a6ccc5892 [Bugfix] Add Responses cache_write_tokens for API compat and CC parity (#57222) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-16 21:52:34 +00:00
Flora Feng e0705731be [Refactor] Remove unused interface methods (#57194) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-16 21:00:48 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
wang.yuqi 67111973ee [Frontend] Move grpc_server to launchers. (#56746) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-09-14 05:03:08 +00:00
Jiatai WangandOpenAI Codex a36dfc93cb [Bugfix][Multimodal] Restore cached audio inputs with UUIDs (#56310) 
Signed-off-by: JiataiWang <wangjiatai@proton.me>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-10 19:20:03 +00:00
Tianyu Guo 479611c0c6 [Frontend][EPD] Use JSON arrays for multimodal metadata (#56090) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-09-10 05:04:14 +00:00
Adababyandshaolila 5133e1d285 [Frontend] Migrate input-validation errors to VLLMValidationError (harmony/mistral/chat_utils/params) (#55735) 
Signed-off-by: shaolila <shaolila@buaa.edu.cn>
Co-authored-by: shaolila <shaolila@buaa.edu.cn>
 2026-09-08 06:45:40 +00:00
Clinton ThomasLucas BourtouleCyrus Leung
 		e962733e08 [Security] Validate cache salts before they reach LMCache (#51444) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-05 03:56:06 +00:00
Taneem Ibrahim e41011129b [CI] Avoid logging test server environment values (#54379) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-03 17:00:29 -04:00
Cyrus Leung 754d5e1f65 [CI/Build] Fix entrypoints coverage (#54750) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-01 03:33:02 -07:00
waizuichougou 810bc3250c [Frontend][Performance] Resolve async media across modalities concurrently (#54537) 
Signed-off-by: waizuichougou <2082431897@qq.com>
 2026-08-31 10:50:17 +00:00
Isotr0py 555ea65e8c [Frontend] Add video embeds input support (#54242) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-31 03:20:47 +00:00
Hung Hoang (黄兴） acd0af90e8 [Bugfix] Raise clear error on interleaved multimodal placeholder overcount (#53999) 
Signed-off-by: hungh <hungh@nvidia.com>
 2026-08-27 09:49:40 +00:00
Venkat Balaji SandBen Browning 161ffd37d2 [Bugfix] Guard tool call argument JSON parsing in chat message postprocessing (#48922) 
Signed-off-by: VBS2004 <venkatbalaji2004@gmail.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-08-26 20:26:00 +00:00
Andreas KaratzasandOpenAI Codex cb58bb9c1e [CI] Harden RemoteVLLMServer GPU cleanup checks (#52282) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 14:17:17 -05:00
wang.yuqi eac636a7fa [Frontend] Move api_server.py out openai folder (#52131) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-19 09:43:46 +00:00
Kaif Kohari a6a2a93f9b [Bugfix][Frontend] Guard remaining before-validators against non-object JSON bodies (#52528) 
Signed-off-by: Kaif <kaifkohari10@gmail.com>
 2026-08-17 02:27:59 +00:00
Recoordinateandlatent-9 0b5b49dde8 [Bugfix][Frontend] Raise VLLMValidationError for user-facing errors in chat_utils.py (#50491) 
Signed-off-by: latent-9 <296084221+latent-9@users.noreply.github.com>
Co-authored-by: latent-9 <296084221+latent-9@users.noreply.github.com>
 2026-07-31 11:54:43 +00:00
Andreas Karatzas e04a30a77c [Frontend] Lazily initialize chat media connectors (#49914) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 14:56:37 +08:00
Zach Zhu 58f9659397 [Frontend][Core] Standardize request error handling with VLLMError hierarchy (#49665) 
Signed-off-by: Zach Zhu <zzqshu@126.com>
 2026-07-29 04:20:40 +00:00
Harry Mellor 89f6aa3a9e [KV Offload][CI] Fall back to buffered I/O without O_DIRECT; fix flaky api-server test (#49734) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-24 13:56:57 -07:00
ErenAta16 0067311536 fix(entrypoints): stop resolve_items leaking in-flight media fetch tasks on partial failure (#48333) 
Signed-off-by: ErenAta16 <erena6466@gmail.com>
 2026-07-11 15:42:08 +00:00
yzong-rh 0cd6f767e3 [Bugfix][Frontend][gpt-oss] Recover raw tail when Harmony parser ends non-terminal (#47379) 2026-07-04 10:46:24 -04:00
yzong-rh b1190d03cc [Refactor][GPT-OSS] Harmony Responses API Refactor to use HarmonyParser (#47185) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-06-30 19:23:20 -04:00
wang.yuqi 996222f4bf [CI] Reorganize entrypoints CI (#44947) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-06-09 00:46:11 -07:00