iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,331 Commits 3 Branches 0 Tags
6a95f62e2fbfebca86ec1e476e3b2e6fd8a574ab
Commit Graph
288 Commits
Author SHA1 Message Date
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
Rohan PotdarClaude Opus 5.5Micah WilliamsonKevin H. LuuAndreas Karatzas
 		b2456c622b [ROCm][CI] Make ROCm 10.0 the default image; keep ROCm 7.2 as -rocm72 (#58761) 
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-17 21:43:59 -05:00
maithilijoshi20andCyrus Leung 92dc3dd393 [Bugfix] Honor OPT token embedding width for prompt embeddings (#61736) 
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-16 04:41:50 +00:00
Andrey TalmanAndrey TalmanClaude Opus 5.5
 		30e79c3a76 [Core] Update PyTorch to 2.14.0, torchvision to 0.29.0, triton to 3.8.0 (release channel) (#52183) 
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com>
Signed-off-by: Andrey Talman <atalman@fb.com>
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 00:17:59 -07:00
Yuvraj Singh BhadoriaYuvrajSinghBhadoria2Cyrus LeungClaude Opus 4.8
 		1a3a7a8f01 [Bugfix] Treat max_tokens as output upper bound, not input-space reservation (#57035) 
Signed-off-by: YuvrajSinghBhadoria2 <245782664+YuvrajSinghBhadoria2@users.noreply.github.com>
Signed-off-by: YuvrajSinghBhadoria2 <yuvrajsinghbhado2030@gmail.com>
Signed-off-by: Yuvraj Singh Bhadoria <yuvrajsinghbhado2030@gmail.com>
Co-authored-by: YuvrajSinghBhadoria2 <245782664+YuvrajSinghBhadoria2@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-10-10 01:15:22 +08:00
Rémi Delhaye a1cd0ec650 [Model] Add zerank-2 reranker (#60415) 
Signed-off-by: rdlhio <contact@rdlh.io>
 2026-10-09 03:01:51 +00:00
Harry MellorandClaude Opus 5.5 5d20979fc9 [Misc] Remove code paths for Transformers < 5.16.1 (#59762) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 11:17:24 +00:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
TaimysCodexLuyi XiaoTaimysHarry MellorClaude Opus 5.5
 		5225f1bacb [MyPy][3/N] Fix MyPy errors in test groups (part 3) (#55939) 
Signed-off-by: Taimys <ffy2281693949@gmail.com>
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Co-authored-by: Taimys <ffy2281693949@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 11:06:07 +00:00
qizixiOpenAI CodexClaude Opus 5.5Nick Hill
 		a4eb3f25d6 [Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer (#57263) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-26 17:01:17 +08:00
mkunreddMahesh KunreddiClaudeAndreas KaratzasCodexAndreas Karatzas
 		4013f3ad6c [ROCm] Fix CI runtime and tests for MI355 DPX (#58244) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Mahesh Kunreddi <mahesh.kunreddi@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 23:15:11 -05:00
Harry MellorandClaude Opus 5.5 d881812500 [Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 14:14:10 +00:00
Andreas KaratzasandOpenAI Codex 8ec00ec2bc [Tests] Release HF embedding weights after prompt extraction (#57868) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-21 05:27:29 +00:00
Andreas KaratzasandOpenAI Codex 76ee20c1ec [ROCm][Tests] Avoid pooling cleanup waits against live shared engines (#57864) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-21 05:21:35 +00:00
Kevin H. LuuandKimi 9d75dd4544 [CI] Deflake pooling shards with GPU teardown fixtures between tests (#57371) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Kimi <noreply@moonshot.cn>
 2026-09-19 11:30:02 +08:00
Kevin H. Luu 2fd6268e6d [CI] Deflake MTEB score tests with per-model mteb_tol and reruns (#57364) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-09-19 11:29:00 +08:00
Cyrus Leung dbf4b89cf2 [Misc] Remove unnecessary Transformers version guards (#57556) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-18 15:01:35 +00:00
Asaf GardinHarry MellorClaude Opus 5
 		44dd18fe0b [Model][Gemma4] Load Weights with AutoWeightsLoader (#55911) 
Signed-off-by: Josephasafg <ajgard7@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-18 12:07:30 +00:00
Kevin H. LuuandKimi f2b6152ec2 [CI] Add flaky rerun markers / timeout skips to sibling tests lacking them (#57359) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Kimi <noreply@moonshot.cn>
 2026-09-18 03:40:32 -07:00
rasmithandAndreas Karatzas d3074acdda [AMD][CI][The Rock] Fix language models standard for The Rock on mi355 (#53837) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-17 22:13:13 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Taneem Ibrahim 2e9f7bb454 [Pooling] Report actual input token usage for scoring APIs (#56573) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-13 09:20:21 -04:00
Asaf Gardinminyichenaoshen02
 		f2d45f26bd [Bugfix][Gemma] Conditionally create KV projections/norms on KV-shared layers (#54917) 
Signed-off-by: Josephasafg <ajgard7@gmail.com>
Co-authored-by: minyichen <96753147+cyc00518@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-07 15:04:00 +01:00
Li, Jiang 8340fe1bb9 [CI/Build] Fix flaky failures in CPU CI image building (#55317) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-09-04 20:27:02 +08:00
djramic d9fbe526c0 [ROCm][CI] Skip ModernBERT FP8 MTEB test when no FP8 ScaledMM kernel exists (#53589) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
 2026-08-25 08:07:09 +00:00
Harry MellorXianbao QIANClaude Opus 5
 		7ca336929c [Model] Remove ten deprecated model architectures (#53608) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Xianbao QIAN <xianbao.qian@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-24 17:06:00 -07:00
Taneem Ibrahim 79bb395eea [Pooling] Improve BGE-M3 sync pooling throughput by up to 3.13% (#53464) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-24 08:33:32 -04:00
ThunGuoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> e8888b2d68 [Model] Add FP8 quantization support for ModernBERT (#53101) 
Signed-off-by: thunguo <tew@apache.org>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-24 03:04:19 +00:00
Kevin H. Luu d29f7f5c92 [Bugfix] Load untied Gemma LM head weights (#53170) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-08-21 00:47:08 +00:00
Taneem Ibrahim 08afae2786 [ModelRunner v2] Enable MRV2 for pooling models by default (#48290) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-18 21:36:05 -07:00
Davis WertheimerClaudeHarry Mellor
 		3130573630 [Model] Add GraniteSWA and GraniteMoeSWA via existing Granite (#52706) 
Signed-off-by: Davis Wertheimer <davis.wertheimer@ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-18 20:32:46 -07:00
Qiming ZhangandKunshang Ji f4b161d7fc [XPU][UT] Fix OOM and skip graph case (#49287) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-08-18 09:47:35 +08:00
stefankoncarevicandAndreas Karatzas 402547d7f0 [Bugfix][CI] Release the shared ColBERT engine before test_colbert_hf_comparison (#52608) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-17 17:50:26 +00:00
Isotr0py 53e211d292 [CI/Build] Reduce more duplicate runner startup in tests (#52570) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-17 05:19:07 +00:00
Andreas KaratzasandOpenAI Codex 1ab2801dde [ROCm] Remove stale SDPA and skinny GEMM workarounds (#50907) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 14:19:16 -04:00
Taneem Ibrahim 9464529612 [ModelRunner v2] Enable decoder token-wise pooling (#50931) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-06 22:21:52 +00:00
music-dino a9b39d6dcb [Bugfix] Enable chunked prefill for qwen3.5-0.8B ppl test (#51153) 
Signed-off-by: Dino Music <Dino.Music@amd.com>
 2026-08-05 13:21:43 +00:00
Jiangyun Zhuandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> f9c74b4b9c [Mamba] enable prefix cache by default (#50991) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-04 15:20:51 +00:00
Andreas Karatzas 8adc840c45 [ROCm][Test] Use BF16 for Jina v5 nano MTEB test (#50917) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-04 08:33:05 +08:00
Omkar Shewale 9a4fd57cac [Model] Support jina-embeddings-v5-text-nano (EuroBERT encoder backbone) (#50688) 
Signed-off-by: Omkar Shewale <omkarshewale2001@gmail.com>
 2026-08-03 01:37:08 +00:00
chun-wan 0033211c0b [Test][V1] Add sleep/wake correctness regression test for hybrid GDN/… (#44972) 
Signed-off-by: chun-wan <chun-wan@amd.com>
 2026-08-02 22:02:42 +00:00
Taneem Ibrahim 55c98e370a [Model Runner v2] Enable BGE M3 pooling embed token_classify (#50661) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-02 12:46:03 -04:00
Taneem Ibrahim 652ba59229 [Model Runner V2] Enable encoder token embedding (#50574) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-01 07:55:36 -04:00
Taneem Ibrahim 0f17394564 [Model Runner V2] Enable encoder token classification (#50293) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-07-31 04:15:36 +00:00
Sebastian WooandClaude bebf918044 [Bugfix][Model] Reject encoder-backbone jina-embeddings-v5 checkpoints with a clear error (fixes #50337) (#50352) 
Signed-off-by: seewoo <seewoo@ucsc.edu>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-31 11:43:31 +08:00
Julien Denize 1a20d23dab [PARSER][Mistral] unified engine-based parser for reasoning and tool calls (#48947) 2026-07-30 15:38:00 +02:00
Martin HickeyandHarry Mellor 38a267cdd5 [MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent (#49570) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-30 05:02:51 -07:00
Taneem Ibrahim 43eaefba5a [ModelRunner V2] Enable sequence pooling for embedding and classification models (#48791) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-07-29 06:30:45 -07:00
stefankoncarevic 625871b52c [CI][Test] Fix pooling truncation test after VLLMError hierarchy change (#50241) 
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
 2026-07-29 12:15:56 +00:00