MOVZX and Hinata
e61dcb9143
[Mini-vLLM] SM86-only cut: models, backends, Rust, tests
...
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-18 19:19:56 +07:00
b965ee564d
[Chore] Use upstream Transformers DeepseekV4Config ( #61392 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 01:08:08 -07:00
Juan Pérez de Algaba and OpenAI Codex
5d88c034f6
[Security] Reject local verifier references in Speculators metadata ( #61587 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-14 15:30:20 +00:00
3facf822b7
[Performance] Load HF tokenizers and processors from a local snapshot ( #60825 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-12 13:45:54 +00:00
stefankoncarevic and Andreas Karatzas
814301b268
[Bugfix][Quant] Read safetensors headers of checkpoints with other file names ( #60411 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-08 01:38:41 -05:00
Harry Mellor and Claude Opus 5
e3cae8d2ac
[Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor ( #57387 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-02 02:25:18 +00:00
Harry Mellor and Claude Opus 5.5
fc2c801ad9
Disable mypy arg-type and assignment checks in tests ( #59428 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 15:18:50 +00:00
a4eb3f25d6
[Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer ( #57263 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-26 17:01:17 +08:00
344fcc252a
[Bugfix] Resolve the Hub revision once per repo ( #56092 )
...
Signed-off-by: Wauplin <lucainp@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-24 08:48:24 -07:00
Harry Mellor and Claude Opus 5.5
d881812500
[Chore] Use Transformers v5 names and drop redundant processor use_fast ( #58550 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-24 14:14:10 +00:00
Julien Debache and Harry Mellor
2bdae2a5a8
[fix] Mistral-Large-3 accuracy regression on main ( #57563 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-18 14:19:16 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Harry Mellor and Claude Opus 5
c191787a68
[Bugfix] Align vLLM YaRN with Transformers and stop re-scaling max_model_len ( #56446 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-11 13:01:38 +00:00
fe26c705f7
[Model] Add Bailing V3 VL support ( #55921 )
...
Signed-off-by: zexplorerhj <zhjoneson@163.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-10 02:46:56 -07:00
Harry Mellor and Claude Opus 5
719284fe15
[Core][Model] Unify XD-RoPE into M-RoPE and derive the channel count ( #56078 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-09 09:23:30 -07:00
Andreas Karatzas
18615ad1ee
[Bugfix] Handle null RoPE parameters for NoPE layers ( #55949 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-09-08 20:20:05 +00:00
Jiangyun Zhu
98ed0856f3
[Model] add GLM-5.3-Flash support ( #53906 )
2026-09-03 09:40:35 -07:00
qizixi and OpenAI Codex
e5377e6727
[Bugfix][Model] Fix K3 DSpark config for 96-head drafts ( #54005 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-27 15:58:28 -07:00
Zhou248
2f55ef254c
[Model] Add Qwen3-Omni DSpark support ( #52560 )
...
Signed-off-by: Zhou248 <630563665@qq.com >
2026-08-22 15:35:22 -07:00
Lee Yongjun and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
6f74337c47
[Bugfix][Spec Decode]Preserve user --speculative-config overrides for speculators-format models ( #42376 )
...
Signed-off-by: leeyongjun <jqueen.astro@gmail.com >
Signed-off-by: Lee Yongjun <35302114+elwhyjay@users.noreply.github.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-08-21 09:06:47 -07:00
Harry Mellor
f0c14b4f77
Fix weight tying ( #51665 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-20 10:31:55 +00:00
Wei Zhao
d1e3eee6fb
[Spec decode] Support Kimi-K3 DCP with DSpark ( #52188 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-08-17 13:08:45 -07:00
6adad08767
Add Muse Glimmer model support ( #51655 )
...
Signed-off-by: Tiezhen Wang <tiezhen@inferact.ai >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: Beto de Paola <betodepaola@meta.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: zixi-qi <zixi@inferact.ai >
2026-08-14 12:28:56 +08:00
Clinton Thomas and Lucas Bourtoule
90fd4a333f
Preserve revision pins in secondary artifact loaders ( #51446 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
2026-08-11 04:33:10 +00:00
Martin Hickey and Harry Mellor
38a267cdd5
[MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent ( #49570 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-30 05:02:51 -07:00
+13
aeeb36b1f1
[New model] Kimi K3 ( #50000 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Bowen Wang <abmfy@icloud.com >
Co-authored-by: gnovack <novackgm@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-07-30 03:49:58 -07:00
Harry Mellor
3f67477497
[CI] Don't try and download files that we already know don't exist ( #46854 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-26 23:56:39 +00:00
Isotr0py
6635279d8a
[Migration] Migrate GGUF quantization support to plugin ( #39612 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-06-12 12:02:21 -07:00
Julien Denize
1c3a72b8b2
[Bugfix] Add fetch_images to MistralCommonImageProcessor ( #45180 )
...
Signed-off-by: juliendenize <julien.denize@mistral.ai >
2026-06-11 16:13:01 +08:00
c9b4b184b4
[Bugfix][Voxtral] Add fetch_audio to MistralCommonFeatureExtractor (transformers>=5.10 compat) ( #44559 )
...
Signed-off-by: Yadan Wei <weiyadan@amazon.com >
Co-authored-by: Yadan Wei <weiyadan@amazon.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-06-06 07:58:09 +00:00
Misa and Misael Casarez
0e884fe638
[Bugfix] Fix _CONFIG_REGISTRY types getting wrong config class when on-disk model_type differs ( #39554 )
...
Signed-off-by: Misa <misaAle@users.noreply.github.com >
Signed-off-by: Misael Casarez <misacasa@amazon.com >
Co-authored-by: Misael Casarez <misacasa@amazon.com >
2026-04-20 19:04:48 -07:00
Injae Ryou and Isotr0py
447ce22212
[GGUF] Support non-standard quant types with prefix (e.g. UD-IQ1_S) ( #39471 )
...
Signed-off-by: Injae Ryou <injaeryou@gmail.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-04-10 07:22:53 +00:00
Hari
a3e2e250f0
[Feature] Add Azure Blob Storage support for RunAI Model Streamer ( #34614 )
...
Signed-off-by: hasethuraman <hsethuraman@microsoft.com >
2026-03-15 19:38:21 +08:00
Cyrus Leung
f96c3ab08c
[Deprecation][1/2] Remove items deprecated in v0.18 ( #36470 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-03-09 03:43:23 -07:00
Jiayi Yan and Harry Mellor
6a895197fa
[Bugfix][CI] fix typos ( #34934 )
...
Signed-off-by: 1195343015 <1195343015@qq.com >
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-03-05 17:05:46 +00:00
Cyrus Leung
e379396167
[Refactor] Clean up processor kwargs extraction ( #35872 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-03-03 19:53:53 -08:00
Michael Goin
29fba76781
[UX] Use gguf repo_id:quant_type syntax for examples and docs ( #33371 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-01-31 12:14:54 +08:00
Julien Denize and Michael Goin
ae5b7aff2b
Improve Mistral format checks. ( #33253 )
...
Signed-off-by: Julien Denize <julien.denize@mistral.ai >
Signed-off-by: juliendenize <julien.denize@mistral.ai >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-01-30 06:23:33 -08:00
Isotr0py
63b1da76ba
[Chore]: Reorganize gguf utils funtions under transformers_utils ( #29891 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-12-02 17:33:23 +00:00
Cyrus Leung
653591d5e7
[Chore] Move tokenizer initialization methods ( #29793 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2025-12-02 13:33:37 +08:00
Cyrus Leung
34a984274e
[Misc] Refactor tokenizer interface ( #29693 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2025-11-29 04:02:21 -08:00
Isotr0py
f946a8d743
[Chore]: Reorganize model repo operating functions in transformers_utils ( #29680 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-11-28 08:46:51 -08:00
Injae Ryou and Isotr0py
794029f012
[Feature]: Improve GGUF loading from HuggingFace user experience like repo_id:quant_type ( #29137 )
...
Signed-off-by: Injae Ryou <injaeryou@gmail.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-11-25 14:28:53 +00:00
57430fc95c
Default model load/config/tokenizer to mistral format if relevant files exist ( #28659 )
...
Signed-off-by: Julien Denize <julien.denize@mistral.ai >
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com >
2025-11-21 13:58:59 -08:00
Junhong Liu and Junhong
59b453eaa2
Speed up mm processor kwargs per request by spliting dynamic and static kwargs ( #26483 )
...
Signed-off-by: Junhong <liujunhong11@huawei.com >
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Co-authored-by: Junhong <liujunhong11@huawei.com >
2025-11-07 07:51:28 +08:00
pwschuurman
f7d2946e99
[Bugfix] Skip gs:// model paths for speculator detection ( #27846 )
...
Signed-off-by: Peter Schuurman <psch@google.com >
2025-11-03 14:31:03 +00:00
Harry Mellor
8fcaaf6a16
Update Optional[x] -> x | None and Union[x, y] to x | y ( #26633 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-10-12 09:51:31 -07:00
Harry Mellor
d6953beb91
Convert formatting to use ruff instead of yapf + isort ( #26247 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-10-05 07:06:22 -07:00
Xingyu Liu and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
9fb74c27a7
[Core] Support configuration parsing plugin ( #24277 )
...
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com >
Signed-off-by: Xingyu Liu <38244988+charlotte12l@users.noreply.github.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-09-10 11:32:43 -07:00
MOVZXandHinata
