380638afbb
[Model] Add Clef-Flash support ( #61308 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-13 05:46:37 +00:00
Canlin Guo and Claude Opus 5.5
fe4df3df0e
[Bugfix][GLM-5.3-Flash] Restore MTP index sharing across draft iterations ( #61079 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 05:41:52 +00:00
Harry Mellor and Claude Opus 5.5
2c99ee9333
[Chore] Remove vendored code made redundant by Transformers 5.16.1 ( #60623 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 09:29:03 +00:00
Stefano Castagnetta and Claude Opus 5
f0b4b36890
[Bugfix][Quantization] Detect NVFP4 in ModelOpt mixed-precision checkpoints ( #56050 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-10-02 12:30:22 +00:00
hcl and Harry Mellor
e35082e7b6
[Bugfix][CLI] Include inherited field docstrings in get_attr_docs ( #49821 )
...
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com >
Signed-off-by: hclsys <chenglunhu@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-01 23:35:58 +00:00
Nils Matteson
9e6550bd05
[Bugfix] Register IR providers before hashing configuration ( #58756 )
...
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com >
2026-10-01 02:34:28 +00:00
Guy Stone and Claude Fable 5
37d6174096
[Metrics] Add --custom-histogram-buckets to override histogram bucket families ( #48867 )
...
The core engine histograms ship with default bucket boundaries tuned for
typical serving workloads. The `--custom-histogram-buckets` option replaces
the boundaries of one or more bucket families. Use it, for example, to track
sub-300ms latency SLOs with the request-phase histograms, whose smallest
default boundary is 0.3s.
Signed-off-by: Guy Stone <guys@spotify.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-10-01 01:03:39 +00:00
Harry Mellor and Claude Opus 5.5
fc2c801ad9
Disable mypy arg-type and assignment checks in tests ( #59428 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 15:18:50 +00:00
apex-mochen
c4fa6f36a7
[Tests][Multimodal] Cover scoped processor kwargs precedence ( #59399 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
2026-09-30 10:54:32 +00:00
Daniel M. García-Ocaña Hernández
295a0b8d64
[Bugfix][Multimodal] Fix flat/scoped mm_processor_kwargs merge and resolution ( #56372 )
...
Signed-off-by: Daniel M. García-Ocaña Hernández <danielgarciaocana@gmail.com >
2026-09-29 03:58:39 +00:00
a7d203efe5
fix(config): apply presence_penalty/frequency_penalty from override-generation-config ( #50769 )
...
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com >
Signed-off-by: hclsys <chenglunhu@gmail.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-24 08:48:18 -07:00
Harry Mellor and Claude Opus 5.5
d881812500
[Chore] Use Transformers v5 names and drop redundant processor use_fast ( #58550 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-24 14:14:10 +00:00
lucamotz and Claude Opus 5.5
ee2f3cdf12
[Bugfix][Spec Decode] Separate DSpark width from MTP stage validation ( #54631 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-24 06:08:22 +00:00
1b3b88ec2b
[Core] structured generation mode for DiffusionGemma model (Jev-like) ( #57250 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Razorback16 <razorback16@protonmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Razorback16 <razorback16@protonmail.com >
2026-09-22 20:51:17 +00:00
aoshen02
feb87b1934
[Bugfix] Narrow AuxOutput KV restrictions to known PD connectors ( #58150 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
2026-09-22 23:09:39 +08:00
Cyrus Leung
c8824a1e42
[MM] Move supports_multimodal_inputs and cache out of registry ( #57913 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-21 07:43:07 -07:00
0b7f11a1ee
[AuxOutput] Add block-keyed storage for routed-expert outputs ( #45635 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-21 10:25:05 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
f37c550bf6
[Feature][Frontend] Expose effective attention block size for DCP ( #56538 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Signed-off-by: Julien Darve <jdarve@NVIDIA.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-16 08:29:43 +00:00
Isotr0py
22258a26bc
[Multimodal] Use GPU NVDEC for EPD encoder-only instance video media IO ( #53675 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-11 22:20:11 +08:00
Isotr0py
1356635d83
[New model][Multimodal] Add DeepSeek-V4-Flash-Vision-Exp support ( #54566 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-02 11:09:09 -07:00
+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00
Tianyu Guo
12c84b98dd
[Config][EC] Normalize producer-only encoder config ( #53656 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-08-25 07:53:04 +00:00
7ca336929c
[Model] Remove ten deprecated model architectures ( #53608 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Xianbao QIAN <xianbao.qian@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-24 17:06:00 -07:00
Tyler Michael Smith and OpenAI Codex
017e9f4448
Promote prefix_cache_retention_interval to an argument and change the default to 0 ( #52216 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-17 13:20:50 +00:00
Giancarlo Delfin and Claude Fable 5
f80b66f548
[Model Runner V2][Spec Decode] Add KV cache support for multi-layer MTP ( #50062 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-13 16:22:29 -07:00
Harry Mellor
c65aa2ee6c
Bump Transformers version to 5.15.0 ( #51668 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-11 09:00:51 -07:00
Clinton Thomas and Lucas Bourtoule
90fd4a333f
Preserve revision pins in secondary artifact loaders ( #51446 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
2026-08-11 04:33:10 +00:00
Harry Mellor
70b84f0bcb
Fix Gemma 4 for upcoming Transformers version ( #49797 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-10 07:48:26 +00:00
Tianyu Guo
b30291cf0c
[EPD] Remove duplicate image preprocessing in EPD and enable preprocess on GPU ( #50390 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-08-05 15:59:54 -07:00
aoshen02 and yihengz
6153dbe036
[R3] Unify routed expert shape configuration ( #50940 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: yihengz <yihengz@nvidia.com >
2026-08-04 20:20:41 -07:00
Isotr0py
1180b604f8
[Multimodal] Expose mm hash algothrim selection to cli args ( #49686 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-07-31 05:25:19 +00:00
Martin Hickey and Harry Mellor
38a267cdd5
[MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent ( #49570 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-30 05:02:51 -07:00
32e657e689
[BugFix] eagle draft max position embeddings ( #49343 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-07-29 01:24:47 -07:00
Tri Vo and Cyrus Leung
fe5145765f
[Core] Keep attention backends eligible for text-only serving of prefix-LM models ( #48796 )
...
Signed-off-by: qtris123 <voquangtri2021@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-07-25 03:06:14 -07:00
Nils Matteson and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
149daf0d72
[Bugfix] Exclude location-derived path vars from torch.compile cache factors ( #47573 )
...
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-22 16:56:09 -07:00
Isotr0py
1750e443f2
[Misc] Move PyNvVideoCodec stuff out of gpu worker ( #49322 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-07-22 10:11:56 +08:00
Michael Goin
02c01f442b
[Model] Use standard ModelOpt config for Inkling NVFP4 ( #48990 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-07-17 18:13:14 -07:00
fb5ec0dc9e
[Model] Add Inkling MTP=1 support [3/N] ( #48869 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-16 13:27:21 -07:00
Nils Matteson and Nils Matteson
d6875196ad
[Bugfix] Exclude kv_cache_memory_bytes from CacheConfig.compute_hash ( #47356 )
...
Signed-off-by: Nils Matteson <nils@thaw.sh >
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com >
Co-authored-by: Nils Matteson <nils@thaw.sh >
2026-07-07 10:46:51 -07:00
cbe9c40f99
[Bugfix] Forward callable hf_overrides to the draft model config ( #45352 )
...
Signed-off-by: HumphreySun98 <humphreysun98@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-07-06 21:12:38 -07:00
alex101-ops and zc02384840
f2aaf59151
[Feature] Support MTP speculative decoding for Bailing hybrid models ( #44880 )
...
Signed-off-by: zc02384840 <zc02384840@antgroup.com >
Co-authored-by: zc02384840 <zc02384840@antgroup.com >
2026-07-06 10:38:50 +08:00
Andreas Karatzas
ceae5bcbda
[ROCm][CI] Fix nixl tests ( #45219 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-06-23 13:11:40 -05:00
Zhanda Zhu and Yubo Gao
5d5c776444
[Perf] FP8 FlashInfer Attn for ViT ( #38065 )
...
Signed-off-by: Zhanda Zhu <zhandazhu@gmail.com >
Co-authored-by: Yubo Gao <ybgao-nvidia@users.noreply.github.com >
2026-04-27 13:44:15 +08:00
realliujiaxu
7845379230
[Bugfix] add support for 'num_attention_groups' in ModelArchConfigConvertorBase for Step3p5 ( #39796 )
...
Signed-off-by: realliujiaxu <realliujiaxu@163.com >
2026-04-16 05:48:00 +00:00
Roger Wang
1dae7b7843
[Bugfix] Exclude language_model_only key from MM AOT compile hash but include in model one ( #34508 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
2026-02-13 13:59:00 +00:00
TJian
5001211369
[ROCm] [CI] fix test_unrecognized_env ( #34350 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-02-11 18:50:44 +00:00
Gregory Shtrasberg
f0ca0671c7
[Feature] Warn about unrecognized environment variables ( #33581 )
...
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com >
2026-02-10 15:45:38 -06:00
kourosh hakhamaneshi
2f6d17cb2f
[rocm][ray] Fix: Unify Ray device visibility handling across CUDA and ROCm ( #33308 )
...
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com >
2026-02-04 10:09:14 -08:00
