Tomas Ruiz
4a5299c93f
feat: spec decode with draft models ( #24322 )
...
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
2026-01-19 16:05:46 -05:00
jiahanc and Robert Shaw
7350331718
[BugFix] Fix TRT-LLM NVFP4 DP/EP ( #32349 )
...
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com >
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
2026-01-19 14:32:24 -05:00
Yanan Cao
9d1e611f0e
[CI] Add Helion as an optional dependency ( #32482 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
2026-01-19 19:09:56 +00:00
qli88
a0490be8f1
[CI][amd] Revert NIXL connector change to avoid crash ( #32570 )
...
Signed-off-by: Qiang Li <qiang.li2@amd.com >
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
2026-01-19 18:39:16 +00:00
Netanel Haber
cd3ac5b797
support dynamic resolution image encoding for Nemotron Nano VL ( #32121 )
...
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
2026-01-19 18:15:58 +00:00
Jee Jee Li
2636d76257
[Misc] Remove unused ModelKeys ( #32608 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
2026-01-19 17:34:59 +00:00
danisereb
aa7f37ccfa
Add support for LoRA adapters in Nemotron-H models ( #30802 )
...
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com >
2026-01-19 22:30:44 +08:00
wang.yuqi
c88860d759
[Frontend] Score entrypoint support data_1 & data_2 and queries & documents as inputs ( #32577 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-01-19 14:07:46 +00:00
Nicolò Lucchesi
758df5afe7
[NIXL][Metrics] Track nixl_num_kv_expired_reqs metric in Prometheus ( #32340 )
...
Add a new metric to track the number of requests that had their KV blocks
expire. The scenario is particularly important to surface and track as it is a
vital indicator of the health of the deployment.
Currently we're resorting to track these failures through unstructured log
parsing (which is, among other thing, error string dependent); current main:
> Releasing expired KV blocks for request cmpl-071d which were retrieved by 0 decode worker(s) within 0 seconds.
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-01-19 12:28:27 +00:00
Nicolò Lucchesi
74c583bc50
[Core] Whisper support torch.compile ( #30385 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-01-19 10:02:31 +00:00
Yuxuan Zhang
71832ba71e
[GLM-4.7] GLM Model support for GLM-Lite ( #31386 )
...
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com >
Signed-off-by: Yuxuan Zhang <2448370773@qq.com >
2026-01-19 01:18:38 -08:00
Hyunkyun Moon and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
3c8740aacb
[Frontend] Add render endpoints for prompt preprocessing ( #32473 )
...
Signed-off-by: HyunKyun Moon <mhg5303@gmail.com >
Signed-off-by: Hyunkyun Moon <mhg5303@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-01-19 12:21:46 +08:00
honglyua
976af2f314
[BugFix] Fix embed_input_ids argument error of QwenVLForConditionalGeneration ( #32462 )
2026-01-19 03:06:02 +00:00
Woosuk Kwon
9a1f16da1e
[Model Runner V2] Refactor update_states ( #32562 )
...
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu >
2026-01-18 17:32:42 -08:00
Woosuk Kwon
bb1848cd62
[Model Runner V2] Support VLM ( #32546 )
...
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu >
2026-01-18 16:58:51 -08:00
Vadim Gimpelson
6101a26dc9
[BUGFIX] Fix degenerate strides in TRTLLM query tensors for FlashInfer backend. Fixes issue #32353 ( #32417 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
2026-01-18 16:57:32 -08:00
Iryna Boiko
f5d1740030
[Bugfix] Add OOT backend option ( #32471 )
...
Signed-off-by: Iryna Boiko <iboiko@habana.ai >
2026-01-18 22:20:39 +00:00
Wentao Ye
eebc58df0c
[Refactor] Remove unused cutlass moe problem size function ( #32047 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-01-18 12:46:59 -08:00
Wentao Ye
16de822c71
[Refactor] Remove unused file pallas_kv_cache_update.py ( #32433 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-01-18 12:46:39 -08:00
Deming
5480c6b1fa
[Doc] Correct comment for _jobs dict in OffloadingConnectorWorker ( #32556 )
2026-01-18 12:46:00 -08:00
Andrey Khalyavin
ba29ab441e
Use the same memory for workspace13 and fused_output. ( #31531 )
...
Signed-off-by: Andrey Khalyavin <halyavin@yandex-team.ru >
2026-01-18 19:14:22 +00:00
bnellnm
327a02d8db
[MoE Refactor] Separate Router into OO Classes ( #30623 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-01-18 11:40:49 -05:00
tjp_zju and Wentao Ye
2f03035a61
"refactor: refactor_repeated_interfaces" ( #32486 )
...
Signed-off-by: tom-zju <tanjianpingzju1990@gmail.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-01-18 22:07:01 +08:00
Isotr0py
38bf2ffb21
[Bugfix] Fix GLM-ASR audio encoder RoPE dim ( #32540 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-01-18 19:17:59 +08:00
Li Xie
c826c72a96
[Model] Support Step1 Model ( #32511 )
...
Signed-off-by: xieli <xieli@stepfun.com >
2026-01-18 10:20:46 +00:00
Canlin Guo
fe36bf5e80
[Model] Remove the unnecessary dtype conversion in MiniCPM ( #32523 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-01-18 08:07:28 +00:00
Woosuk Kwon
963dc0b865
[Model Runner V2] Minor optimization for eagle input processing ( #32535 )
...
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu >
2026-01-17 21:55:17 -08:00
Isotr0py
8cc26acd8b
[Performance] Improve Triton prefill attention kernel's performance ( #32403 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-01-17 20:19:59 -08:00
Robert Shaw and Robert Shaw
4a6af8813f
[MoE Refactor] Move Test Impl into Test Dirs ( #32129 )
...
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com >
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com >
2026-01-18 12:16:59 +08:00
Woosuk Kwon
4147910f1e
[Model Runner V2] Move mrope_positions buffer to MRopeState ( #32532 )
...
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu >
2026-01-17 20:09:48 -08:00
Karan Bansal
3055232ba0
[Feature] Add FIPS 140-3 compliant hash algorithm option for multimodal hashing ( #32386 )
...
Signed-off-by: Karan Bansal <karanb192@gmail.com >
2026-01-18 11:02:01 +08:00
Guofang.Tang and Guofang Tang
2b99f210f5
[Misc] Fix typo: seperator -> separator in flashmla_sparse.py ( #32411 )
...
Signed-off-by: Guofang Tang <tinggofun@gmail.com >
Co-authored-by: Guofang Tang <tinggofun@gmail.com >
2026-01-17 12:18:30 +00:00
Kim Hee Su
1646fea672
[Model] Molmo2: Enable quantized weight mapping for vision backbone ( #32385 )
...
Signed-off-by: kimheesu <wlskaka4@gmail.com >
2026-01-17 09:33:05 +00:00
Paul Pak
d3317bbba4
[Models] Lfm2Moe: minor name changes for resolving lora conflicts ( #29063 )
...
Signed-off-by: Paul Pak <paulpak58@gmail.com >
2026-01-16 22:12:55 -08:00
Matthew Bonanni
2e7c89e708
Revert "[Attention][MLA] Make FLASHINFER_MLA the default MLA backen… ( #32484 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-01-17 04:42:39 +00:00
vanshil shah
037a6487af
apply _validate_input to MistralTokenizer token-id chat prompts ( #32448 )
...
Signed-off-by: Vanshil Shah <vanshilshah@gmail.com >
2026-01-17 03:23:45 +00:00
Chenyaaang
484e22bc18
[TPU][Core] Enable Pipeline Parallelism on TPU backend ( #28506 )
...
Signed-off-by: Chenyaaang <chenyangli@google.com >
2026-01-16 15:29:20 -08:00
Andrew Xia and Andrew Xia
4c82b6fac7
[responsesAPI] allow tuning include_stop_str_in_output ( #32383 )
...
Signed-off-by: Andrew Xia <axia@fb.com >
Co-authored-by: Andrew Xia <axia@fb.com >
2026-01-16 21:14:40 +00:00
Xin Yang
a884bc62d6
[LoRA] Update LoRA expand kernel heuristic ( #32425 )
...
Signed-off-by: Xin Yang <xyangx@amazon.com >
2026-01-16 18:38:07 +00:00
Hashem Hashemi
7a1030431a
Atomics Reduce Counting Optimization for SplitK Skinny GEMMs. ( #29843 )
...
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com >
2026-01-16 11:45:04 -06:00
Ilya Markov and Tyler Michael Smith
c9a533079c
[EPLB][BugFix]Possible deadlock fix ( #32418 )
...
Signed-off-by: ilmarkov <markovilya197@gmail.com >
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-01-16 09:11:01 -05:00
Cyrus Leung
180e981d56
[Chore] Replace swish with silu ( #32459 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-01-16 08:22:45 +00:00
Rabi Mishra
b66b0d6abb
fix(rocm): Enable non-gated MoE (is_act_and_mul=False) support on ROCm ( #32244 )
...
Signed-off-by: rabi <ramishra@redhat.com >
2026-01-16 15:31:10 +08:00
Hongxin Xu and arlenxu
03da3b52ef
[Bugfix] Refactor to support DP parallel in R3 ( #32306 )
...
Signed-off-by: xhx1022 <1737006628@qq.com >
Co-authored-by: arlenxu <arlenxu@tencent.com >
2026-01-16 15:13:58 +08:00
Lucas Wilkinson
14ce524249
[CI] Breakup h200 tests ( #30499 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-01-16 06:23:22 +00:00
wang.yuqi
4ae77dfd42
[Frontend][1/n] Make pooling entrypoints request schema consensus | CompletionRequest ( #32395 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-01-16 06:17:04 +00:00
XiongfeiWei
73f635a75f
[Bug] Add TPU backend option ( #32438 )
...
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com >
2026-01-16 05:17:12 +00:00
cjackal and j0shuajun
35bf5d08e8
[bugfix] Fix online serving crash when text type response_format is received ( #26822 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
Signed-off-by: j0shuajun <59368606+j0shuajun@users.noreply.github.com >
Co-authored-by: j0shuajun <59368606+j0shuajun@users.noreply.github.com >
2026-01-16 12:23:54 +08:00
Kebe and Lucas Wilkinson
5de6dd0662
[Bugfix] [DeepSeek-V3.2] fix sparse_attn_indexer padding ( #32175 )
...
Signed-off-by: Kebe <mail@kebe7jun.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-01-16 03:21:55 +00:00
709502558c
[Model] Add Step3vl 10b ( #32329 )
...
Signed-off-by: luotingdan <luotingdan@stepfun.com >
Signed-off-by: ltd0924 <32387785+ltd0924@users.noreply.github.com >
Co-authored-by: luotingdan <luotingdan@stepfun.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-01-15 19:04:16 -08:00
Tomas Ruiz