 Nick Hill
|
e392bf7a68
|
[BugFix][MRV2] Ensure all req slots are accounted for when scheduling (#46974)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-02 10:19:24 -07:00 |
|
|
Nick Hill
|
443e68cfa6
|
[Bugfix] Fix pooled Whisper encoder sliding-window kernel size (#47437)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-02 10:19:11 -07:00 |
|
|
Chauncey
|
320ee285c9
|
[Model Runner V2][Perf] Warm up GLM-5.2 DSA indexer prefill metadata kernel (#47285)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-07-02 16:31:31 +00:00 |
|
|
Bugen Zhao
|
ec0ffaacc8
|
[Rust Frontend] Improve scheduler stats logging parity (#47435)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-02 15:55:25 +01:00 |
|
|
Yuxuan Zhang
|
178fd56094
|
support GLM-5.2 gate use FP32 (#47410)
Signed-off-by: zRzRzRzRzRzRzR <Yuxuan.Zhang2@liverpool.ac.uk>
|
2026-07-02 22:45:39 +08:00 |
|
|
Woosuk KwonandClaude Fable 5
|
a47f38f825
|
[Bugfix][Model Runner V2][Spec Decode] Fix int32 offset overflow in block verification kernels (#47383)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-02 07:32:38 -07:00 |
|
|
Nick Hill
|
3e158ae62d
|
[ModelRunner V2] Fix Mamba2 crash on non-spec-decode (#47428)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-02 07:05:16 -07:00 |
|
|
Wentao YeandNick Hill
|
a2f713002d
|
[ModelRunner V2] Enable by default for all dense models (#44443)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-02 18:48:57 +08:00 |
|
|
TJian
|
de2a8fc042
|
[ROCm] [PyTorch] Move to stable abi since ROCm upgraded to torch 2.11 (#47128)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
|
2026-07-02 18:34:07 +08:00 |
|
|
Michael Goin
|
84b9c2762f
|
Update DeepGEMM tag to point to latest nv-dev branch for sm120 support (#47304)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-07-02 18:33:44 +08:00 |
|
|
Bugen Zhao
|
25fcb65d51
|
[Rust Frontend] Use enum-backed domain types for engine outputs and structured outputs (#47283)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-02 10:41:46 +01:00 |
|
|
Pranav ThakurandBugen Zhao
|
08a8a4af3f
|
feat(rust): expose profiler control routes in Rust frontend (#46306)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-02 08:46:07 +00:00 |
|
|
|
b0b8a286dd
|
[Model] Add LLaVA-OneVision-2 (LlavaOnevision2ForConditionalGeneration) (#44785)
Signed-off-by: chengzheng345 <209475443+chengzheng345@users.noreply.github.com>
Co-authored-by: chengzheng345 <209475443+chengzheng345@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-02 16:41:49 +08:00 |
|
|
|
3af8789559
|
[Feature] Universal speculative decoding for heterogeneous vocabularies (TLI) (#38174)
Signed-off-by: wan-danfeng <wandanfeng0802@gmail.com>
Signed-off-by: Wonderful <wandanfeng0802@gmail.com>
Co-authored-by: Wan_DF <wonderful199082@126.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-07-02 01:34:20 -07:00 |
|
|
Chaojun Zhang
|
8357226f4f
|
[XPU][CI] Split test_punica_ops into separate pytest invocations for stability (#47376)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
|
2026-07-02 07:50:55 +00:00 |
|
|
Hiki
|
2665ed704b
|
[Bugfix][Kernel] Correct FlashInfer CUTLASS MoE tuning token bound (#46838)
Signed-off-by: Haobin Guo <haobing@nvidia.com>
|
2026-07-02 05:11:00 +00:00 |
|
|
xaguilar-amd
|
09663abde0
|
[ROCm][MLA] Fuse MLA q/kv RMSNorm + FP8 per-token quant in the FP8 attention path (#44977)
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com>
Signed-off-by: Xavier Aguilar <Xavier.AguilarFruto@amd.com>
|
2026-07-02 13:00:41 +08:00 |
|
|
Giancarlo Delfin
|
d63c8e9444
|
[BugFix][Spec Decode] Compact shared topk indices buffer after first MTP draft step (#47238)
|
2026-07-01 21:38:51 -07:00 |
|
|
Jee Jee Li
|
1360c42fe6
|
[UX] Include NVTX in cuda.txt (#47319)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-07-01 19:38:50 -07:00 |
|
|
Jee Jee Liandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
d0a2584773
|
[Misc] Use functions instead of PTX for the PDL instruction (#46984)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-01 19:38:35 -07:00 |
|
|
peizhang56andClaude
|
7fe7fa9cda
|
[CI][Bugfix] Rerun test_engine_log_metrics_ray on Ray GCS startup timeout (#47208)
Signed-off-by: pei.zhang <pei.zhang@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-01 21:32:09 -05:00 |
|
|
Michael Goin
|
2b753ad200
|
[Spec Decode] DSpark speculators checkpoint support (#47093)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-07-01 17:32:27 -07:00 |
|
|
Andrey TalmanandAndrey Talman
|
e196268bad
|
[Docker] Remove unused Dockerfile.nightly_torch (#47338)
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>
|
2026-07-01 16:19:42 -07:00 |
|
|
Andrey TalmanandAndrey Talman
|
e91f5f8439
|
[CI] Remove torch_nightly mirror tags (superseded by TORCH_NIGHTLY full-nightly build) (#47342)
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>
|
2026-07-01 16:19:06 -07:00 |
|
|
Yongye ZhuandClaude Opus 4.8
|
fa248139a0
|
[MoE] Plumb gemm1_alpha/beta/clamp_limit into TRT-LLM FP8 MoE (#45723)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-07-01 14:34:05 -07:00 |
|
|
Yongye Zhu
|
d3229431f9
|
[DSV4] Better MXFP8 quantization kernel (#47229)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-07-01 14:33:51 -07:00 |
|
|
Nick Hill
|
4787f2dd1b
|
[Bugfix] Don't read KV cache past seq_len in triton paged attn kernels (#47305)
|
2026-07-01 12:43:00 -07:00 |
|
|
Nick Hill
|
8cfeb84dba
|
[ModelRunner V2] Warmup cross-attn properly in encoder-decoder case (#47308)
|
2026-07-01 12:36:48 -07:00 |
|
|
Chaitanya Sri Krishna Lolla
|
5fd442187c
|
[ROCm][P/D] MoRIIO toy proxy: support JSON Content-Type for OpenAI clients. (#46482)
Signed-off-by: lcskrishna <lollachaitanya@gmail.com>
|
2026-07-01 19:17:05 +00:00 |
|
|
Yiliu DongandNick Hill
|
00eb7cefa3
|
[Bugfix] Prevent padding placeholders from reaching embeddings (#47029)
Signed-off-by: qianlihuang <91178480+qianlihuang@users.noreply.github.com>
Signed-off-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-01 09:26:03 -07:00 |
|
|
Michał Ganczarenko
|
c8bdcc0116
|
[Bench][BugFix] Fix empty decoder prompt for Cohere ASR in throughput benchmark (#47135)
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
|
2026-07-01 15:42:27 +00:00 |
|
|
|
f5a8d73377
|
[Spec Decode] DSpark (#46995)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-07-01 08:30:24 -07:00 |
|
|
Michał GanczarenkoandKunshang Ji
|
63fcce4de1
|
[Bugfix] Fix GraniteMoeShared weight loading broken by #41184 (#47031)
Signed-off-by: <Michal Ganczarenko> <michal.ganczarenko@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-07-01 22:39:12 +08:00 |
|
|
Bugen Zhao
|
c638f9216a
|
[Rust Frontend] Split engine core DTOs into separate modules (#47265)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-01 15:28:21 +01:00 |
|
|
Chaojun Zhang
|
13c49f9845
|
[xpu][lora]: Align LoRA implementation with Punica GPU: fix _apply_expand rank mismatch, add_inputs hardcode, and MoE EP (#45368)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
|
2026-07-01 22:14:04 +08:00 |
|
|
Nick Hill
|
f1cf6b0086
|
[CI] Fix segfault in tracing test (#47299)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-01 14:00:37 +00:00 |
|
|
Harry Mellor
|
a78c15616f
|
Migrate GPTBigCode and Starcoder2 to the Transformers modeling backend (#30966)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-01 13:41:36 +00:00 |
|
|
|
5c4db60f01
|
docs(security): document gRPC interface as insecure for private use only (#45903)
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
Co-authored-by: Russell Bryant <russell.bryant@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
|
2026-07-01 12:39:57 +00:00 |
|
|
Fangzhou AiandClaude
|
4e5ca89cfe
|
[ROCm][MiniMax-M3] Cross-layer lightning-indexer top-k sharing (#47269)
Signed-off-by: Fangzhou Ai <fangzhouai@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-01 10:50:09 +00:00 |
|
|
Harry Mellor
|
a22e0dfc69
|
[Model] Remove AyaVision, MusicFlamingo (#47263)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-01 10:39:33 +00:00 |
|
|
stevenkuang
|
cc56379e28
|
[Model] Support Hy3 token suffix and JSON Schema array types (#47192)
Signed-off-by: stevenkuang-tencent <stevenkuang@tencent.com>
|
2026-07-01 10:16:07 +00:00 |
|
|
Aleksei IvashovandNickLucche
|
024b06b0dc
|
[Bugfix] Expose usage field in GenerateResponse for disaggregated serving (#42748)
Signed-off-by: AIvashov <ivashov.aleksey@proton.me>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-07-01 10:00:19 +00:00 |
|
|
Harry Mellor
|
e7d0fcbc09
|
[CI] Fix various failures on main (#47197)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-01 10:35:34 +01:00 |
|
|
akii96
|
aa8bb5562e
|
[ROCm][Perf][Bugfix] DSv4 indexer: use platform FP8 dtype (fnuz) for Q-quant on gfx942 (#46730)
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
|
2026-07-01 17:33:55 +08:00 |
|
|
Andy Lo
|
fa4bec9056
|
[Bugfix] Fix pooled Whisper sliding-window KV sizing (#47071)
Signed-off-by: Andy Lo <andy@mistral.ai>
|
2026-07-01 11:33:19 +02:00 |
|
|
Jyothirmai KottuandClaude
|
dee5da1dec
|
[Test] Run SageMaker handler-override tests in-process via TestClient (#47250)
Signed-off-by: Jyothirmai Kottu <jkottu@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-01 09:14:00 +00:00 |
|
|
|
ed41aa270a
|
[ROCm][DSV4] Use aiter mHC pre/post as the default ROCm path (#43950)
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Signed-off-by: Fangzhou-Ai <fangzhouai@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-01 16:27:42 +08:00 |
|
|
Aaron HaoandClaude Sonnet 4.6
|
77a9c5ae28
|
Weight sync refactor + move sparse nccl engine (#44353)
Signed-off-by: hao-aaron <ahao@anyscale.com>
Signed-off-by: haoaaron <ahao@anyscale.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
2026-07-01 01:25:19 -07:00 |
|
|
Yejing LaiandKunshang Ji
|
f651a8a9a4
|
[XPU][UT]Enable ut qk_norm_rope_fusion (#42486)
Signed-off-by: Lai, Yejing <yejing.lai@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-07-01 07:38:03 +00:00 |
|
|
Jee Jee Li
|
8f82be5705
|
[CI/Build] Fix LoRA testing (#47242)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-07-01 15:36:13 +08:00 |
|