 Cyrus Leung
|
c8824a1e42
|
[MM] Move supports_multimodal_inputs and cache out of registry (#57913)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-09-21 07:43:07 -07:00 |
|
|
|
0b7f11a1ee
|
[AuxOutput] Add block-keyed storage for routed-expert outputs (#45635)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-21 10:25:05 +00:00 |
|
|
liuzhenweiandKunshang Ji
|
6654bcbfca
|
[MRV2][XPU] use xpu sample kernel in mrv2 sampler (#57277)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-09-21 13:24:41 +08:00 |
|
|
czhu-cohere
|
3918f3c5a3
|
[Profiler] Unify platform-aware torch profiling (#57460)
|
2026-09-20 19:17:23 -07:00 |
|
|
Yifan QiaoandClaude Fable 5.1
|
d2983f2f16
|
[Bugfix][Spec Decode] Stop dummy draft decode steps from writing KV through stale block-table rows (#56734)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-21 01:06:25 +00:00 |
|
|
Misha GoinandErinYin
|
e5fce7b56b
|
[Core][Kernel] Share persistent workspaces for Marlin and Humming (#57421)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: ErinYin <liang.yin@daocloud.io>
|
2026-09-20 17:28:33 +08:00 |
|
|
Mi Jiazhiandjiazhimi
|
e378275a8f
|
[Model] Pass intermediate_tensors to the model when capturing CUDA gr… (#57745)
Co-authored-by: jiazhimi <jiazhimi@tencent.com>
|
2026-09-19 23:23:29 -07:00 |
|
|
Matt MastracciandClaude Fable 5.1
|
601e1c7c51
|
[Perf] Give a prefill-only batch the model state's number of logit rows (#57416)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-20 04:58:30 +00:00 |
|
|
|
5302d1fe40
|
[Model Runner V2] Support custom logits processors (#56497)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-19 17:46:49 +00:00 |
|
|
Cyrus Leung
|
751f6807d9
|
[Refactor] Use dynamic MM cache in processor (#57674)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-09-19 15:38:00 +00:00 |
|
|
Jiang WuandJiang Wu
|
6af44e2887
|
[Bugfix][CPU] Fix DeepSeek V4.1 import without Triton (#57654)
Signed-off-by: Jiang Wu <245355730+JiangLLM@users.noreply.github.com>
Co-authored-by: Jiang Wu <245355730+JiangLLM@users.noreply.github.com>
|
2026-09-19 05:46:13 +00:00 |
|
|
waizuichougou
|
a1bf8ac12d
|
[Bugfix][MRV2] Match fast-prefill padding to active LoRA batches (#56456)
Signed-off-by: waizuichougou <2082431897@qq.com>
Signed-off-by: waizuichougou <105572299+waizuichougou@users.noreply.github.com>
|
2026-09-18 22:13:23 +00:00 |
|
|
|
1dc2d854c1
|
[Feat][Model] Support encoder-side SWA-bounded replay for DeepSeek-V4.1-Flash (#56227)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-09-18 15:01:24 -07:00 |
|
|
Micah WilliamsonandAndreas Karatzas
|
d5f0a6e829
|
[Bugfix][DBO] Fix DeepEP low-latency profiling crash with DP+EP+DBO (#57502)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-09-18 06:33:41 +00:00 |
|
|
Juntian LiuandWoosuk Kwon
|
bf13ecc2f7
|
[Perf][MRV2] Share token-to-request mappings across KV cache groups (#57102)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-09-18 03:14:36 +00:00 |
|
|
|
092bdd6d57
|
[Frontend][Core] Add release_kv_cache_memory() API (#44890)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-18 07:12:12 +08:00 |
|
|
|
ac2f0ea82c
|
[MoE][Bugfix] Skip SP padded rows in grouped MoE routing (#56079)
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-09-17 19:36:19 +00:00 |
|
|
HieDeanandhiedean
|
493d4b3c04
|
[bugfix] Mark draft tokens to rebuilt their embeddings. (#57356)
Signed-off-by: HieDean <799287043@qq.com>
Co-authored-by: hiedean <hiedean@tju.edu.cn>
|
2026-09-17 15:20:37 +00:00 |
|
|
|
9e1cd262cb
|
[Bugfix] Use DP index for dense DP weight updates and EC CPU region (#56950)
Signed-off-by: lxy-alexander <lxy_alexander@outlook.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-17 22:44:36 +08:00 |
|
|
Robert ShawandClaude Opus 5
|
861a299adc
|
[Bugfix][Model Runner V2] Route dummy tokens to MoE experts during profiling (#57270)
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-17 10:11:14 -04:00 |
|
|
jiangkuaixue123andCodex
|
de24e51908
|
[MM][V2] Enable encoder-only ViT CUDA graph capture (#56922)
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-17 16:12:17 +08:00 |
|
|
Matt
|
2d8a5c5741
|
[ROCm][Bugfix] Reduce CUDA graph divergences (#57229)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
|
2026-09-17 05:12:13 +00:00 |
|
|
omerpaz95
|
fa3622a4a1
|
[EC Connector] Add Metrics Collection (#54960)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
|
2026-09-17 10:36:23 +08:00 |
|
|
Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
|
Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
|
|
18553c5c9f
|
[Kernel] Use Murmur3 RNG for Gumbel sampling (#51367)
Signed-off-by: freyfwt <110148731+freyfwt@users.noreply.github.com>
Signed-off-by: freyfwt <freytian1996@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-09-15 23:52:26 -07:00 |
|
|
Chaojun ZhangandKunshang Ji
|
622934e61f
|
[XPU] Honor device_ids for worker device placement (#56015)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-09-16 11:53:23 +08:00 |
|
|
Giancarlo Delfin
|
85c1f58d50
|
[BugFix][Model Runner V2][Spec Decode] Fix decode instance's multi-layer MTP kv caches during P/D (#55055)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-09-15 17:31:25 -07:00 |
|
|
Nick Hill
|
f2aad6aa70
|
[MRV2] Buffer util simplifications (#56888)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-15 20:36:51 +00:00 |
|
|
zack
|
0136df94b0
|
[Bugfix][Spec Decode][MoE] Avoid uninitialized EPLB state in DeepSeek V4.1 DSpark drafter (#56387)
Signed-off-by: zack <51604064+luoyuctl@users.noreply.github.com>
|
2026-09-15 14:35:47 +00:00 |
|
|
|
676650397f
|
[Feature][PCP] Support decode-only FULL CUDA graphs (#53867)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: QiuChunshuo <chunshuoq@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-15 16:27:46 +08:00 |
|
|
Nick Hill
|
7dbc3d62c3
|
[Cleanup] Remove vestigial tpu_input_batch.py (#56898)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-14 21:29:15 -07:00 |
|
|
Nick Hill
|
f84b0c4bce
|
[Model Runner V2] Tolerate lack of pinned memory support (#56908)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-14 21:28:32 -07:00 |
|
|
Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
ba2ae9f239
|
[MRV2] Run pooling post processing on non-final PP ranks (#56666)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-14 16:11:11 -07:00 |
|
|
Giancarlo Delfin
|
3bb0a03f35
|
[Model Runner V2] Acceptance estimation for adaptive verification (#52228)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-09-14 15:58:48 -07:00 |
|
|
鐘天楽andNick Hill
|
0a5747d410
|
[Profiler] Add Proton CUDA graph attribution for MRV2 (#51084)
Signed-off-by: Luossu <zhluosuu@outlook.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-14 19:02:07 +00:00 |
|
|
Anjie HouandCodex
|
73d2a8cf86
|
[2/2][Model Runner V2] FULL CUDA graph capture for microbatched steps (DBO) (#51700)
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-13 22:42:33 -07:00 |
|
|
Roberto L. CastroandCodex
|
186a1e6222
|
[Warmup] Gemma 4 de-JITification (#55768)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-13 16:47:21 -04:00 |
|
|
Lucas WilkinsonandOpenAI Codex
|
7fe8fc803d
|
[NIXL][PCP][DCP] Expose PCP producer KV shards as transfer ranks (#56645)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-13 20:23:15 +00:00 |
|
|
Taneem Ibrahim
|
0cf266a469
|
[Pooling] Support prompt embeddings in MRV2 decoder pooling (#56639)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-09-13 05:49:45 +00:00 |
|
|
Woosuk KwonandOpenAI Codex
|
2671fedfc7
|
[MRV2] Revert explicit Triton JIT warmup migration (#56654)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-13 04:07:56 +00:00 |
|
|
|
aed894c190
|
[6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-12 17:24:24 -07:00 |
|
|
|
ebe1dec2da
|
[PCP][DCP] Enable PCP+DCP on sparse-MLA models (#56157)
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai>
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai>
Signed-off-by: PatrykSaffer <patryk.saffer@mistral.ai>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai>
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patryk Saffer <PatrykSaffer@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-12 23:24:44 +00:00 |
|
|
|
7ee8a6dd01
|
[watermarking] Dual-key gumbel-max watermarking for speculative decoding support (#56122)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
|
2026-09-12 18:42:01 +00:00 |
|
|
Itay Alroy
|
658c8131c7
|
[Elastic EP] Reuse CUDA graphs across reconfiguration (#54985)
|
2026-09-12 10:26:39 -04:00 |
|
|
Canlin GuoandCodex
|
dca96bf97b
|
[Bugfix][Spec Decode] Avoid fastsafetensors deadlock for PP draft models (#54416)
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-12 14:22:30 +08:00 |
|
|
|
d43bb2f37f
|
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-09-12 00:36:47 -04:00 |
|
|
Roberto L. CastroandCodex
|
120ec4ebd2
|
[5/N][warmup][DSv4] Migrate NVIDIA CuTeDSL attention kernels (#53566)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-09-12 00:05:59 -04:00 |
|
|
Giancarlo Delfin
|
9dcf6bf344
|
[BugFix] Fix DP token padding in dflash attention metadata (#56181)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-09-11 14:33:53 -07:00 |
|
|
Nick HillandAndreas Karatzas
|
cc5dd0a857
|
[MRV1] Scope breakable cudagraphs to the piecewise path only (#56312)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-09-11 19:55:34 +00:00 |
|