 Roberto L. Castro
|
d86c66c981
|
[Feat] Add runtime monitor for post-warmup CuTeDSL compilation (#46167)
|
2026-06-23 23:33:17 +00:00 |
|
|
hillel.darshanandCursor
|
3554ada5d8
|
[CPU][Bugfix][Speculative Decoding] Accept USE_FP64_GUMBEL in CPU recovered-tokens sampler (#46069)
Signed-off-by: hillel.darshan <hillel.darshan@intel.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-06-23 11:54:07 +00:00 |
|
|
|
44d95069e9
|
Enable DeepSeek V4 and GLM-5.1 on SM120 (#43477)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-06-22 11:54:14 -07:00 |
|
|
Nick Hill
|
7df3d7dada
|
[Core] Ensure memory is pinned prior to async h2d copy (#45424)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-20 20:02:24 -07:00 |
|
|
Jee Jee Li
|
22cc891108
|
[Kernel] Add PDL support for DeepGEMM kernel (#46006)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-06-18 20:49:01 +08:00 |
|
|
Micah Williamson
|
e945169207
|
Revert "[Kernel] Add PDL support for DeepGEMM kernel" (#45999)
|
2026-06-17 22:59:48 -07:00 |
|
|
Jee Jee Li
|
4403af8fb5
|
[Kernel] Add PDL support for DeepGEMM kernel (#42996)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-06-17 20:37:17 -07:00 |
|
|
Wentao Ye
|
9c7c74bf10
|
[Log] Update deepgemm log (#45857)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-06-17 15:34:22 +00:00 |
|
|
Ting SUNandShengqi Chen
|
c45f681932
|
[Bugfix][Core] Fall back when numactl --membind is blocked in constrained containers (#45438)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
|
2026-06-16 09:49:56 -07:00 |
|
|
wang.yuqi
|
c4fd9794e9
|
[Frontend] Remove AsyncMicrobatchTokenizer. (#45759)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-06-16 08:02:11 +00:00 |
|
|
maobaolongandLi, Jiang <jiang1.li@intel.com>
|
b675cb7d0f
|
[Bugfix][CPU] Honor cgroup memory limit when computing KV cache size (#45086)
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-06-14 19:26:50 -07:00 |
|
|
Ekagra Ranjanandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
e0b9fb1290
|
[ASR] Optimize CPU preproc to get 2.5x RTFx via multi-threading (#44612)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-11 21:05:11 -07:00 |
|
|
wentian-byte
|
b8142294b7
|
[Bugfix] Restrict FlashInfer cuDNN FP8 ViT attention gate to Blackwell (SM 100) (#45251)
Signed-off-by: Wentian Byte <3400259131@qq.com>
|
2026-06-11 16:39:24 +00:00 |
|
|
Michael GoinandClaude
|
5edf7ff489
|
[Core] Release cached device memory under pressure on UMA GPUs during weight loading (#45179)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-06-11 17:49:50 +03:00 |
|
|
Michael GoinandClaude
|
87c15d46e3
|
[Bugfix] Lazily import the humming quantization backend (#44921)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-06-10 06:06:17 -07:00 |
|
|
jeff.yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
7c2aa3108a
|
fix: prevent MM cache hang from stale LRU order keys (#43595)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-08 22:48:31 -07:00 |
|
|
Tyler Michael SmithandClaude Opus 4.6
|
e2f993dc41
|
[WideEP] Integrate DeepEP v2 (#41183)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-06-08 18:07:29 -07:00 |
|
|
bnellnm
|
dc68bd8c41
|
[MoE Refactor] FusedMoE/MoERunner inversion refactor (#41184)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-06-08 10:42:58 -04:00 |
|
|
Yongye Zhu
|
b5235fca2e
|
[DSv4] Adding TRTLLM gen attention kernel (#43827)
|
2026-06-04 07:35:09 -07:00 |
|
|
Wei ZhaoandWei Zhao
|
2427094152
|
[Feature] Support EPLB for DeepSeek v4 Mega Moe (#43339)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Wei Zhao (Engrg-Hardware 1) <weizha@login-lyris01.lyris.clusters.nvidia.com>
|
2026-06-02 10:56:44 -07:00 |
|
|
JooHo Lee
|
a045c7425f
|
[MM][CG] Profile encoder CUDA graph pool memory (#41714)
Signed-off-by: JooHo Lee <jooho414@gmail.com>
|
2026-06-02 12:27:34 +08:00 |
|
|
|
29d69332aa
|
[BugFix] Fix _has_module to verify native deps via trial import (#44035)
Signed-off-by: esmeetu <jasonailu87@gmail.com>
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: esmeetu <jasonailu87@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-31 22:06:33 -07:00 |
|
|
Vadim GimpelsonandCursor
|
bf18d7e0b4
|
[Misc][NUMA] Auto-bind to PCT priority cores on DGX B300 + widen EngineCore across shard NUMA nodes (#43270)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Cursor <noreply@cursor.com>
|
2026-05-29 10:07:44 +08:00 |
|
|
TJian
|
0ba46d4b11
|
[ROCm][DSV4] Enable Tilelang MHC replacing torch/triton mhc (#43679)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
|
2026-05-28 07:05:28 +00:00 |
|
|
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
c02c758ea4
|
[Deprecation] Deprecate functions as scheduled for v0.21.0 (#43358)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-26 19:56:21 -07:00 |
|
|
Li, Jiang
|
65b7a812a2
|
[CPU] Experimentally enable Triton and MRV2 (#43225)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-05-22 01:48:17 -07:00 |
|
|
|
0ddd7dd656
|
[Frontend] DP Supervisor (#40841)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: robertgshaw2-redhat <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-21 20:33:16 -07:00 |
|
|
meena-at-workandClaude Sonnet 4.6
|
363fc84407
|
Integrate flashinfer b12x MoE and FP4 GEMM kernels for SM120/121 (#40082)
Signed-off-by: Meenakshi Venkataraman <meenakshiv@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
2026-05-20 17:21:11 +00:00 |
|
|
Wei Zhao
|
8c296de63b
|
[Perf] Re-enable flashinfer autotune by default and cleanup (#42857)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
|
2026-05-18 09:12:27 -07:00 |
|
|
Tianmu Liandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
cac81b6eda
|
[CPU Backend] Improve cpu thread utilization (#42666)
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-18 03:04:41 -07:00 |
|
|
Dipika Sikka
|
a7b801e26d
|
[MXFP4] Support for linear layers + compressed-tensors integration (#41664)
|
2026-05-12 07:49:33 -04:00 |
|
|
Jonathan MamouandBenjamin Chislett
|
a54f0d1049
|
[CPU] Fix spec decode kernel signatures for synthetic mode compatibility (#41932)
Signed-off-by: jmamou <jonathan.mamou@intel.com>
Signed-off-by: Jonathan Mamou <jonathan.mamou@intel.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
|
2026-05-10 12:07:15 +00:00 |
|
|
baonudesifeizhaiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
bc5fdc1e6a
|
Add NVFP4 all-gather GEMM fusion for AsyncTP (#41882)
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-10 01:13:22 +00:00 |
|
|
Thien Tran
|
530d371302
|
[DSv4] Improved fused Indexer Q quant kernel (#41428)
|
2026-05-09 01:20:32 -07:00 |
|
|
Nick Hill
|
989c176c0a
|
[Perf][3/n] Eliminate GPU<->CPU syncs in attention impls (#41434)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-07 19:44:24 -07:00 |
|
|
lyd1992
|
5d0fd87038
|
[CPU][RISC-V] Auto-bind OMP threads and harden nobind path (#40569)
Signed-off-by: liuyudong <liuyudong@iscas.ac.cn>
|
2026-05-06 11:38:08 +00:00 |
|
|
Martin Hickey
|
6fca518157
|
[BugFix][MyPy]: Module has no attribute "sched_getaffinity" [attr-defined] (#41465)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-05-05 13:20:37 +00:00 |
|
|
|
66dfee7121
|
[Bugfix] Fix degenerate KV cache stride causing TMA cudaErrorIllegalInstruction (#40737)
Signed-off-by: David Oy <david@baseten.co>
Signed-off-by: David Oy <58150256+the-david-oy@users.noreply.github.com>
Signed-off-by: David Oy <david.oy@baseten.co>
Co-authored-by: David Oy <david@baseten.co>
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
|
2026-05-03 23:52:18 +00:00 |
|
|
Yongye Zhu
|
bcf5cac9fb
|
[DSV4] Add knob to enable pre-attn gemm (#41443)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-05-01 12:23:17 -07:00 |
|
|
baonudesifeizhai
|
c3868bbbe4
|
[compile] Add FlashInfer FP8 async TP fusion and preserve allreduce fusion ordering #27893 (#39505)
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com>
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com>
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
|
2026-05-01 05:08:34 +00:00 |
|
|
Yongye Zhu
|
5aa371dc8e
|
[DSV4] Enable Multi-stream for Pre-Attn GEMM (#41061)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-04-28 09:08:55 -07:00 |
|
|
Zhanda ZhuandYubo Gao
|
5d5c776444
|
[Perf] FP8 FlashInfer Attn for ViT (#38065)
Signed-off-by: Zhanda Zhu <zhandazhu@gmail.com>
Co-authored-by: Yubo Gao <ybgao-nvidia@users.noreply.github.com>
|
2026-04-27 13:44:15 +08:00 |
|
|
+7
|
4d51588e23
|
[Feat] DeepSeek V4 Rebased (#40860)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
|
2026-04-26 18:31:08 -07:00 |
|
|
Andreas Karatzas
|
95995bbef8
|
[ROCm][Engine] Fix GPU memory leaks in engine shutdown and test workaround for async KV prefix cache reset (#38503)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-04-25 05:25:20 +00:00 |
|
|
lyd1992
|
100c7b65e7
|
[Platform] Fix RISC-V platform detection (lscpu parsing + non-NUMA meminfo) (#40427)
Signed-off-by: liuyudong <liuyudong@iscas.ac.cn>
|
2026-04-24 04:33:05 +00:00 |
|
|
Neil Schemenauer
|
56bdf85e10
|
[Feature] Avoid eager import of the "mistral_common" package. (#40043)
Signed-off-by: Neil Schemenauer <nas@arctrix.com>
|
2026-04-24 02:49:16 +00:00 |
|
|
Nicolò Lucchesi
|
9c2492e501
|
[Misc] Support Human-readable (k/K/m/M..) json cli arg (#40473)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2026-04-23 09:42:23 +02:00 |
|
|
Wentao Yeandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
8f87eb4622
|
[Refactor] Clean up log once scope="local" (#40540)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-04-22 16:42:43 -04:00 |
|
|
bnellnm
|
5e584ce9ec
|
[MoE Refactor] Remove SharedFusedMoE class (#35782)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-04-21 18:12:12 -04:00 |
|
|
Vadim Gimpelson
|
6d85b36a9f
|
Revert #38730 and #38791 (#40032)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
|
2026-04-21 11:44:11 -04:00 |
|