 Hongxia YangandYifan Qiao
|
cd7b7c265a
|
[ROCm] Cpu offload for ROCm 7.13+ to align the hipMemcpyBatchAsync params and perf in 7.14x (#43018)
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-08-21 09:07:03 -07:00 |
|
|
Morrison TurnanskyandTyler Michael Smith
|
1183f04b74
|
[Bugfix] test_batch_inference_correctness now uses batch invariance (#48040)
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-08-21 09:06:59 -07:00 |
|
|
yimdevandusberkeley
|
574e6a00ef
|
[Bugfix][Attention] Normalize FlashInfer prefill LSE before merging (#52796)
Signed-off-by: yimdev <5779256+yimdev@users.noreply.github.com>
Co-authored-by: usberkeley <150880684+usberkeley@users.noreply.github.com>
|
2026-08-21 11:31:14 -04:00 |
|
|
Frederik GossenandClaude
|
5ee84d3c52
|
[CI][Bugfix] Use a prompt that survives offload-resume rounding in mamba offload test (#53146)
Signed-off-by: Frederik Gossen <frgossen@meta.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-21 10:14:52 -04:00 |
|
|
Zihan Zhang
|
b389ac2946
|
[Spec Decode] DFlash2: local convolution + candidate selector (#52816)
|
2026-08-20 22:27:22 -07:00 |
|
|
Chauncey
|
c6e19b3be2
|
[Bugfix][Structured Output] Avoid spurious FSM errors after speculative reasoning end (#53046)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-08-21 10:01:06 +08:00 |
|
|
|
7cfb97e337
|
[Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915)
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-20 15:42:49 -07:00 |
|
|
Thanh Phan
|
d56bbf3995
|
[Bugfix] Support MistralCommonBackend tokenizers in structured output (#52720)
Signed-off-by: Thanh Phan <thanhphantuan1110@gmail.com>
|
2026-08-20 19:08:50 +01:00 |
|
|
|
01af92e175
|
[Feature][Model Runner V2] Support extract_hidden_states speculation (#49811)
Signed-off-by: Zupeng Wang <71580390+zupengwang@users.noreply.github.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI <support@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-20 11:13:43 -04:00 |
|
|
Turner JabbourandClaude Sonnet 5
|
1eab6fef01
|
[CI] replace shellcheck script with shellcheck-py hook (#52572)
Signed-off-by: Turner <doubleujabbour@gmail.com>
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
|
2026-08-20 10:09:59 +00:00 |
|
|
Ziming Huang
|
5b1e7a812b
|
[BugFix][Mooncake] Fix Mooncake saves from sparse Mamba block tables (#51362)
Signed-off-by: ZeldaHuang <zelda.huanghuang@gmail.com>
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com>
|
2026-08-20 03:05:43 -07:00 |
|
|
Eoin-HoustounandClaude Fable 5
|
30e2394c83
|
[Bugfix] Record non-ImportError attention backend probe failures instead of crashing engine init (#51703)
Signed-off-by: Eoin <eoin@turintech.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-08-20 02:36:00 -07:00 |
|
|
Ankit Nakhawa
|
d66300a1ba
|
[Bugfix][EPD] Fix encoder round-robin fan-out (#52491)
Signed-off-by: AnkitNakhawa <ankitnakhawa@gmail.com>
|
2026-08-20 07:30:40 +00:00 |
|
|
|
a1c5b1fd9f
|
[Core][V1] Support trace_decode_token_ids for deterministic decode replay (#46701)
Signed-off-by: ianzxliu <ianzxliu@tencent.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Zhixia Liu <liuzhixia.nju@gmail.com>
Co-authored-by: ianzxliu <ianzxliu@tencent.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-20 15:12:43 +08:00 |
|
|
|
76fb6d210a
|
[Bugfix][Core] Reserve the KV null block when validating max_model_len (#47272)
Signed-off-by: hyungjun oh <92hyungjun@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-20 02:49:23 +00:00 |
|
|
Summer YangandWoosuk Kwon
|
58e5ee0158
|
[refactor] consolidate cp attn ops (#52839)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-08-19 18:04:11 -07:00 |
|
|
Guanyi Chenandz-zanez
|
bf2866f8bf
|
[KV Connector] Add decode offloading to Mooncake Store consumers (#52466)
Signed-off-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: Guanyi Chen <939416532@qq.com>
Co-authored-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
|
2026-08-19 17:40:35 -07:00 |
|
|
Giancarlo Delfin
|
d4f4d3f40f
|
[Model Runner V2][Spec Decode] Fix draft logits cache column stride in gumbel_sample (#53017)
|
2026-08-19 17:39:30 -07:00 |
|
|
lxyand“新颐”
|
0c8c3f41cf
|
[Bugfix][V1] Sync mamba_block_size via EngineCoreReadyResponse (#50809)
Signed-off-by: “新颐” <lxy537258@antgroup.com>
Co-authored-by: “新颐” <lxy537258@antgroup.com>
|
2026-08-19 16:44:36 +00:00 |
|
|
|
3a386cfaf5
|
[ROCm] Give EngineCore cleanup grace after request abort (#52281)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-19 11:39:59 -05:00 |
|
|
Xiaoan (Sean) LiuandNick Hill
|
db92053e97
|
[Core] Skip broadcasting mm tensor data to workers for prefix-cache-covered items (#52041)
Signed-off-by: Xiaoan (Sean) Liu <shawnliu0327@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-19 07:16:39 -07:00 |
|
|
Yufeng He
|
340b7e4909
|
fix: reject string schemas that mix pattern/format with length bounds (#49996)
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
|
2026-08-19 17:29:32 +08:00 |
|
|
|
63ff748f65
|
[Attention][MLA] FlashMLA sparse: DCP on the fp8_ds_mla mixed-batch path + MTP (#46514)
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-19 00:02:09 -04:00 |
|
|
Matthew Bonanniandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
8e46accab2
|
[Attention] Vectorize sparse MLA mask loads (#52217)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-18 20:32:41 -07:00 |
|
|
Willow Lopez
|
f936a267f9
|
[Bugfix][XPU] Fix Mamba state pointer overflow (#48109)
Signed-off-by: Oxygen56 <jiangth99@163.com>
|
2026-08-19 10:45:41 +08:00 |
|
|
Li, Jiang
|
e4d61d0d22
|
[CPU] Add AMX-only high-performance MLA backend for DeepSeek V2/V3/R1 (#52616)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-08-19 10:28:52 +08:00 |
|
|
Chaojun Zhang
|
f485081e8b
|
[XPU] Support EC connector KV Offloading on XPU (#49532)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
|
2026-08-19 10:25:23 +08:00 |
|
|
liuzhenwei
|
b1d9337e92
|
[EPD] Allow KV consumers to omit MM embeddings (#52697)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
|
2026-08-19 02:16:08 +00:00 |
|
|
Itay Alroy
|
a2257f95b7
|
[Elastic EP] Reduce eager-mode reconfiguration downtime (#51885)
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
|
2026-08-19 02:13:06 +00:00 |
|
|
Russell BryantandClaude Opus 4.8
|
ef47a897e2
|
[Core] Make prefix-cache NONE_HASH deterministic by default (#51875)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-18 23:14:39 +00:00 |
|
|
yinfengLiuandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
5d8a4cf976
|
[ROCm] Pad non-aligned AITER MLA heads (#51647)
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-18 16:37:10 -05:00 |
|
|
Flora Feng
|
12f64b39d2
|
[Bugfix][Structured Output] Stop XGrammar token batches at termination (#52805)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-08-18 21:21:43 +00:00 |
|
|
aoshen02andClaude Opus 5
|
7ddb50788d
|
[Bugfix][DP] Don't assume the engines started when forwarding a wake (#51481)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-08-18 13:49:35 -07:00 |
|
|
Sage Moore
|
9842d70145
|
[DBO][CI] Increase the coverage of prefill DBO in test_dbo.py (#48628)
Signed-off-by: Sage Moore <sage@neuralmagic.com>
|
2026-08-18 19:00:04 +00:00 |
|
|
Bugen ZhaoandNick Hill
|
41f179b57a
|
[Rust Frontend] Simplify data-parallel size ownership (#52575)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-18 09:06:05 +00:00 |
|
|
Qiming ZhangandKunshang Ji
|
f4b161d7fc
|
[XPU][UT] Fix OOM and skip graph case (#49287)
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-08-18 09:47:35 +08:00 |
|
|
|
0db502c8d8
|
[Kimi-K3] support DCP partial prefix cache hit (#50493)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-08-17 17:05:51 -07:00 |
|
|
Tommy AsaiandRoger Wang
|
58aa1e3d26
|
[Bugfix][SM120][MLA] Disable dense prefill for FlashInfer sparse MLA (#51395)
Signed-off-by: tommy-asai-sonarsource <yosuke.asai@sonarsource.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-08-17 16:58:02 -07:00 |
|
|
|
5ae2d38821
|
[Perf][Structured Output] Skip unused request-local reasoners (#52573)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-17 16:30:41 -07:00 |
|
|
Canlin GuoandNick Hill
|
455edc022b
|
[ModelRunnerV2] Support prompt embeds (#42963)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Canlin Guo <961750412@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-17 21:58:50 +00:00 |
|
|
Connor Carpenterandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
f08a95f8d8
|
[Rust Frontend][gRPC] Advertise LoRA capabilities (#52031)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-17 13:27:56 -07:00 |
|
|
Wei Zhao
|
d1e3eee6fb
|
[Spec decode] Support Kimi-K3 DCP with DSpark (#52188)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
|
2026-08-17 13:08:45 -07:00 |
|
|
Roberto L. CastroandCodex
|
49905ad94d
|
[3/N][Feat][Perf] Add new warmup infrastructure for JITs. Add provider registry and orchestration for JIT warmup (#50174)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-08-17 10:11:14 -07:00 |
|
|
Tyler Michael SmithandOpenAI Codex
|
017e9f4448
|
Promote prefix_cache_retention_interval to an argument and change the default to 0 (#52216)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-17 13:20:50 +00:00 |
|
|
|
70afdedc10
|
[K3] support recoverssm for K3 (#51855)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-17 19:34:14 +08:00 |
|
|
|
f27ae25473
|
[Bugfix][CPU] Take an attention group's query head count from its layers (#51852)
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Signed-off-by: R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-08-17 18:45:59 +08:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
a02cfccbc6
|
[Bugfix][Mamba] Fix overlapping state copy race (#50729)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-17 08:29:41 +02:00 |
|
|
Cheng RuiandRui Rui4 Cheng
|
6664d397bf
|
[Performance][MRV2] Cache logits-processing request state (#52329)
Signed-off-by: Rui Rui4 Cheng <chengrui4@lenovo.com>
Co-authored-by: Rui Rui4 Cheng <chengrui4@lenovo.com>
|
2026-08-17 02:36:41 +00:00 |
|
|
Shanshan ShenandAndreas Karatzas
|
0ad04cff1b
|
[ROCm][CI] Enable ViT CUDA graph tests on AMD gfx950 GPUs (#52256)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-08-17 09:52:02 +08:00 |
|
|
Guanyi Chen
|
dc9ae4b8ac
|
[Bugfix][Mooncake] Reference GPU blocks for in-flight store jobs and key the store ledger by store_job_id (#52372)
Signed-off-by: Guanyi Chen <939416532@qq.com>
|
2026-08-16 16:09:46 -07:00 |
|