 aoshen02
|
b978df3299
|
Revert "[Frontend] Omit absent fields from /inference/v1/generate stream chunks" (#57302)
|
2026-09-16 23:31:59 -07:00 |
|
|
aoshen02andClaude Fable 5.1
|
8538017f4b
|
[Frontend] Omit absent fields from /inference/v1/generate stream chunks (#57264)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-17 05:58:56 +00:00 |
|
|
Qiming Zhang
|
e6b1a5e5e3
|
[XPU] Enable XPU eplb (#44987)
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com>
|
2026-09-17 13:19:13 +08:00 |
|
|
|
2743dc4f92
|
[Feature][Multimodal] Add cross-encoder caching to Mooncake P2P (#56242)
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-17 13:16:41 +08:00 |
|
|
Matt
|
2d8a5c5741
|
[ROCm][Bugfix] Reduce CUDA graph divergences (#57229)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
|
2026-09-17 05:12:13 +00:00 |
|
|
MattandClaude Code
|
68f2c7e74f
|
[ROCm][CI] Fix Nixl+Offloading PD edge cases on TheRock image (#57249)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
|
2026-09-16 22:56:13 -05:00 |
|
|
Itay EtelisandOr Ozeri
|
9854b580df
|
[Bugfix][KV Offload] Compact canonical MLA rows (#56799)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-17 06:24:06 +03:00 |
|
|
Nick Hill
|
5efbf0886a
|
[CI] Support FORCE_COLOR env var; enable colors in CPU CI output (#56913)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-16 22:40:21 -04:00 |
|
|
omerpaz95
|
fa3622a4a1
|
[EC Connector] Add Metrics Collection (#54960)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
|
2026-09-17 10:36:23 +08:00 |
|
|
Oxana KorzhandAndreas Karatzas
|
2307a1d16c
|
[Core] Don't issue a blocking collective RPC during engine handshake (#57226)
Signed-off-by: okorzh <okorzh@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-09-16 21:30:59 -05:00 |
|
|
Chuan (Richard) LiandCursor
|
f730a93d2b
|
[Scheduler] Add --max-num-active-seqs to cap RUNNING admission (#56758)
Signed-off-by: Li <chuali@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-09-17 10:22:12 +08:00 |
|
|
Jiangyun ZhuandCodex
|
b1f9ebc0c3
|
[Test][Core] Add hybrid model prefix cache hit-rate coverage (#57127)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-16 19:17:45 -07:00 |
|
|
dongluwandAndreas Karatzas
|
19b6ff62f2
|
[ROCm][Bugfix] Ignore descales for unquantized AITER caches (#56726)
Signed-off-by: donglu <donglu@cohere.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-09-17 01:29:50 +00:00 |
|
|
|
3e267bae70
|
[ROCm] triton+triton_kernels 3.8 mxfp4 MoE support (gpt-oss + DeepSeek-V4) (#55934)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Matt <156021403+mawong-amd@users.noreply.github.com>
|
2026-09-16 20:03:34 -05:00 |
|
|
Divakar VermaandAndreas Karatzas
|
95f4925c3a
|
[ROCm][CI] Enable AITER FP8/unquantized cases in modular-kernel sweep + fix MoRI per-tensor FP8 dispatch (#57074)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-09-16 19:53:06 -05:00 |
|
|
Thillai Chithambaram
|
a615f53364
|
[Bugfix][Metrics][MFU] Size activation traffic from the model dtype (#57070)
Signed-off-by: Thillai Chithambaram <thillaichithambaram.a@gmail.com>
|
2026-09-17 00:23:28 +00:00 |
|
|
|
c58ff535f4
|
[Bugfix][Spec Decode] Fix EAGLE and dense draft startup with EP (#56930)
Signed-off-by: Kyle Liang <kyleliang-nv@users.noreply.github.com>
Co-authored-by: Kyle Liang <kyleliang-nv@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-16 17:03:36 -07:00 |
|
|
andrewor14
|
62f9482583
|
[Tests] Delete deprecate torchao tests for v1 configs (#52956)
Signed-off-by: andrewor14 <andrewor14@gmail.com>
|
2026-09-16 23:11:27 +00:00 |
|
|
xinnywinneandBenjamin Chislett
|
fbf2c5e8be
|
[Frontend] Add per-request metrics to Responses API (#55084)
Signed-off-by: wxin <wxin@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-09-16 22:52:05 +00:00 |
|
|
aoshen02
|
0983aef8da
|
[Frontend] Return prompt metadata from /inference/v1/generate (#53187)
|
2026-09-16 15:29:07 -07:00 |
|
|
Robert ShawandClaude
|
6c3d9816af
|
[BugFix][KV Connector] Fix Deadlock with KVConnector + MTP under KV Pressure (#57104)
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-16 22:22:07 +00:00 |
|
|
yzong-rhandFlora Feng
|
5a6ccc5892
|
[Bugfix] Add Responses cache_write_tokens for API compat and CC parity (#57222)
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-09-16 21:52:34 +00:00 |
|
|
|
24a63b3340
|
[CI][Bugfix] Initialize _transfer_layer_group_ids in region_pull_worker fixture (#57211)
Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com>
Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-09-16 21:20:28 +00:00 |
|
|
Flora Feng
|
e0705731be
|
[Refactor] Remove unused interface methods (#57194)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-09-16 21:00:48 +00:00 |
|
|
Woosuk Kwon
|
30b847c1fa
|
[Perf][DSV4.1] Remove MegaMoE padding and shared padding workaround (#57204)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-09-16 19:09:30 +00:00 |
|
|
Shanshan Shen
|
bc0f47cd03
|
[ROCm][Bugfix] Revert #56433 + #51692 to fix accuracy breakdown for DeepSeek-V4 (#57132)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
|
2026-09-16 18:58:23 +00:00 |
|
|
Nicolò Lucchesi
|
fc8132a5e5
|
[Bugfix] Fix np.float64 leaking into the KV transfer metrics log (#57068)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-16 18:09:05 +00:00 |
|
|
|
9f9e1dac26
|
[Bugfix][Model] Restore causal image SWA for DeepSeek V4.1 (#57152)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-16 18:09:00 +00:00 |
|
|
Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
|
Rebecca LeeandCursor
|
6ca2b23e22
|
[ROCm] Expose kFp8DynamicTokenSym on AITER PTPC linears (#54248)
Signed-off-by: Rebecca Lee <rebecca.lee@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-09-16 11:48:30 -05:00 |
|
|
Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
|
Nicolò Lucchesi
|
804e5377aa
|
[Bugfix][HiSparse][PD] Align region-mapped pulls across logical block sizes (#57077)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-16 18:39:07 +02:00 |
|
|
Robert ShawandClaude
|
42919b49c5
|
[P/D] Report prefill worker cache hits in prompt_tokens_details (#54222)
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-16 15:23:31 +00:00 |
|
|
|
dff1bde84d
|
[Model] Qwen4Exp: fp8_e4m3 main KV cache on the QSA path (#55557)
Signed-off-by: Andrea Semeraro <andrea.semeraro@sezione1.it>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-09-16 15:07:52 +00:00 |
|
|
linitra24andJee Jee Li
|
c81eb3e69c
|
[LoRA] Support modules_to_save for sequence classification (#53555)
Signed-off-by: linitra24 <Joy25810@foxmail.com>
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-09-16 07:15:42 -07:00 |
|
|
sashko-zakharchukandzjy0516
|
f8b5c11468
|
[Bugfix][Kimi-K3] Do not classify a stateless first chunk as a decode (#51483)
Signed-off-by: Oleksandr Zakharchuk <oleksandr.zakharchuk@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
|
2026-09-16 21:24:48 +08:00 |
|
|
Nicolò Lucchesi
|
8b1d188046
|
[Bugfix][Mooncake] Report request-level KV load failures under HMA (#56855)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-16 14:55:40 +02:00 |
|
|
|
903285fbcc
|
[Bugfix] Let an optional Literal flag accept the None it advertises (#52370)
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-09-16 12:14:54 +00:00 |
|
|
|
d6a1677d55
|
[Model][DSv4.1] FlashMLA mega attention and the NVFP4 compressed KV cache (#56935)
Signed-off-by: Yongye Zhu <yongye@inferact.ai>
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Yunxiao Ning <73625538+foraxe@users.noreply.github.com>
Co-authored-by: AI Assistant <noreply@openai.com>
|
2026-09-16 03:07:50 -07:00 |
|
|
Ganesh RandCursor
|
82731931ae
|
[Bugfix][CPU] Fall back to CpuPlatform when zentorch fails to import (#54923)
Signed-off-by: R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-09-16 17:15:12 +08:00 |
|
|
Canlin GuoandCodex
|
03f67b3ad1
|
[Perf][DSV4.1] Pad shared experts for native MegaMoE fusion (#56568)
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-16 17:08:42 +08:00 |
|
|
|
f37c550bf6
|
[Feature][Frontend] Expose effective attention block size for DCP (#56538)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-16 08:29:43 +00:00 |
|
|
dev
|
8be5205abb
|
[Parser] Fix: correct parser frontend handling of reasoning end and boundary tokens (#56635)
Signed-off-by: DEV TYAGI <devtyagi3909@gmail.com>
|
2026-09-16 01:17:22 -07:00 |
|
|
Ama Senevirathneandamasen02
|
9ca6dbba71
|
fix(multimodal): tolerate malformed EXIF metadata during hashing (#56527) (#56576)
Signed-off-by: amasen02 <amasen02@users.noreply.github.com>
Co-authored-by: amasen02 <amasen02@users.noreply.github.com>
|
2026-09-16 07:56:50 +00:00 |
|
|
Li, Jiang
|
b68408043d
|
[Bugfix][CPU] Add per-tensor FP8 W8A16 kernel to fix Ministral crash on CPU (#56985)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-09-16 15:49:17 +08:00 |
|
|
MINJUN GILandOr Ozeri
|
f12fe10c5c
|
[Bugfix][KV Offload] Track cache recency once per request (#51787)
Signed-off-by: mindungil <alswnsrlf12@naver.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-16 10:47:26 +03:00 |
|
|
|
ab35354c21
|
[ROCm][AITER] Skip AITER norm kernels when flattening to 2D would copy (#55991)
Signed-off-by: Zheng Gong <zgong@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
|
2026-09-16 07:26:56 +00:00 |
|
|
Wei Zhao
|
f30a195bbb
|
[Bugfix] Fix incorrect Mamba block allocation estimate that prevents request admission (#57050)
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-09-16 07:23:20 +00:00 |
|
|
|
18553c5c9f
|
[Kernel] Use Murmur3 RNG for Gumbel sampling (#51367)
Signed-off-by: freyfwt <110148731+freyfwt@users.noreply.github.com>
Signed-off-by: freyfwt <freytian1996@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-09-15 23:52:26 -07:00 |
|
|
|
0f8fa53acf
|
[Bugfix][KV Offload] Check cgroup memory before SHM allocation (#54014)
Signed-off-by: Alex <jihui.huang@daocloud.io>
Signed-off-by: AlexHuang <alex.tech.lab@outlook.com>
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
|
2026-09-16 09:17:50 +03:00 |
|