Nicolò Lucchesi
|
164dca8d1e
|
[Nixl] Separate transport-failure metrics from KV expiry (#55854)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-18 18:08:59 +02:00 |
|
Nicolò Lucchesi
|
a7156060c1
|
[Core] Make parallel sampling (n>1) reqs admission atomic (#53936)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-18 18:08:44 +02:00 |
|
Simon Danielsson
|
70df48dc3d
|
[Bugfix][KV Cache][GLM-5.3-Flash] Disable slot mapping kernel for the kpool tail buffer (#57317)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
|
2026-09-17 22:42:54 -04:00 |
|
Ren Yuzhou
|
7b94293627
|
[Bugfix][ROCm][KV Offload] Use private pinned tensors for CPU KV offload (#57160)
Signed-off-by: Yuzhou Ren <ryzcqu@outlook.com>
|
2026-09-18 02:35:23 +00:00 |
|
  
|
092bdd6d57
|
[Frontend][Core] Add release_kv_cache_memory() API (#44890)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-18 07:12:12 +08:00 |
|
Wei Zhao
|
ccde9c61e9
|
[Bugfix] Restore KV cache metadata GET method for external event consumer (#56925)
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-09-17 23:01:35 +00:00 |
|
 Woosuk KwonandCodex
|
80447d2765
|
[Bugfix][DSv4.1] Fix FlashInfer DSpark non-causal attention (#57432)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-17 15:14:37 -07:00 |
|
Misha Goin
|
db7a24c230
|
[Perf] Add fused DFlash2 grouped convolution (#55960)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-09-17 21:38:59 +00:00 |
|
Doug Brown
|
2b02c6c29b
|
[Model] Add Cohere2MoE Eagle3 auxiliary hidden states (#49819)
Signed-off-by: Doug Brown <doug@github.douggo.com>
|
2026-09-17 20:09:36 +00:00 |
|
 ![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
ac2f0ea82c
|
[MoE][Bugfix] Skip SP padded rows in grouped MoE routing (#56079)
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-09-17 19:36:19 +00:00 |
|
 
|
67e5b0acc9
|
[Bugfix][HiSparse][NIXL] Import full blocks without tail prefill on D (#57049)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-09-17 15:26:10 -04:00 |
|
Michele Campi
|
6a344d8c85
|
[Bugfix][Metrics] Do not log a 0.0% prefix cache hit rate before any query (#54990)
Signed-off-by: Michele Campi <215741962+MicheleCampi@users.noreply.github.com>
|
2026-09-17 18:19:13 +00:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)   
|
4f9a41ac94
|
[CI/Build][Hardware][NVIDIA] Add Rubin CUDA 13.4 nightly images (#55953)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-17 08:05:31 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)   
|
9e1cd262cb
|
[Bugfix] Use DP index for dense DP weight updates and EC CPU region (#56950)
Signed-off-by: lxy-alexander <lxy_alexander@outlook.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-17 22:44:36 +08:00 |
|
 Robert ShawandClaude Opus 5
|
861a299adc
|
[Bugfix][Model Runner V2] Route dummy tokens to MoE experts during profiling (#57270)
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-17 10:11:14 -04:00 |
|
 CaiJohnandjohn
|
75c71390d5
|
[Bugfix] Escape control characters in xgrammar choice grammar (#48115)
Signed-off-by: john <6712432+CaiJohn@users.noreply.github.com>
Co-authored-by: john <6712432+CaiJohn@users.noreply.github.com>
|
2026-09-17 15:57:47 +03:00 |
|
 Martín el CheikhandClaude
|
2e50824766
|
[Bugfix] Fall back to native sampling when FlashInfer cannot target the GPU (#48956)
Signed-off-by: martin el cheikh <martinelcheikh@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-17 12:27:54 +00:00 |
|
 aoshen02andaoshen02
|
9ff08f9493
|
[Bugfix] Honor skip_reading_prefix_cache for KV connector hits (#57269)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
|
2026-09-17 03:46:28 -07:00 |
|
 
|
0d72627565
|
[Perf][EPD] Batch image requests per encoder (#57095)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-17 03:45:52 -07:00 |
|
 Itay EtelisandOr Ozeri
|
b6e7c1f1f0
|
[kv_offload] Skip scratch groups (#57145)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-17 13:22:49 +03:00 |
|
  
|
40b40d1d39
|
[Bugfix][KV Offload] Register the offload region in chunks (#51081)
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Colton Ottley <coltonottley@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-17 13:19:56 +03:00 |
|
 jiangkuaixue123andCodex
|
de24e51908
|
[MM][V2] Enable encoder-only ViT CUDA graph capture (#56922)
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-17 16:12:17 +08:00 |
|
  
|
2743dc4f92
|
[Feature][Multimodal] Add cross-encoder caching to Mooncake P2P (#56242)
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-17 13:16:41 +08:00 |
|
 MattandClaude Code
|
68f2c7e74f
|
[ROCm][CI] Fix Nixl+Offloading PD edge cases on TheRock image (#57249)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
|
2026-09-16 22:56:13 -05:00 |
|
 Itay EtelisandOr Ozeri
|
9854b580df
|
[Bugfix][KV Offload] Compact canonical MLA rows (#56799)
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-17 06:24:06 +03:00 |
|
omerpaz95
|
fa3622a4a1
|
[EC Connector] Add Metrics Collection (#54960)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
|
2026-09-17 10:36:23 +08:00 |
|
 Oxana KorzhandAndreas Karatzas
|
2307a1d16c
|
[Core] Don't issue a blocking collective RPC during engine handshake (#57226)
Signed-off-by: okorzh <okorzh@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-09-16 21:30:59 -05:00 |
|
 Chuan (Richard) LiandCursor
|
f730a93d2b
|
[Scheduler] Add --max-num-active-seqs to cap RUNNING admission (#56758)
Signed-off-by: Li <chuali@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-09-17 10:22:12 +08:00 |
|
 Jiangyun ZhuandCodex
|
b1f9ebc0c3
|
[Test][Core] Add hybrid model prefix cache hit-rate coverage (#57127)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-16 19:17:45 -07:00 |
|
 dongluwandAndreas Karatzas
|
19b6ff62f2
|
[ROCm][Bugfix] Ignore descales for unquantized AITER caches (#56726)
Signed-off-by: donglu <donglu@cohere.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-09-17 01:29:50 +00:00 |
|
Thillai Chithambaram
|
a615f53364
|
[Bugfix][Metrics][MFU] Size activation traffic from the model dtype (#57070)
Signed-off-by: Thillai Chithambaram <thillaichithambaram.a@gmail.com>
|
2026-09-17 00:23:28 +00:00 |
|
 Robert ShawandClaude
|
6c3d9816af
|
[BugFix][KV Connector] Fix Deadlock with KVConnector + MTP under KV Pressure (#57104)
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-16 22:22:07 +00:00 |
|
 
|
24a63b3340
|
[CI][Bugfix] Initialize _transfer_layer_group_ids in region_pull_worker fixture (#57211)
Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com>
Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-09-16 21:20:28 +00:00 |
|
Nicolò Lucchesi
|
fc8132a5e5
|
[Bugfix] Fix np.float64 leaking into the KV transfer metrics log (#57068)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-16 18:09:05 +00:00 |
|
 
|
9f9e1dac26
|
[Bugfix][Model] Restore causal image SWA for DeepSeek V4.1 (#57152)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-16 18:09:00 +00:00 |
|
 Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
 Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
Nicolò Lucchesi
|
804e5377aa
|
[Bugfix][HiSparse][PD] Align region-mapped pulls across logical block sizes (#57077)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-16 18:39:07 +02:00 |
|
 Robert ShawandClaude
|
42919b49c5
|
[P/D] Report prefill worker cache hits in prompt_tokens_details (#54222)
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-16 15:23:31 +00:00 |
|
Nicolò Lucchesi
|
8b1d188046
|
[Bugfix][Mooncake] Report request-level KV load failures under HMA (#56855)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-16 14:55:40 +02:00 |
|
 
|
f37c550bf6
|
[Feature][Frontend] Expose effective attention block size for DCP (#56538)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-16 08:29:43 +00:00 |
|
Li, Jiang
|
b68408043d
|
[Bugfix][CPU] Add per-tensor FP8 W8A16 kernel to fix Ministral crash on CPU (#56985)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-09-16 15:49:17 +08:00 |
|
 MINJUN GILandOr Ozeri
|
f12fe10c5c
|
[Bugfix][KV Offload] Track cache recency once per request (#51787)
Signed-off-by: mindungil <alswnsrlf12@naver.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-16 10:47:26 +03:00 |
|
Wei Zhao
|
f30a195bbb
|
[Bugfix] Fix incorrect Mamba block allocation estimate that prevents request admission (#57050)
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-09-16 07:23:20 +00:00 |
|
 ![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
18553c5c9f
|
[Kernel] Use Murmur3 RNG for Gumbel sampling (#51367)
Signed-off-by: freyfwt <110148731+freyfwt@users.noreply.github.com>
Signed-off-by: freyfwt <freytian1996@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-09-15 23:52:26 -07:00 |
|
 
|
0f8fa53acf
|
[Bugfix][KV Offload] Check cgroup memory before SHM allocation (#54014)
Signed-off-by: Alex <jihui.huang@daocloud.io>
Signed-off-by: AlexHuang <alex.tech.lab@outlook.com>
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
|
2026-09-16 09:17:50 +03:00 |
|
 Jiangyun ZhuandTJian
|
4fe9e6f6e5
|
[Refactor][GLM-5.3-Flash] Move sparse_attn_indexer_kpool into the model folder and split AMD/NVIDIA (#55358)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
|
2026-09-16 05:50:35 +00:00 |
|
 
|
961c5b6b7e
|
[Bugfix][KV Offload] Restore MTP-retained sliding-window history (#56709)
Signed-off-by: jacklin78911-collab <jacklin78911@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-16 08:14:09 +03:00 |
|
alberto
|
75dc588269
|
[KV Offload] Add per-request max_load_tokens control (#55885)
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
|
2026-09-16 06:57:38 +03:00 |
|
 Qiming ZhangandKunshang Ji
|
4ace5cd9d8
|
[XPU] fix_test_worker_memory_snapshot_for_xpu (#50097)
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-09-16 11:05:43 +08:00 |
|