 
|
f98ae6a9fd
|
[Bugfix][Core] Keep resumable sessions that stop while preempted (#58778)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Zong <yzong@redhat.com>
|
2026-10-17 09:08:14 +00:00 |
|
 
|
d486219fc2
|
[Bugfix][Core] Uncount stop-trimmed spec tokens from num_computed_tokens (#58668)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-15 12:50:06 -07:00 |
|
  
|
ed3050add4
|
[Core] Apply prefill_schedule_interval outside data parallelism (#54627)
Signed-off-by: lifulu <fululi12@amd.com>
Signed-off-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-10-15 11:03:13 -05:00 |
|
Taksh Kothari
|
12b005eefa
|
[Bugfix] Don't split a multimodal item that a prefill chunk starts exactly on (#52412)
Signed-off-by: Taksh <takshkothari09@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-13 15:32:38 +00:00 |
|
 ztysthandOpenAI Codex
|
1bdd3fa2d5
|
[Bugfix][Core] Check multimodal chunk boundaries before encoder cache reuse (#59721)
Signed-off-by: ztysth <rnk.liulw@gmail.com>
Signed-off-by: ztysth <166926442+ztysth@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-10-12 19:25:35 +00:00 |
|
 Woosuk KwonandClaude Opus 5.5
|
e1a5086256
|
[Attention] Remove cascade attention (#61126)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-12 10:41:07 -07:00 |
|
bzsuni
|
01abe975b5
|
[Bugfix][KV Connector] Fix speculative block reservations across KV cache groups (#59880)
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
|
2026-10-10 20:55:41 -07:00 |
|
 Matthew BonanniandClaude Opus 5.5
|
d5f356dae0
|
[HiSparse] Own GPU residency instead of editing core block-table rows (#60647)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-09 21:25:44 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)    
|
d8fa463e2b
|
[Bugfix][Prefix caching] Fix inconsistent hybrid Mamba prefix cache boundaries (#60533)
Signed-off-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-08 18:33:48 +00:00 |
|
 
|
7816afdb25
|
[Bugfix][Core] Drain async KV loads in pause(mode="wait"), and pause the drain test only once a request is in flight (#60572)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-08 09:33:41 +00:00 |
|
 
|
ef63c23d35
|
[Bugfix][DSv4.1] Skip SWA bounded replay for KV loads that carry the window (#59197)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 15:28:42 -07:00 |
|
 
|
e6fc81bc78
|
[Bugfix][Scheduler] Let pooling chunked prefill use full context (#48039)
Signed-off-by: Tommy <tran.tommy@hotmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-10-07 07:32:00 +00:00 |
|
  
|
e37e51dd24
|
[Metrics] Expose cached prompt tokens by cache tier (#56318)
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
Signed-off-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-06 14:52:05 -04:00 |
|
  
|
2b32d9ac6a
|
[Bugfix][Core] Drop stale block hashes when a session truncates tokens (#59103)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Co-authored-by: Brian Woods <me@brian.biz>
Co-authored-by: Change72 <changg@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-10-06 02:51:03 +00:00 |
|
 bzsuniandcjackal
|
528772a4bf
|
[Bugfix][Core] Retain encoder cache references for repeated multimodal inputs (#59942)
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
|
2026-10-05 15:52:46 +00:00 |
|
 
|
ca0df1f122
|
[Bugfix][Core] Exempt exactly the blocks an async KV load writes from zeroing (#59504)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-03 06:26:12 +00:00 |
|
 
|
d58da79de7
|
[Metrics][P/D] Add KV-fetch stage gauges for async KV loads (#58874)
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-02 01:24:33 +00:00 |
|
   
|
34c458d27f
|
[Bugfix] Don't let a structured-output request sample from an unmasked row (#54442)
Signed-off-by: ArcheyChen <45146080+ArcheyChen@users.noreply.github.com>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: GPT-5.6 <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-10-01 02:11:44 +00:00 |
|
 kliuaeandNick Hill
|
7e583e615c
|
[Bugfix][Core] Fix mamba prefill checkpoint block reservation and prompt-end eviction in align mode (#59175)
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-30 21:26:54 +00:00 |
|
 Giulio De PasqualeandIsotr0py
|
2e52a558f0
|
[Bugfix][Core] Schedule encoder-only prompts larger than one step (#59029)
Signed-off-by: Giulio De Pasquale <git@depasquale.giugl.io>
Co-authored-by: Isotr0py <2037008807@qq.com>
|
2026-09-30 22:04:28 +08:00 |
|
Mia Ojeda
|
e006d761a5
|
[Bugfix][CPU][DiffusionGemma] Support narrower canvas w/ sync scheduling (#59107)
Signed-off-by: ojeda-e <estefania.b.ojeda@gmail.com>
|
2026-09-29 23:28:50 -04:00 |
|
 aoshen02andOpenAI Codex
|
4861833ae2
|
[Bugfix][Core] Allow AuxOutput reset after abort without running requests (#59060)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-29 09:23:32 +00:00 |
|
 0xsenseiandOpenAI Codex
|
d28795f1a7
|
[Bugfix][Scheduler] Refresh max tokens for streaming continuations (#57676)
Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-28 20:31:57 +00:00 |
|
 Nick HillandRobert Shaw
|
42dbe4578c
|
[Core] Rework scheduler skipped_waiting queue (#58947)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-09-28 20:03:16 +00:00 |
|
0xsensei
|
2840ca7a33
|
[Bugfix][Scheduler] Preserve logprobs across streaming continuations (#57447)
Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com>
|
2026-09-28 19:50:27 +00:00 |
|
yzong-rh
|
2f4a975901
|
[Bugfix] Fix resumable request + async scheduling handoff race (#58259)
Signed-off-by: Yifan Zong <yzong@redhat.com>
|
2026-09-28 15:26:57 -04:00 |
|
 
|
b721a4c709
|
[Feature] Add fixed-token prefill scoring (#54335)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-09-28 07:50:09 -07:00 |
|
 YiSheng5andKunshang Ji
|
b05f7d5fbc
|
[PP][XPU]Add the flag to control microbatch feature on MRV2+PP (#55145)
Signed-off-by: yisheng <yi.sheng@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-09-25 12:05:35 +08:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) jackLeiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
bbc4ddeca9
|
[Bugfix][V1] Reject encoder-cache hits with mismatched embedding counts (#57696)
Signed-off-by: jackLei0901 <42642542+jackLei0901@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-24 08:40:47 +00:00 |
|
Robert Shaw
|
6697a7dd30
|
[Scheduler] Tune --long-prefill-token-threshold adaptiveness (#58459)
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
|
2026-09-23 22:23:18 -04:00 |
|
 Nicolò LucchesiandYifan Qiao
|
9488318c6a
|
[Mooncake] Address review nits from #56855 (#57174)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Yifan Qiao <17067717+ivanium@users.noreply.github.com>
|
2026-09-23 10:15:45 +02:00 |
|
   
|
1b3b88ec2b
|
[Core] structured generation mode for DiffusionGemma model (Jev-like) (#57250)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Razorback16 <razorback16@protonmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Razorback16 <razorback16@protonmail.com>
|
2026-09-22 20:51:17 +00:00 |
|
 Robert ShawandRobert Shaw
|
c64b15cde5
|
[Scheduler] Soften Long Prefill Tokens Threshhold (#57951)
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
|
2026-09-21 17:19:42 -07:00 |
|
Cyrus Leung
|
c8824a1e42
|
[MM] Move supports_multimodal_inputs and cache out of registry (#57913)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-09-21 07:43:07 -07:00 |
|
    
|
0b7f11a1ee
|
[AuxOutput] Add block-keyed storage for routed-expert outputs (#45635)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-21 10:25:05 +00:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) roikoren755andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
49ee12d742
|
[Misc] Rename --enable-mamba-fine-grained-prefix-cache (#53945 follow-up) (#57382)
Signed-off-by: Roi Koren <roik@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-20 09:50:34 -07:00 |
|
 
|
1dc2d854c1
|
[Feat][Model] Support encoder-side SWA-bounded replay for DeepSeek-V4.1-Flash (#56227)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-09-18 15:01:24 -07:00 |
|
 aoshen02andaoshen02
|
9ff08f9493
|
[Bugfix] Honor skip_reading_prefix_cache for KV connector hits (#57269)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
|
2026-09-17 03:46:28 -07:00 |
|
omerpaz95
|
fa3622a4a1
|
[EC Connector] Add Metrics Collection (#54960)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
|
2026-09-17 10:36:23 +08:00 |
|
 Chuan (Richard) LiandCursor
|
f730a93d2b
|
[Scheduler] Add --max-num-active-seqs to cap RUNNING admission (#56758)
Signed-off-by: Li <chuali@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-09-17 10:22:12 +08:00 |
|
 Robert ShawandClaude
|
6c3d9816af
|
[BugFix][KV Connector] Fix Deadlock with KVConnector + MTP under KV Pressure (#57104)
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-16 22:22:07 +00:00 |
|
 Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
 Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
 Robert ShawandClaude
|
42919b49c5
|
[P/D] Report prefill worker cache hits in prompt_tokens_details (#54222)
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-16 15:23:31 +00:00 |
|
Nicolò Lucchesi
|
8b1d188046
|
[Bugfix][Mooncake] Report request-level KV load failures under HMA (#56855)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-16 14:55:40 +02:00 |
|
Giancarlo Delfin
|
85c1f58d50
|
[BugFix][Model Runner V2][Spec Decode] Fix decode instance's multi-layer MTP kv caches during P/D (#55055)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-09-15 17:31:25 -07:00 |
|
yzong-rh
|
e6960af33b
|
[Refactor] StructuredOutputManager x Speculative Decoding Refactor (#48200)
|
2026-09-15 09:48:49 -04:00 |
|
 
|
84e5f3edac
|
[Bugfix] Validate routed-expert prompt offsets before engine submission (#56844)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Bhushan Asati <bhushanasati25@gmail.com>
Co-authored-by: Megha Agarwal <19240983+meghaagr13@users.noreply.github.com>
|
2026-09-15 09:17:41 +08:00 |
|
Kebe
|
c351fd3c64
|
[Core] Fix ValueError on KV load failure with a hybrid KV cache (#50388)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2026-09-13 21:03:05 +08:00 |
|
      
|
d43bb2f37f
|
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-09-12 00:36:47 -04:00 |
|