 Taneem Ibrahim
|
bbd7c24d6e
|
[MRV2] Validate MRV2 entrypoint logits processors (#57728)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-09-24 09:07:26 +00:00 |
|
|
Wentao Ye
|
fae5cbfc8b
|
[Refactor] Remove dead code multiple places (#58002)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-09-22 16:36:03 -04:00 |
|
|
|
0b7f11a1ee
|
[AuxOutput] Add block-keyed storage for routed-expert outputs (#45635)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-21 10:25:05 +00:00 |
|
|
liuzhenweiandKunshang Ji
|
6654bcbfca
|
[MRV2][XPU] use xpu sample kernel in mrv2 sampler (#57277)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-09-21 13:24:41 +08:00 |
|
|
|
5302d1fe40
|
[Model Runner V2] Support custom logits processors (#56497)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-19 17:46:49 +00:00 |
|
|
Jiang WuandJiang Wu
|
6af44e2887
|
[Bugfix][CPU] Fix DeepSeek V4.1 import without Triton (#57654)
Signed-off-by: Jiang Wu <245355730+JiangLLM@users.noreply.github.com>
Co-authored-by: Jiang Wu <245355730+JiangLLM@users.noreply.github.com>
|
2026-09-19 05:46:13 +00:00 |
|
|
Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
|
Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
|
|
18553c5c9f
|
[Kernel] Use Murmur3 RNG for Gumbel sampling (#51367)
Signed-off-by: freyfwt <110148731+freyfwt@users.noreply.github.com>
Signed-off-by: freyfwt <freytian1996@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-09-15 23:52:26 -07:00 |
|
|
Woosuk KwonandOpenAI Codex
|
2671fedfc7
|
[MRV2] Revert explicit Triton JIT warmup migration (#56654)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-13 04:07:56 +00:00 |
|
|
|
aed894c190
|
[6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-12 17:24:24 -07:00 |
|
|
|
7ee8a6dd01
|
[watermarking] Dual-key gumbel-max watermarking for speculative decoding support (#56122)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
|
2026-09-12 18:42:01 +00:00 |
|
|
|
ea40bb9e90
|
[feature] Watermarked generation and detection (Gumbel-max algorithm) (#54053)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-09 21:00:15 -07:00 |
|
|
|
560ef78bfe
|
[Perf][Model Runner V2] Compact sampling masks on GPU instead of unpacking the full-vocab bitmask on CPU (#54901)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: vx120 <893600387@qq.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: vx120 <893600387@qq.com>
|
2026-09-03 19:03:36 -07:00 |
|
|
Artem Perevedentsev
|
39e276eaeb
|
[Structured Output] Let terminal grammars stop under min_tokens (II) (#54218)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
|
2026-08-31 20:39:27 +04:00 |
|
|
Giancarlo Delfin
|
fe755c8899
|
[Bugfix][Model Runner V2][Spec Decode] Decouple the draft's gumbel noise stream from the target's (#54282)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-08-29 09:58:29 -07:00 |
|
|
Naveenraj KamalakannanandBenjamin Chislett
|
d5cadcee86
|
[Feature][DSpark]: Logprobs adaptive verification (#52242)
Signed-off-by: Naveenraj Kamalakannan <therealnaveenkamal@gmail.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-08-25 10:22:33 -04:00 |
|
|
Giancarlo Delfin
|
d154d90d6c
|
[Model Runner V2] batch-sharded sample (#50465)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-08-24 15:49:57 -07:00 |
|
|
Zihan Zhang
|
b389ac2946
|
[Spec Decode] DFlash2: local convolution + candidate selector (#52816)
|
2026-08-20 22:27:22 -07:00 |
|
|
|
a1c5b1fd9f
|
[Core][V1] Support trace_decode_token_ids for deterministic decode replay (#46701)
Signed-off-by: ianzxliu <ianzxliu@tencent.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Zhixia Liu <liuzhixia.nju@gmail.com>
Co-authored-by: ianzxliu <ianzxliu@tencent.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-20 15:12:43 +08:00 |
|
|
Giancarlo Delfin
|
d4f4d3f40f
|
[Model Runner V2][Spec Decode] Fix draft logits cache column stride in gumbel_sample (#53017)
|
2026-08-19 17:39:30 -07:00 |
|
|
Cheng RuiandRui Rui4 Cheng
|
6664d397bf
|
[Performance][MRV2] Cache logits-processing request state (#52329)
Signed-off-by: Rui Rui4 Cheng <chengrui4@lenovo.com>
Co-authored-by: Rui Rui4 Cheng <chengrui4@lenovo.com>
|
2026-08-17 02:36:41 +00:00 |
|
|
|
1b079c40ff
|
[Bugfix][Model Runner V2][Spec Decode] Fix off-by-one in bad_words draft-prefix matching (#52311)
Signed-off-by: jyan <r_02213@sjtu.edu.cn>
Co-authored-by: jyan <r_02213@sjtu.edu.cn>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-16 09:02:20 +00:00 |
|
|
|
50ba4bc6b2
|
[Feature] Mask Replay (#49577)
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-13 08:44:31 +00:00 |
|
|
|
b64a2708b0
|
Bound generation inputs before expensive work (#51447)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-11 22:03:50 +08:00 |
|
|
ChaunceyandNick Hill
|
72c0d67657
|
[Feat] Support thinking_token_budget in Model Runner V2 (#46727)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-07 03:36:18 +00:00 |
|
|
Giancarlo Delfin
|
81bc196913
|
[Model Runner V2] Cache draft logits in model's LM head dtype (#50910)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-08-05 19:12:46 -07:00 |
|
|
|
d2ca3002d9
|
[MRV2][Performance] Skip no-op FP32 logits materialization (#47711)
Signed-off-by: jesse <szxfml@gmail.com>
Signed-off-by: Song Zhixin <szxfml@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-07-27 15:31:36 +00:00 |
|
|
Michael GoinandNick Hill
|
ac36a7a1e7
|
[MRV2][Spec Decode] Avoid rejection sampler OOM by chunking (#48630)
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-23 18:13:13 +08:00 |
|
|
vllm-agentandvllm-agent
|
1134545b6f
|
Revert "[Sampler] Stop upcasting logits to fp32 in apply_sampling_params" (#48641) (#49033)
Co-authored-by: vllm-agent <vllm-agent@users.noreply.github.com>
|
2026-07-21 09:36:45 +01:00 |
|
|
|
425c4eafb0
|
[Sampler] Stop upcasting logits to fp32 in apply_sampling_params (#48641)
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-17 18:15:20 -07:00 |
|
|
|
41ea2dd44a
|
[Bugfix][V1/V2] Fix prompt_logprobs to respect logprobs_mode (#47680)
Signed-off-by: Wojciech Wais <wojciech.wais@gmail.com>
Signed-off-by: Federico Kamelhar <209537060+fede-kamel@users.noreply.github.com>
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: Wojciech Wais <wojciech.wais@gmail.com>
Co-authored-by: Federico Kamelhar <209537060+fede-kamel@users.noreply.github.com>
|
2026-07-17 21:58:59 +01:00 |
|
|
|
f5a8d73377
|
[Spec Decode] DSpark (#46995)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-07-01 08:30:24 -07:00 |
|
|
Giancarlo Delfin
|
3cecee40f3
|
[Model Runner V2][Spec Decode] Fix stale values in idx_mapping from CG num reqs padding (#47066)
|
2026-06-30 11:25:32 -07:00 |
|
|
Giancarlo Delfin
|
b6caeb5a09
|
[Model Runner V2][Spec Decode] Use fp32 uniform threshold for acceptance (#46878)
|
2026-06-27 14:09:25 -07:00 |
|
|
Nick Hill
|
32bb3195f0
|
[ModelRunner V2] Bound memory for large logprobs requests (#46746)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-25 18:04:06 -07:00 |
|
|
|
3c43237233
|
[Bugfix][Model Runner V2][Spec Decode] Fix int32 offset overflow in sampler kernels (#46560)
Signed-off-by: xiaojun.wei <jessiewei747@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-06-24 11:00:57 -07:00 |
|
|
Ting SUN
|
a46f3eb232
|
[Bugfix][Model Runner V2] Preserve all allowed_token_ids in the logit bias kernel (#46245)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-23 07:01:13 +00:00 |
|
|
Ting SUN
|
183a430c13
|
[Bugfix][Model Runner V2] Fix min_tokens off-by-one in the V2 GPU sampler (#46243)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-21 05:06:49 +00:00 |
|
|
Woosuk Kwon
|
16908e132e
|
[MRV2] Make FP32 Gumbel sampling more accurate (#45996)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-06-18 19:42:09 +00:00 |
|
|
Wentao Ye
|
2ecf7d0eb4
|
[Model Runner V2] Fix openai.InternalServerError: Error code: 500 - 'list index out of range' (#45467)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-06-13 01:44:16 -07:00 |
|
|
|
eb28452b10
|
[Model] Add DiffusionGemma Support (#45163)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Martin Kukla <martin.kukla@cantab.net>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Dipika Sikka <dsikka@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Alec Kohlhoff <134344302+aleckohlhoff@users.noreply.github.com>
Co-authored-by: Porras Huang <20535584+porrashuang@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com>
|
2026-06-11 22:17:35 -07:00 |
|
|
Benjamin ChislettandGiancarlo Delfin
|
0bae1d3848
|
[MRV2][Spec Decode] DFlash (#44586)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-06-10 08:47:46 -07:00 |
|
|
Nick Hill
|
ec8d60bea8
|
[Model Runner V2] Use FlashInfer sampler (#42472)
|
2026-06-03 07:59:31 -07:00 |
|
|
Nick Hill
|
e15f20258b
|
[ModelRunnerV2] Avoid pipeline parallel bubbles (#42187)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-02 14:02:01 -07:00 |
|
|
Wentao Ye
|
e26736973a
|
[Model Runner V2] Fix prompt logprobs calculation Sizes of tensors must match error (#42778)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-05-18 08:27:21 -07:00 |
|
|
PatchyTISandPatchouliTaisa
|
0162596603
|
[Model Runner V2] FP32 gumbel sampling. (#41775)
Signed-off-by: PatchouliTaisa <patchychen@tencent.com>
Co-authored-by: PatchouliTaisa <patchychen@tencent.com>
|
2026-05-15 09:20:08 -07:00 |
|
|
Lucas Kabela
|
4e498b5e5c
|
[Bugfix][Performance Improvement] Improve penalties triton kernel performance (#40657)
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
|
2026-05-12 05:47:20 +00:00 |
|
|
|
d7af6b34d8
|
[Model Runner V2] Bug fix: logprob dtype int64/int32 issue (#41761)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-11 21:55:43 +00:00 |
|
|
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
bbee532988
|
[Perf][1/n] Eliminate various GPU<->CPU syncs (#41429)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-11 20:36:03 +00:00 |
|