10 Commits
Author SHA1 Message Date
7b6310c109 [Perf][Sampler] Speed up the small-batch split top-p pipeline (#61996)
Signed-off-by: Bowen Wang <bowen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
2026-10-16 21:22:11 -07:00
5302d1fe40 [Model Runner V2] Support custom logits processors (#56497)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
2026-09-19 17:46:49 +00:00
Woosuk KwonandOpenAI Codex 2671fedfc7 [MRV2] Revert explicit Triton JIT warmup migration (#56654)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-13 04:07:56 +00:00
aed894c190 [6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-12 17:24:24 -07:00
Wentao Ye 2ecf7d0eb4 [Model Runner V2] Fix openai.InternalServerError: Error code: 500 - 'list index out of range' (#45467)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-06-13 01:44:16 -07:00
Nick Hill ec8d60bea8 [Model Runner V2] Use FlashInfer sampler (#42472) 2026-06-03 07:59:31 -07:00
Andy Lo 0a7165fd71 [ModelRunnerV2] Rename sampler functions and variables for clarity (#35459)
Signed-off-by: Andy Lo <andy@mistral.ai>
2026-03-02 19:48:56 -08:00
Woosuk Kwon 9ca768c740 [Model Runner V2] Minor cleanup for Sampler (#34563)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-02-14 18:29:03 -08:00
Nick Hill e535d90deb [ModelRunner V2] Misc minor simplifications and optimizations (#33467)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-02-01 22:17:14 +00:00
Woosuk Kwon 90c0836902 [Model Runner V2] Refactor Sampler (#32245)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
2026-01-13 17:58:12 -08:00