|
763e3f7c91
|
[Bugfix][Spec Decode] Disable dynamic speculative decoding for fixed-K proposers instead of crashing (#46399)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-10-17 12:41:04 -04:00 |
|
|
Ting SUN
|
eb8de8d1a1
|
[Bugfix][OpenAI] Respect allowed_token_ids in beam search (#47630)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-10-17 08:50:40 +00:00 |
|
|
Ting SUN
|
f4114e068e
|
[Bugfix][Frontend] Preserve token IDs in chat derender responses (#61065)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-10-12 07:55:06 +00:00 |
|
|
Ting SUNandCyrus Leung
|
b99fdbe1f8
|
[Bugfix] Respect the configured offline chat template (#61061)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-11 16:27:21 +00:00 |
|
|
|
51144dd83e
|
[Bugfix][Spec Decode] Trim token_ids/logprobs left past a stop string under speculative decoding (#47616)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-09 14:22:31 -07:00 |
|
|
Ting SUN
|
0f112d1800
|
[Bugfix][Pooling] Preserve cache_salt for chunked embeddings (#47696)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-10-08 07:33:28 +00:00 |
|
|
Ting SUNandFlora Feng
|
4a705c7b54
|
[Bugfix][Frontend] Preserve logprobs when top_logprobs is null (#47838)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-05 22:43:31 +00:00 |
|
|
Ting SUNandFlora Feng
|
50b404e71e
|
[Bugfix][Frontend] Preserve abort finish_reason for scale-out token streams (#47933)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-04 02:57:26 +00:00 |
|
|
Ting SUNandFlora Feng
|
fcf4750466
|
[Bugfix][Frontend] Constrain forced named tool choice with empty parameters to a JSON object (#45290)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-09-30 05:44:35 +00:00 |
|
|
Ting SUNandFlora Feng
|
329e6eee3f
|
[Bugfix][Frontend] Report named Anthropic tool calls as tool_use (#47598)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-09-30 05:32:38 +00:00 |
|
|
Ting SUN
|
b33c8a657d
|
[Bugfix][Frontend] Keep length finish_reason for max_tokens-truncated streaming tool calls (#46303)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-09-23 14:31:10 +08:00 |
|
|
Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
a33b3bac5d
|
[Bugfix][Qwen2.5-VL] Honor video fps for temporal M-RoPE (#47736)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-22 05:56:12 +00:00 |
|
|
Ting SUN
|
50708f95ae
|
[Bugfix][OpenAI] Fix streamed completion logprob offsets with echo (#47815)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-08-27 18:19:07 +08:00 |
|
|
Ting SUN
|
0303f37a54
|
[Bugfix][Pooling] Align CrossEncoder token type ids after truncation (#47772)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-07-08 03:59:22 +00:00 |
|
|
Ting SUNandNick Hill
|
8b91cd5b20
|
[Bugfix][Core] Close underlying iterator in merge_async_iterators single-iterator fast path (#44726)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-07 05:13:41 -07:00 |
|
|
|
b4cfbc24d3
|
[Bugfix][Core] Fix host memory leak from undrained new_block_ids (#44490)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-07-07 07:32:55 +00:00 |
|
|
Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
373eb314af
|
[Bugfix][Core] Fix num_output_placeholders underflow with async scheduling + spec decode (#46066)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-06 13:50:38 +00:00 |
|
|
Ting SUN
|
8974ed89cd
|
[Bugfix][Voxtral Realtime] Fix token feedback timeout silent hang (#44461)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-07-05 05:42:36 -07:00 |
|
|
|
fb2faceacd
|
[Bugfix][Model] Fix crash loading Mamba/Mamba2 checkpoints without an architectures field (#46037)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Ting SUN <suntcrick@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-05 05:42:32 -07:00 |
|
|
Ting SUN
|
9226613043
|
[Bugfix][Pooling] Forward instruction to Jina reranker scoring prompts (#47590)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-07-05 05:39:13 +00:00 |
|
|
Ting SUN
|
91b5647300
|
[Bugfix][Model] Allow Run:ai memory_limit sentinel values (#47337)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-07-05 00:08:34 +00:00 |
|
|
Ting SUN
|
d2afe39647
|
[Bugfix][Frontend] Preserve default sampling params in batch chat (#47597)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-07-04 19:06:39 +00:00 |
|
|
Ting SUNandCyrus Leung
|
379950191f
|
[Bugfix][Multimodal] Normalize direct PIL image inputs (#47566)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-07-03 14:27:14 -07:00 |
|
|
Ting SUNandBugen Zhao
|
1aeabec355
|
[Bugfix][Rust Frontend] Tolerate out-of-vocab prompt ids in detokenizer (#44682)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-03 03:41:53 +00:00 |
|
|
Ting SUN
|
ac521f6237
|
[Bugfix][Structured Outputs] Reject degenerate structured_outputs that crash EngineCore (#45346)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-30 22:41:33 +00:00 |
|
|
Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
f1a6703edd
|
[Bugfix][Config] Keep pydantic validation for fields with a TYPE_CHECKING Literal alias (#46220)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-24 12:25:50 +00:00 |
|
|
Ting SUNandFlora Feng
|
899d72a58c
|
[Bugfix][ToolParser] Handle braces in required tool streaming strings (#45389)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-06-23 20:29:34 +00:00 |
|
|
Ting SUN
|
a46f3eb232
|
[Bugfix][Model Runner V2] Preserve all allowed_token_ids in the logit bias kernel (#46245)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-23 07:01:13 +00:00 |
|
|
Ting SUN
|
12fe2a9aac
|
[Bugfix][Qwen3-VL] Fix multi-video crash with list-valued fps/num_frames (#46305)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-21 14:31:23 -07:00 |
|
|
Ting SUN
|
183a430c13
|
[Bugfix][Model Runner V2] Fix min_tokens off-by-one in the V2 GPU sampler (#46243)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-21 05:06:49 +00:00 |
|
|
Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
79ca54d221
|
[Bugfix][Quantization] Don't reject fp8_e5m2 KV cache for non-fp8 quantized checkpoints (#45040)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-18 14:18:25 -04:00 |
|
|
Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
702214146c
|
[Bugfix][Frontend] Fix Anthropic count_tokens decorator order driving server load negative (#44725)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-17 23:56:46 -07:00 |
|
|
Ting SUN
|
9d4b87f4f0
|
[Bugfix][Model] Validate DefaultModelLoader / LoadConfig and fail with clear errors (#45196)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-17 21:46:33 +00:00 |
|
|
Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
a46abb7ae6
|
[Bugfix][Quantization] Reject unsupported compressed tensors KV cache schemes (#45312)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-17 05:08:41 +00:00 |
|
|
Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
7b5d60cc37
|
[Bugfix][V1] Clean up compiled-model bytecode hooks on VllmRunner exit (#45195)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-16 20:31:17 -07:00 |
|
|
Ting SUNandShengqi Chen
|
c45f681932
|
[Bugfix][Core] Fall back when numactl --membind is blocked in constrained containers (#45438)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
|
2026-06-16 09:49:56 -07:00 |
|
|
Ting SUN
|
48df95c43e
|
[Feature][Frontend] Report multimodal token counts in usage.prompt_tokens_details (#45458)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-15 05:20:58 +00:00 |
|
|
Ting SUN
|
3d6ce816f0
|
[Bugfix][Model] Validate runai_streamer model_loader_extra_config (#45291)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-14 19:23:30 -07:00 |
|
|
Ting SUN
|
c1076839c9
|
[Bugfix][Model] Pass revision by name in Run:ai and bitsandbytes index downloads (#45308)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-11 20:21:46 -07:00 |
|
|
Ting SUNandBugen Zhao
|
1f9dd7900d
|
[Bugfix][Rust Frontend] Validate out-of-vocab token ids in request params (#44680)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-11 03:14:11 -07:00 |
|
|
Ting SUN
|
2d481f8a94
|
[Bugfix][Rust Frontend] Stop unescaping XML-style tool-call parameter values (#45025)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-10 19:05:23 -07:00 |
|
|
Ting SUNandBugen Zhao
|
ebf53ba373
|
[Bugfix][Rust Frontend] Set a structured-output backend so requests do not 500 (#44729)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-08 22:30:54 -07:00 |
|
|
Ting SUN
|
ca73293fa6
|
[Bugfix][Rust Frontend] Fix UTF-8 char-boundary panic in incremental detokenizer (#44620)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-05 07:36:17 +00:00 |
|