 stefankoncarevic
|
3ca00a8261
|
[CI] Skip EmbeddingGemma2 pooling tests needing transformers >= 5.19.0 (#60371)
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
|
2026-10-07 10:22:31 +00:00 |
|
|
SeongJun Lee
|
6b75dfb8a1
|
[Misc] Report a missing DeepSelect extension only when it is requested (#60242)
Signed-off-by: lesj0610 <lesj0610@gmail.com>
|
2026-10-07 09:53:17 +00:00 |
|
|
Kevin H. LuuandClaude Opus 5.5
|
02d8a94e61
|
[CI][ROCm] Skip DSv4.1 decoder replay graph test on ROCm (#60378)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 02:38:31 -07:00 |
|
|
|
df417f780a
|
[Bugfix][MoRIIO] Exclude synchronous READ destinations from KV zeroing (#59164)
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: Hexiang Wang <56632993+whx-sjtu@users.noreply.github.com>
Co-authored-by: Yichao Zhu <Yichao.Zhu@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-07 16:35:56 +08:00 |
|
|
|
42ecc8eebe
|
[CI][RL] Consolidate RL entrypoint tests under tests/entrypoints/rl (#59948)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 08:29:09 +00:00 |
|
|
Bugen ZhaoandClaude Opus 5.5
|
bc8dd4ad27
|
[Rust Frontend] Share SchemaRoot between argument coercion and grammars (#59408)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 08:28:27 +00:00 |
|
|
stefankoncarevic
|
90f4936d9e
|
[CI/Build] Widen the ragged prefill scoring tolerance, drop batch invariance (#60192)
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
|
2026-10-07 16:21:47 +08:00 |
|
|
Bugen ZhaoandClaude Opus 5.5
|
7436a7f119
|
[Rust Frontend] Backtrack from safe text at a complete marker (#59563)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 07:45:15 +00:00 |
|
|
|
e6fc81bc78
|
[Bugfix][Scheduler] Let pooling chunked prefill use full context (#48039)
Signed-off-by: Tommy <tran.tommy@hotmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-10-07 07:32:00 +00:00 |
|
|
|
cbee82324c
|
[Docs][RL] Move pause/resume and batch invariance examples to examples/rl (#59949)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 06:22:10 +00:00 |
|
|
SrirajandClaude
|
43b4aaea3e
|
[Bugfix][Frontend] Return 4xx for client errors in /cohere/v2/chat (#60309)
Signed-off-by: sricursion <sricursion@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-07 04:55:34 +00:00 |
|
|
Turner JabbourandClaude Sonnet 5
|
aecb717f10
|
[CI] Allowlist-shrink batch 2: wire 9 tests/models/ files into Buildkite, fix 2 stale bugs found (#59050)
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
|
2026-10-06 21:16:53 -07:00 |
|
|
Mohammad HijjawiandCyrus Leung
|
ef014bdf06
|
[Bugfix][Frontend] Return logprobs in Cohere Chat v2 responses (#59072)
Signed-off-by: Mohammad Hijjawi <mohammad.hijjawi1997@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-07 04:13:52 +00:00 |
|
|
Hyukjoon LeeandClaude Opus 5.5
|
2342343746
|
[ROCm][DCP] Default MLA DCP verify to the round-robin asm decode where it applies (#59965)
Signed-off-by: hyukjlee <hyukjlee@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 03:10:54 +00:00 |
|
|
|
8a26869bca
|
[Bugfix] Index Mamba align state copies by request slot in MRV2 (#60210)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Karl0007 <44745209+Karl0007@users.noreply.github.com>
Co-authored-by: Yongqin Wang <yongqinwang@roblox.com>
|
2026-10-07 02:32:46 +00:00 |
|
|
|
956960f85f
|
[Perf][MRV2] Avoid stream syncs in NonUvaBuffer host-to-device copies (#60215)
Signed-off-by: cheesecake <farzanaman99@gmail.com>
Co-authored-by: William Joyner <133391209+joynerwk03@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 01:51:12 +00:00 |
|
|
Robert ShawandClaude Code
|
283d76d56a
|
[Bugfix] Widen ep_gather output offset to int64 (#58373)
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
|
2026-10-07 01:45:04 +00:00 |
|
|
|
fe51194e2d
|
[Bugfix][MiniMax-M3] Fence decode indexer shared-memory stage reuse (#60194)
Signed-off-by: Vladimir Anisimov <vanisimov@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-10-07 01:35:12 +00:00 |
|
|
Kevin H. LuuandClaude Opus 5.5
|
4b4eb6277a
|
[CI] Remove AMD mirror of CPU-only Rust frontend cargo steps (#60323)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-06 18:30:52 -07:00 |
|
|
Micah WilliamsonandKevin H. Luu
|
13f8c453ee
|
[CI][Releases] Trigger perf-eval Through Release Pipeline (#52213)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-06 18:27:27 -07:00 |
|
|
Miłosz Grunwald
|
703c29179c
|
[TEST][XPU][CI] Generalize model runner prefill tail classification tests to all platforms (#60201)
Signed-off-by: microslaw <milosz.grunwald@intel.com>
|
2026-10-07 08:59:20 +08:00 |
|
|
SrirajandFlora Feng
|
db6e3cd8c4
|
[Bugfix][Frontend] Log the full request body when max_log_len is unset (#60310)
Signed-off-by: sricursion <sricursion@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-07 00:42:17 +00:00 |
|
|
jxjandOpenAI Codex
|
6c2925fe5f
|
[Bugfix][Core] Preserve speculative metrics when coalescing outputs (#55708)
Signed-off-by: git-jxj <65210887+git-jxj@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-10-07 08:35:14 +08:00 |
|
|
SrirajandFlora Feng
|
d77cf81156
|
[Bugfix][Frontend] Honor default penalties in batch chat completions (#60307)
Signed-off-by: sricursion <sricursion@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-07 00:21:00 +00:00 |
|
|
|
5ab4e4445b
|
[Perf][MoE] Skip padded work in block-FP8 DeepGEMM experts (#59128)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-06 18:49:57 -05:00 |
|
|
Cheese CakeandClaude Opus 5.5
|
b6d8e8afd9
|
[Bugfix][MRV2][Spec Decode] Accept num_speculative_tokens in NgramGPUSpeculator.propose (#60300)
Signed-off-by: cheesecake <farzanaman99@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-06 15:02:04 -07:00 |
|
|
|
ae19af883a
|
[CI] Require transformers 5.19 for the EmbeddingGemma2 registry test (#60295)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-10-06 14:50:10 -07:00 |
|
|
aditi-amd
|
385d86a6cf
|
[Attention] Add UltraQuant 4-bit KV cache backend (FlyDSL D=256) (#57057)
Signed-off-by: Aditi Rana <aditi.rana@amd.com>
|
2026-10-06 20:42:52 +00:00 |
|
|
|
819852df6b
|
[ModelRunner V2] Speculative Decoding NGram GPU Implementations (#40704)
Signed-off-by: PatchouliTaisa <patchychen@tencent.com>
Signed-off-by: PatchouliTaisa <pyramkar@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Xuanan Chen <xuananchenc@nvidia.com>
Co-authored-by: PatchouliTaisa <patchychen@tencent.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Xuanan Chen <xuananchenc@nvidia.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
|
2026-10-06 13:32:59 -07:00 |
|
|
Benjamin ChislettandClaude Opus 5.5
|
dac35c5b66
|
[Misc] Add chat-completion options to gsm8k_eval.py CLI (#60114)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-06 20:13:44 +00:00 |
|
|
PatrykSafferandMisha Goin
|
e43db1f5e2
|
Pad to 64 not 128 Q heads in flashMLA sparse (#60029)
Signed-off-by: Patryk Saffer <patryk.saffer@mistral.ai>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-06 16:03:05 -04:00 |
|
|
|
d1f3d8b870
|
[Quantization] Prefer Humming before Marlin backends on SM90 (#56997)
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-06 12:31:35 -07:00 |
|
|
Roger WangandKimi Code
|
bb87d227d4
|
[Bugfix][Model] Fix mypy safe-super error in EmbeddingGemma2Model (#60280)
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
|
2026-10-06 15:10:00 -04:00 |
|
|
|
e37e51dd24
|
[Metrics] Expose cached prompt tokens by cache tier (#56318)
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
Signed-off-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-06 14:52:05 -04:00 |
|
|
Nishant RevurandClaude
|
ad0f67a7ec
|
[Bugfix][Spec Decode] Keep heterogeneous-vocab draft models on Model Runner V1 (#59541)
Signed-off-by: Nishant Revur <kumrfni@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-06 18:03:44 +00:00 |
|
|
|
3403e0f176
|
[Bugfix] Persist FlashInfer autotune cache per rank to fix the rank-0-only cache-hit deadlock (#57635)
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-10-06 17:51:01 +00:00 |
|
|
Luciano MartinsandLuciano Martins
|
02b83919aa
|
[Model] Support EmbeddingGemma2 multimodal pooling architecture (#60254)
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
|
2026-10-06 13:43:38 -04:00 |
|
|
|
7b665f7a58
|
[Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) (#58181)
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-06 17:22:47 +00:00 |
|
|
Matthew BonanniandClaude Opus 5.5
|
e82b80099a
|
[Perf][HiSparse] Cache per-request residency instead of rescanning every page each step (#60083)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-06 13:04:55 -04:00 |
|
|
Wentao Ye
|
fba31f31a9
|
[GLM5.3 Perf] Switch to fp8 kv cache by default for GLM, 2.3%~5.5% E2E Throughput Improvement (#60140)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-10-06 12:54:20 -04:00 |
|
|
|
f8d92cae41
|
[Quantization] Layer re-quantization for linear layers through online quantization API (MXFP8 -> FP8 PTPC showcase) (#55684)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-06 11:38:18 -05:00 |
|
|
eky-amdandClaude Fable 5
|
29f955cce0
|
[ROCm] Fuse MLA dual RMSNorm + FP8 group quant for DeepSeek-R1 (#54857)
Signed-off-by: Ethan Ky <ethan.ky@amd.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-10-06 11:26:38 -05:00 |
|
|
stefankoncarevic
|
4516896626
|
[CI/Build] Warm the DP engines before measuring load balance in test_load (#60241)
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
|
2026-10-06 11:23:37 -05:00 |
|
|
Giancarlo Delfin
|
fcf53d240c
|
[Spec Decode] Remove eager metadata rebuild during MTP fused multi-step decode (#58463)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-10-06 12:17:30 -04:00 |
|
|
SeongJun Lee
|
4ea0c28bc4
|
[Bugfix] Warn only on per-request do_normalize/do_rescale overrides (#60240)
Signed-off-by: lesj0610 <lesj0610@gmail.com>
|
2026-10-06 15:55:27 +00:00 |
|
|
Jeff-Tseng-dev
|
aad4579bbe
|
Upgrade tpu-inference to v0.31.0 (#60190)
|
2026-10-06 15:44:56 +00:00 |
|
|
Chris LeonardandShengqi Chen
|
cc73cca9c8
|
[Build] Migrate vendored DeepGEMM from pybind to TORCH_LIBRARY (abi3) (#48962)
Signed-off-by: Chris Leonard <chleonar@redhat.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
|
2026-10-06 08:39:04 -07:00 |
|
|
ovidiusm
|
cbe1f9740d
|
[NIXL] Bump NIXL version to 1.5.0 (#56907)
Signed-off-by: Ovidiu Mara <ovidium@nvidia.com>
|
2026-10-06 16:55:03 +02:00 |
|
|
Bugen ZhaoandClaude Opus 5.5
|
0719975cee
|
[CI][Rust Frontend] Pass --locked to cargo binstall (#60182)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-06 22:53:17 +08:00 |
|
|
|
8bd7737a03
|
[Docs] Clarify that vLLM does not isolate tenants sharing a server (#59811)
Signed-off-by: SunnyR <sunny.rangnani@chainguard.dev>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Juan Pérez de Algaba <124347725+jperezdealgaba@users.noreply.github.com>
|
2026-10-06 22:39:59 +08:00 |
|