Commit Graph
20660 Commits
Author SHA1 Message Date
Jeff (Junze) MaandOpenAI Codex 30dd1a7954 [DecodeBenchConnector] Fix HMA cache-group mapping (#54647)
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-01 05:52:50 +00:00
Andreas Karatzas 40b2f62061 [ROCm][CI] Stabilize the sqrt-softplus top-k tie oracle (#54403)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
2026-09-01 05:34:40 +00:00
sychen52 7c5dc571cb [Attention][DSA] Enable W4A16 DSA (#51724)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
2026-09-01 12:37:47 +08:00
Rohan PotdarClaude Opus 4.8mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ce2e343be1 [ROCm] Keep GLM-5.2 on MRV1 and disable default breakable cudagraph (#53155)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-31 23:17:54 -05:00
afriedriandAndreas Karatzas dc9114b201 [ROCm][MoE] Split AITER CK and Triton MXFP4 W4A16 into separate backends (#50622)
Signed-off-by: afriedri <afriedri@amd.com>
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-08-31 23:14:14 -05:00
Gabriel WuYongye Zhumergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
446c769482 [Distributed] Add opt-in FlashInfer PCIe IPC all-reduce backend (#53576)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-01 03:34:56 +00:00
Roberto L. CastroandCodex e16b5e518d [1/N][warmup][DSv4] Migrate generic MLA metadata and indexing kernels (#50175)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
2026-08-31 23:26:05 -04:00
xiaolinchenandClaude Opus 4.8 188716ace7 [Bugfix][EC Connector] Fall back when MADV_POPULATE_WRITE is unsupported (#53190)
Signed-off-by: wentian-byte <2990624738@qq.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-01 03:17:45 +00:00
Amy (Yu-Hsuan) Lin 923949e6e3 [Feat] Add request-level preemption count histogram metric (#49984) 2026-08-31 23:04:54 -04:00
Lain 882ca8d696 [Kernel] add Flashinfer cutedsl w4a16 linear (#53014)
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
2026-09-01 02:49:14 +00:00
Andreas KaratzasandOpenAI Codex 907b1a7f22 [CI][ROCm] Avoid redundant image pulls during smoke validation (#54408)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-08-31 21:48:25 -05:00
Nick Hill c28feab989 [Core][MRV2] Freeze gc during V2 CG capture; skip per-descriptor cleanup (#54646)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-08-31 19:37:46 -07:00
Jorge Menjivar 225aec4809 [Rust Frontend] Migrate to new tekken crate (#53056)
Signed-off-by: Jorge Menjivar <jorge@menjivar.ai>
2026-09-01 02:35:21 +00:00
Flora Feng 45aed9b0cd [CI] Broaden tool-calling issue auto-labeling (#54650)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
2026-09-01 10:07:21 +08:00
22df3a34e0 [Perf][Rust Frontend] Count the tokenizer vocabulary once at construction (#54449)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: Feathbow <feathbow@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-09-01 01:25:06 +00:00
Zheng GongandCursor Agent 58dace61fa [Kernel] Make prefix-prefill tiling independent of the KV page size (#54194)
Signed-off-by: Zheng Gong <zgong@amd.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-09-01 09:19:55 +08:00
4c21d41733 [XPU] Route activation CustomOps to SYCL kernels (#53734)
Signed-off-by: Marceli Fylcek <marceli.fylcek@intel.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-09-01 09:19:52 +08:00
8600db5dff [CI] Build CPU image against torch nightly for TORCH_NIGHTLY runs (#48750)
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com>
Signed-off-by: Andrey Talman <atalman@fb.com>
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-08-31 17:25:57 -07:00
Zhenzhong Xu e29af0a2af [XPU] bump up auto-round-lib to 0.15.0 (#54515)
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com>
2026-09-01 08:24:28 +08:00
d4329ba53d [Bugfix][Rust Frontend] Fix adjacent DeepSeek V4 user content rendering (#53281)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-09-01 00:23:24 +00:00
Flora Feng 89df6fcb80 [CI] Broaden structured-output issue auto-labeling (#54645)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
2026-08-31 23:51:55 +00:00
AusarandClaude 6bafc049aa [Bugfix][PP] Never drop a decoding request from the sampled-token broadcast (#54436)
Signed-off-by: ArcheyChen <45146080+ArcheyChen@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-31 23:49:54 +00:00
xiangdongandKunshang Ji b05acd2ae0 [XPU] [CI] Add retry for v1/sample in Intel GPU CI (#53669)
Signed-off-by: zengxian <xiangdong.zeng@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-08-31 23:34:57 +00:00
Wentao Ye 91752b7a3e [K3 Bug] Fix Kimi-K3 RecoverSSM startup failure 'MambaAttentionBackendEnum.GDN_ATTN declares 4 states, but provides 2 state copy funcs' (#54634)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-08-31 23:22:49 +00:00
Wentao Ye 3a2ed6cbae [Kimi Bug] Fix gdn build_attn_metadata 'KimiK3KDAMetadataBuilder' object has no attribute 'layer_names' (#54636)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-08-31 19:21:01 -04:00
Yongye Zhu d6d6658543 [Kimi-K3][Perf] Make native CUDA AttnRes the SM100 default (#54261)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
2026-08-31 23:18:12 +00:00
mobicham 07ea9350ba [Kernel][Gemma4] Prune Triton sliding-window tiles for multimodal prefixes (#53147)
Signed-off-by: mobicham <hichamb@dropbox.com>
2026-08-31 22:21:22 +00:00
Zihan ZhangandClaude Opus 5 d61b6e1878 [Bugfix][Spec Decode] Take the DFlash draft's RoPE layout from its own config (#54373)
Signed-off-by: SubSir <tiancaizhangdaxian@sjtu.edu.cn>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 18:13:20 -04:00
Kevin H. Luu 24d42f3553 [CI] Mark 1-GPU L4 test steps with device: l4 for EKS migration (#54549)
Signed-off-by: khluu <khluu000@gmail.com>
2026-08-31 14:56:27 -07:00
Roberto L. CastroCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f5c3cc240b [Perf][Kernel] Tune cooperative topk for medium batch-sizes (#53382)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-31 13:44:38 -07:00
Juhi MittalandClaude Opus 5 85c1365bd9 [Bugfix] NemotronHMTP: add hf_to_vllm_mapper so quant exclusions reach the MTP draft (#53790)
Signed-off-by: Juhi Mittal <juhim@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 12:10:24 -07:00
Andreas Karatzas a9dc631429 [Bugfix] Reject empty bad-word tokenizations (#53433)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-08-31 12:07:17 -07:00
Andreas KaratzasandOpenAI Codex f9c7c6e090 [Rust Frontend][CI] Remove TCP port races from mock-engine tests (#54481)
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
2026-09-01 03:02:55 +08:00
Qiming Zhang f5e441de10 [Bugfix][Test] Fix off-by-one error in sampled token rank causing flaky logprobs test (#53976)
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
2026-08-31 18:49:43 +00:00
Joe RowellandMisha Goin 65ce85fcdc Add Laguna-XS-2.1-INT4 to nightly CI (#52961)
Signed-off-by: Joe Rowell <joe@poolside.ai>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
2026-08-31 14:22:00 -04:00
YukioZzzandandyluo7 2ba984a5d0 [ROCm][DSpark][DCP] Serve prefix cache hits under DCP for Kimi-K3 (#53598)
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
Co-authored-by: andyluo7 <andy.luo@amd.com>
2026-08-31 11:18:10 -07:00
dbb7fffddb [ROCm][MLA][DCP] Support causal multi-token verification (#51705)
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
Signed-off-by: billishyahao <bill.he@amd.com>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: billishyahao <bill.he@amd.com>
2026-08-31 10:42:28 -07:00
Artem Perevedentsev 39e276eaeb [Structured Output] Let terminal grammars stop under min_tokens (II) (#54218)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-08-31 20:39:27 +04:00
Nicolò Lucchesi e9dd6d4834 [CI] Exclude kv_transfer changes from broad spec-decode/kernels/multimodal triggers (#54365)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
2026-08-31 09:17:12 -07:00
rasmithandAndreas Karatzas 3593c964de [ROCm] Add TheRock preview docker updates, Keep Python 3.12 and Ubuntu 22.04 (#49925)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-08-31 11:09:46 -05:00
Fanli LinandKunshang Ji 82936c409d [Tests][XPU] Limit Qwen2-VL generation length to avoid flaky numerical divergence (#54172)
Signed-off-by: Lin, Fanli <fanli.lin@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-08-31 22:59:57 +08:00
Zhenzhong XuandKunshang Ji bd575a0d0b [AutoRound] Support AutoRound Format Block-Wise FP8 in vLLM (#47434)
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com>
Signed-off-by: Zhenzhong Xu <zhenzhong.xu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-08-31 22:56:20 +08:00
Nicolò Lucchesi dafbef15a1 [Core] Add max_num_queued_reqs and max_num_queued_tokens for queue size management (#49445)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
2026-08-31 14:26:49 +00:00
andyluo7andOpenAI Codex 76ff0cdff2 [Bugfix][ROCm] Preserve AITER unified-attention metadata during graph replay (#53821)
Signed-off-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-08-31 21:53:12 +08:00
c5d840ff6a [KV Connector][Offloading] Certify attention-only hybrids in the canonical portability gate (#51689)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
2026-08-31 15:53:19 +03:00
4c58a0c398 [Bugfix][KV Offload] Unlink /dev/shm region after all workers map it (barrier variant of #51317) (#52596)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
2026-08-31 15:53:00 +03:00
Uttam KumarandOr Ozeri e0d27040dd [Bugfix][KV Offload][P2P] Preserve aborted loads until abort completion (#52571)
Signed-off-by: Uttam Kumar <uttam.kumar912@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
2026-08-31 14:34:48 +03:00
Jee Jee Li e2c8eeac40 [kernel] Fused embedding kernel (#53677)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-08-31 11:34:14 +00:00
Itay EtelisItay Etelismergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
bed3280f50 [KV offload] Order CPU->GPU loads against the compute stream (#50696)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-31 14:31:18 +03:00
waizuichougou 810bc3250c [Frontend][Performance] Resolve async media across modalities concurrently (#54537)
Signed-off-by: waizuichougou <2082431897@qq.com>
2026-08-31 10:50:17 +00:00