d8eabdbfbe
[ROCm][MLA] Use asm decode for non-divisor small head counts ( #50578 )
...
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com >
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com >
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com >
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Markus Hartikainen <markus.hartikainen@amd.com >
Co-authored-by: Liuyinfeng01 <yinfeliu@amd.com >
2026-08-06 14:56:53 -04:00
7b9f2dad89
[Frontend] Watch frontend processes during engine startup ( #43417 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-06 11:17:24 -07:00
Divakar Verma
4f851bef6c
[ROCm][CI] Update AITER AR+RMS e2e fusion counts for final-norm coverage ( #51273 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
2026-08-06 13:05:16 -05:00
c56f169d9a
[Bugfix] Keep mamba align prefill chunks block-aligned past last_cache_position ( #51113 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Hernan <kodek@eat1337.com >
Co-authored-by: yanghui1-arch <3053034939@qq.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-08-06 10:21:00 -07:00
81be2e09ae
[Weight processing] Copy over new_data attributes in replace_parameter ( #49601 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 13:03:37 -04:00
b38e111d3e
[Attention][MLA] Per-request scheduling for MLA chunked context ( #50613 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 16:45:44 +00:00
Harry Mellor
e7b8d59460
Fully generalise input embedding handling in Transformers modelling backend ( #51247 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-06 16:21:24 +00:00
Michael Goin and OpenAI Codex
566c80edf9
[CI] Run basic fullgraph correctness on one GPU ( #51271 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-06 12:09:30 -04:00
gnovack
7b4ed49628
attn_res kernel latency improvements ( #50185 )
2026-08-06 08:53:20 -07:00
Chris Leonard and Shengqi Chen
41e7746b82
Update vllm to point to flash-attention commit that builds FA3 with torch stable API. (Retry) ( #49599 )
...
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Signed-off-by: Shengqi Chen <harry-chen@outlook.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-08-06 08:37:46 -07:00
Ning Xie
62a86318de
[VocabParallelEmbedding] fix extra_repr fields concat ( #51224 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
2026-08-06 05:22:18 -07:00
Liangqiusong
1e05b21d61
Remove the XPU branch of topk_softplus_sqrt ( #51242 )
...
Signed-off-by: xiaolong <xiaolong.guo@intel.com >
2026-08-06 12:18:45 +00:00
AlexHuang and OpenAI Codex
46e6a83ce1
[Bugfix][KV Offload] Clean up resources after initialization failure ( #51227 )
...
Signed-off-by: Alex <jihui.huang@daocloud.io >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-06 14:42:13 +03:00
yjz and JianDan0212
5fba75aefe
[Bugfix][Model] Add missing fused_qkv_a_proj to Kimi-Linear packed_modules_mapping ( #51249 )
...
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com >
Co-authored-by: JianDan0212 <zhangyj0212@gmail.com >
2026-08-06 11:39:17 +00:00
Sage
865781e627
[Rust Frontend] Add standalone Rust renderer ( #50289 )
2026-08-06 11:23:26 +00:00
ef2615c2e0
[Bugfix][KV Offload] Fall back when MADV_POPULATE_WRITE is unsupported ( #51116 )
...
Signed-off-by: Guy Girmonsky <guygir@gmail.com >
Signed-off-by: Alex <jihui.huang@daocloud.io >
Co-authored-by: Guy Girmonsky <guygir@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-06 13:48:01 +03:00
Lyu Han
22013f74ff
Interns2mobius support ( #51149 )
...
Signed-off-by: Lyu Han <lvhan_028@163.com >
2026-08-06 10:39:35 +00:00
Jared Wen
e07532b035
[MISC][Bench] refactor throughput and reuse serve's get samples ( #50981 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
2026-08-06 10:31:23 +00:00
anhtra3889 and Khang Ngo
872fd5973e
[Bugfix][LoRA] Guard TrtLlm BF16 MoE LoRA gate on activation type ( #51002 )
...
Signed-off-by: Anh Tran <anhtra@nvidia.com >
Co-authored-by: Khang Ngo <khngo@nvidia.com >
2026-08-06 18:12:54 +08:00
wang.yuqi and Cyrus Leung
2fa490470d
[Model] Fused mm preprocess normalisation on the Device ( #50411 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-08-06 03:11:14 -07:00
Matej Sirovatka
9c22668436
[Quantization] Preserve precision in online NVFP4 expert packing ( #50029 )
...
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
2026-08-06 02:23:54 -07:00
Chaojun Zhang and Kunshang Ji
276f0bb5c1
[XPU] Register fake meta kernel for fp4_gemm ( #50946 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-06 17:18:42 +08:00
maobaolong and Li, Jiang <jiang1.li@intel.com>
13726c80fe
[CPU] Add MLA backend so DeepSeek-V2/V3 can run on CPU ( #49453 )
...
Signed-off-by: baoloongmao <baoloongmao@tencent.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-08-06 16:51:41 +08:00
Wentao Ye
ad5280255b
[GLM Perf] DSv32/glm use skip topk for MTP case, 2.0x kernel performance improvement ( #50904 )
2026-08-06 01:48:36 -07:00
2dfb8ba590
[Perf][CUDA] Programmatic dependent launch for the DSA decode kernels ( #50230 )
...
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 00:53:17 -07:00
zofia and opencode
febea17f6a
[Model] Fix weight prefix mapping for native Qwen3.5 text-only checkp… ( #50355 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: opencode <opencode@anomaly.co >
2026-08-06 07:40:44 +00:00
470297c143
[HPC Attention Backend] hpc attention backend support bf16 kv cache with fp8 weight ( #50980 )
...
Signed-off-by: chengvjiang <chengvjiang@tencent.com >
Co-authored-by: chengvjiang <chengvjiang@tencent.com >
Co-authored-by: VAthree <56064364+VAthree@users.noreply.github.com >
2026-08-06 15:40:14 +08:00
Andreas Karatzas
7e724dca89
[ROCm] Fix AITER all-reduce fusion coverage ( #50802 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-08-06 14:52:23 +08:00
Canlin Guo
5370461649
[BugFix][KV Cache] Fix hybrid prefix caching with hidden-state extraction ( #51108 )
...
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com >
2026-08-06 06:42:37 +00:00
zofia and Kunshang Ji
8060260e1d
[Linear] [Kernel] add block-wise scaled_mm ( #49932 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-06 14:40:26 +08:00
liuzhenwei and Kunshang Ji
8c0f029ac7
[XPU] update warning of XPU Graph ( #50236 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-06 06:33:33 +00:00
Gabriel Wu and Yongye Zhu
2e35c529b8
[Bugfix][Quantization] Fix MXFP4 conversion for FlashInfer CUTLASS ( #51038 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-08-05 23:19:22 -07:00
JooHo Lee
c0202c5603
[Bugfix][Spec Decode] Auto-enable async scheduling for draft models ( #48341 )
...
Signed-off-by: BWAAEEEK <jooho414@gmail.com >
2026-08-06 13:51:40 +08:00
b976e980ef
[XPU] Route AWQ linear through choose_mp_linear_kernel ( #50840 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: opencode <noreply@opencode.ai >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 13:50:11 +08:00
liuzhenwei
2a0323b4e0
[XPU][Test] Support MultiConnector accuracy testing on XPU ( #51160 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
2026-08-06 13:49:48 +08:00
d779835a11
[XPU] Support MXFP8 linear weights for INC DeepSeek V4 model ( #48476 )
...
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com >
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-06 13:17:48 +08:00
Qiuyang Yue
777b01d1a8
[MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference ( #45254 )
...
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com >
2026-08-06 05:04:19 +00:00
Li Juqi and Cyrus Leung
821717118f
[BugFix][Pooling] Skip weight-prefix probe when model has WeightsMapper ( #50890 )
...
Signed-off-by: Juqi Li <2223621784@qq.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-05 21:12:33 -07:00
Yunzhu Lu and Cyrus Leung
f84df12c6a
[Bugfix][MM] Fix MiniCPM-V placeholder replacement and image processor loading on Transformers v5 ( #48413 )
...
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com >
Signed-off-by: Yunzhu Lu <lucia.yunzhu@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-05 21:12:00 -07:00
Chauncey
22170354d8
[Feature] Parse request priority from HTTP header ( #51089 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-08-05 21:10:29 -07:00
baodi
2e09247c2d
[Docs] List Intel XPU attention backends ( #51215 )
...
Signed-off-by: baodii <di.bao@intel.com >
2026-08-06 10:58:28 +08:00
liuzhenwei
d54b58cc48
[Hardware] Use torch.accelerator.empty_host_cache() for host cache cl… ( #51107 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
2026-08-06 10:27:57 +08:00
Giancarlo Delfin
81bc196913
[Model Runner V2] Cache draft logits in model's LM head dtype ( #50910 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-08-05 19:12:46 -07:00
Jee Jee Li
f85c1d2f84
K3: remove the add operation for megamoe path ( #51146 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-08-06 01:48:25 +00:00
Aarushi Jain
71f975af43
[ROCm][CI] Add MLA decode accuracy and determinism tests ( #50480 )
...
Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com >
2026-08-05 20:35:56 -05:00
Thien Tran and OpenAI Codex
9f3169960a
[MoE] Align TRTLLM MXFP4 autotune buckets ( #50942 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-05 18:17:53 -07:00
b50fdebce0
[Bugfix] Fix level-2 sleep/wake/reload with enable_lora=True ( #39935 )
...
Signed-off-by: Silen Naihin <silen.naihin@gmail.com >
Co-authored-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-08-06 09:13:10 +08:00
aoshen02 and Andy Lo
8cfa01cdd2
[BugFix] Dense multinode DP rescope with regression test ( #49212 )
...
Signed-off-by: Andy Lo <andy@mistral.ai >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Andy Lo <andy@mistral.ai >
2026-08-05 18:12:03 -07:00
gabriel-peracio and Kimi K3
47a4e410ba
[Bugfix][Spec Decode] Fix NaN handling in rejection sampler tl.argmax ( #50183 )
...
Signed-off-by: Gabriel Peracio <gabriel@sigsonic.com >
Co-authored-by: Kimi K3 <noreply@moonshot.ai >
2026-08-05 18:01:09 -07:00
LZW and Roger Wang
d36f24b66a
[KV Connector][Mooncake] Add tenant ID support to MooncakeStoreConnector ( #48069 )
...
Signed-off-by: lzw <2994587066@qq.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-08-05 17:31:18 -07:00
