d8eabdbfbe
[ROCm][MLA] Use asm decode for non-divisor small head counts ( #50578 )
...
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com >
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com >
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com >
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Markus Hartikainen <markus.hartikainen@amd.com >
Co-authored-by: Liuyinfeng01 <yinfeliu@amd.com >
2026-08-06 14:56:53 -04:00
7b9f2dad89
[Frontend] Watch frontend processes during engine startup ( #43417 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-06 11:17:24 -07:00
Divakar Verma
4f851bef6c
[ROCm][CI] Update AITER AR+RMS e2e fusion counts for final-norm coverage ( #51273 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
2026-08-06 13:05:16 -05:00
c56f169d9a
[Bugfix] Keep mamba align prefill chunks block-aligned past last_cache_position ( #51113 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Hernan <kodek@eat1337.com >
Co-authored-by: yanghui1-arch <3053034939@qq.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-08-06 10:21:00 -07:00
81be2e09ae
[Weight processing] Copy over new_data attributes in replace_parameter ( #49601 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 13:03:37 -04:00
b38e111d3e
[Attention][MLA] Per-request scheduling for MLA chunked context ( #50613 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 16:45:44 +00:00
Harry Mellor
e7b8d59460
Fully generalise input embedding handling in Transformers modelling backend ( #51247 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-06 16:21:24 +00:00
Michael Goin and OpenAI Codex
566c80edf9
[CI] Run basic fullgraph correctness on one GPU ( #51271 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-06 12:09:30 -04:00
ef2615c2e0
[Bugfix][KV Offload] Fall back when MADV_POPULATE_WRITE is unsupported ( #51116 )
...
Signed-off-by: Guy Girmonsky <guygir@gmail.com >
Signed-off-by: Alex <jihui.huang@daocloud.io >
Co-authored-by: Guy Girmonsky <guygir@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-06 13:48:01 +03:00
Lyu Han
22013f74ff
Interns2mobius support ( #51149 )
...
Signed-off-by: Lyu Han <lvhan_028@163.com >
2026-08-06 10:39:35 +00:00
Jared Wen
e07532b035
[MISC][Bench] refactor throughput and reuse serve's get samples ( #50981 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
2026-08-06 10:31:23 +00:00
wang.yuqi and Cyrus Leung
2fa490470d
[Model] Fused mm preprocess normalisation on the Device ( #50411 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-08-06 03:11:14 -07:00
Matej Sirovatka
9c22668436
[Quantization] Preserve precision in online NVFP4 expert packing ( #50029 )
...
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
2026-08-06 02:23:54 -07:00
maobaolong and Li, Jiang <jiang1.li@intel.com>
13726c80fe
[CPU] Add MLA backend so DeepSeek-V2/V3 can run on CPU ( #49453 )
...
Signed-off-by: baoloongmao <baoloongmao@tencent.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-08-06 16:51:41 +08:00
zofia and opencode
febea17f6a
[Model] Fix weight prefix mapping for native Qwen3.5 text-only checkp… ( #50355 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: opencode <opencode@anomaly.co >
2026-08-06 07:40:44 +00:00
Andreas Karatzas
7e724dca89
[ROCm] Fix AITER all-reduce fusion coverage ( #50802 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-08-06 14:52:23 +08:00
Canlin Guo
5370461649
[BugFix][KV Cache] Fix hybrid prefix caching with hidden-state extraction ( #51108 )
...
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com >
2026-08-06 06:42:37 +00:00
zofia and Kunshang Ji
8060260e1d
[Linear] [Kernel] add block-wise scaled_mm ( #49932 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-06 14:40:26 +08:00
Gabriel Wu and Yongye Zhu
2e35c529b8
[Bugfix][Quantization] Fix MXFP4 conversion for FlashInfer CUTLASS ( #51038 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-08-05 23:19:22 -07:00
JooHo Lee
c0202c5603
[Bugfix][Spec Decode] Auto-enable async scheduling for draft models ( #48341 )
...
Signed-off-by: BWAAEEEK <jooho414@gmail.com >
2026-08-06 13:51:40 +08:00
liuzhenwei
2a0323b4e0
[XPU][Test] Support MultiConnector accuracy testing on XPU ( #51160 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
2026-08-06 13:49:48 +08:00
Qiuyang Yue
777b01d1a8
[MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference ( #45254 )
...
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com >
2026-08-06 05:04:19 +00:00
Li Juqi and Cyrus Leung
821717118f
[BugFix][Pooling] Skip weight-prefix probe when model has WeightsMapper ( #50890 )
...
Signed-off-by: Juqi Li <2223621784@qq.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-05 21:12:33 -07:00
Yunzhu Lu and Cyrus Leung
f84df12c6a
[Bugfix][MM] Fix MiniCPM-V placeholder replacement and image processor loading on Transformers v5 ( #48413 )
...
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com >
Signed-off-by: Yunzhu Lu <lucia.yunzhu@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-05 21:12:00 -07:00
Giancarlo Delfin
81bc196913
[Model Runner V2] Cache draft logits in model's LM head dtype ( #50910 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-08-05 19:12:46 -07:00
Aarushi Jain
71f975af43
[ROCm][CI] Add MLA decode accuracy and determinism tests ( #50480 )
...
Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com >
2026-08-05 20:35:56 -05:00
b50fdebce0
[Bugfix] Fix level-2 sleep/wake/reload with enable_lora=True ( #39935 )
...
Signed-off-by: Silen Naihin <silen.naihin@gmail.com >
Co-authored-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-08-06 09:13:10 +08:00
aoshen02 and Andy Lo
8cfa01cdd2
[BugFix] Dense multinode DP rescope with regression test ( #49212 )
...
Signed-off-by: Andy Lo <andy@mistral.ai >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Andy Lo <andy@mistral.ai >
2026-08-05 18:12:03 -07:00
gabriel-peracio and Kimi K3
47a4e410ba
[Bugfix][Spec Decode] Fix NaN handling in rejection sampler tl.argmax ( #50183 )
...
Signed-off-by: Gabriel Peracio <gabriel@sigsonic.com >
Co-authored-by: Kimi K3 <noreply@moonshot.ai >
2026-08-05 18:01:09 -07:00
LZW and Roger Wang
d36f24b66a
[KV Connector][Mooncake] Add tenant ID support to MooncakeStoreConnector ( #48069 )
...
Signed-off-by: lzw <2994587066@qq.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-08-05 17:31:18 -07:00
7c77868cdf
[Bugfix] Size and iterate w13 by shard count for non-gated MoE ( #51125 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Matej Sirovatka <matej.sirovatka@gmail.com >
Co-authored-by: Rishi Puri <riship@nvidia.com >
Co-authored-by: Hsiao-Yuan Chen <littlecircle0730@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-05 17:24:47 -07:00
Tianyu Guo
b30291cf0c
[EPD] Remove duplicate image preprocessing in EPD and enable preprocess on GPU ( #50390 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-08-05 15:59:54 -07:00
Rohan Potdar and Claude Opus 4.8
38ebd97bca
[ROCm] Relax MLA rope+cache test tolerances for bf16 ( #51083 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-05 17:50:52 -05:00
bnellnm
373fe8b83e
[MoE Refactor] Remove MoE legacy code ( #51078 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-08-05 14:35:36 -07:00
Micah Williamson and Andreas Karatzas
4282fe52d5
[ROCm][CI] Add More AITER quantization/MoE kernel tests ( #49375 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-05 21:27:33 +00:00
Netanel Haber
482c613423
[Bugfix][Humming] Preserve ModelOpt FP8 weight dimensions ( #51093 )
...
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
2026-08-05 14:08:55 -07:00
stefankoncarevic and Andreas Karatzas
65addac701
[ROCm][CI] Keep rocprofiler-sdk out of DeepEP HT MoE test workers ( #51173 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-05 15:28:41 -05:00
Rohan Potdar and Claude Opus 4.8
c2d8009044
[ROCm] Work around DeepEP teardown SIGSEGV in MoE test harness ( #51174 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-05 14:44:42 -05:00
Michael Goin
e6d67fddb4
[CI] Prune PyTorch Fullgraph Test ( #51074 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-08-05 12:27:27 -07:00
Michael Goin and OpenAI Codex
8f158d0ee2
[MoE] Share apply_moe_activation support metadata ( #44359 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-05 12:05:47 -07:00
Wentao Ye
613411a90c
[CI Bug] Fix Chunked prefill is required for mamba cache mode 'align'. ( #51177 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-05 11:33:45 -07:00
9833aa53d5
[Bugfix] Fail fast with a clear error when CPU offload region exceeds available space ( #50358 )
...
Signed-off-by: Alex <jihui.huang@daocloud.io >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-08-05 20:59:55 +03:00
Ronen Schaffer and Or Ozeri
8543522ca7
[KV Offload] Support out-of-tree secondary tier managers via module_path ( #51007 )
...
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-08-05 20:30:53 +03:00
zexplorerhj
d4da0c55af
[Model][Frontend] Add Ling 3.0 Flash BF16, MTP, and parser support ( #51045 )
...
Signed-off-by: zexplorerhj <zhjoneson@163.com >
2026-08-06 00:38:28 +08:00
66b3c0e61f
[Kernel][Model] Optimize FA4 mm_prefix range lookup ( #50294 )
...
Signed-off-by: Juqi Li <2223621784@qq.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 07:33:49 -07:00
Chauncey and Or Ozeri
62c5e21621
[KV Offloading] Support partial-tail prefix reuse with fine-grained prefix matching ( #50507 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-08-05 22:31:50 +08:00
Gabriel Wu and Pavani Majety
08b8613b7b
[Bugfix][Model] Fix MiniMax-M3 NVFP4 inference correctness ( #48929 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Pavani Majety <pmajety@nvidia.com >
2026-08-05 07:25:46 -07:00
music-dino
a9b39d6dcb
[Bugfix] Enable chunked prefill for qwen3.5-0.8B ppl test ( #51153 )
...
Signed-off-by: Dino Music <Dino.Music@amd.com >
2026-08-05 13:21:43 +00:00
fd859d2e06
[KV Connector][Mooncake] Add store group semantics ( #44956 )
...
Signed-off-by: Schatten <czhengt@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-08-05 08:25:38 +00:00
MINJUN GIL
b92352ca2c
[Perf][KV Offload] Avoid quadratic ARC batch eviction ( #50992 )
...
Signed-off-by: MINJUN GIL <alswnsrlf12@naver.com >
2026-08-05 10:58:36 +03:00
