b976e980ef
[XPU] Route AWQ linear through choose_mp_linear_kernel ( #50840 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: opencode <noreply@opencode.ai >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 13:50:11 +08:00
d779835a11
[XPU] Support MXFP8 linear weights for INC DeepSeek V4 model ( #48476 )
...
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com >
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-06 13:17:48 +08:00
Qiuyang Yue
777b01d1a8
[MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference ( #45254 )
...
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com >
2026-08-06 05:04:19 +00:00
Li Juqi and Cyrus Leung
821717118f
[BugFix][Pooling] Skip weight-prefix probe when model has WeightsMapper ( #50890 )
...
Signed-off-by: Juqi Li <2223621784@qq.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-05 21:12:33 -07:00
Yunzhu Lu and Cyrus Leung
f84df12c6a
[Bugfix][MM] Fix MiniCPM-V placeholder replacement and image processor loading on Transformers v5 ( #48413 )
...
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com >
Signed-off-by: Yunzhu Lu <lucia.yunzhu@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-05 21:12:00 -07:00
Chauncey
22170354d8
[Feature] Parse request priority from HTTP header ( #51089 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-08-05 21:10:29 -07:00
liuzhenwei
d54b58cc48
[Hardware] Use torch.accelerator.empty_host_cache() for host cache cl… ( #51107 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
2026-08-06 10:27:57 +08:00
Giancarlo Delfin
81bc196913
[Model Runner V2] Cache draft logits in model's LM head dtype ( #50910 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-08-05 19:12:46 -07:00
Jee Jee Li
f85c1d2f84
K3: remove the add operation for megamoe path ( #51146 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-08-06 01:48:25 +00:00
Thien Tran and OpenAI Codex
9f3169960a
[MoE] Align TRTLLM MXFP4 autotune buckets ( #50942 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-05 18:17:53 -07:00
b50fdebce0
[Bugfix] Fix level-2 sleep/wake/reload with enable_lora=True ( #39935 )
...
Signed-off-by: Silen Naihin <silen.naihin@gmail.com >
Co-authored-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-08-06 09:13:10 +08:00
aoshen02 and Andy Lo
8cfa01cdd2
[BugFix] Dense multinode DP rescope with regression test ( #49212 )
...
Signed-off-by: Andy Lo <andy@mistral.ai >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Andy Lo <andy@mistral.ai >
2026-08-05 18:12:03 -07:00
gabriel-peracio and Kimi K3
47a4e410ba
[Bugfix][Spec Decode] Fix NaN handling in rejection sampler tl.argmax ( #50183 )
...
Signed-off-by: Gabriel Peracio <gabriel@sigsonic.com >
Co-authored-by: Kimi K3 <noreply@moonshot.ai >
2026-08-05 18:01:09 -07:00
LZW and Roger Wang
d36f24b66a
[KV Connector][Mooncake] Add tenant ID support to MooncakeStoreConnector ( #48069 )
...
Signed-off-by: lzw <2994587066@qq.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-08-05 17:31:18 -07:00
7c77868cdf
[Bugfix] Size and iterate w13 by shard count for non-gated MoE ( #51125 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Matej Sirovatka <matej.sirovatka@gmail.com >
Co-authored-by: Rishi Puri <riship@nvidia.com >
Co-authored-by: Hsiao-Yuan Chen <littlecircle0730@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-05 17:24:47 -07:00
Qiu and Lucas Wilkinson
811622c410
[Refactor][PCP] Make PCPManager construction extensible ( #50066 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-08-05 23:05:14 +00:00
Tianyu Guo
b30291cf0c
[EPD] Remove duplicate image preprocessing in EPD and enable preprocess on GPU ( #50390 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-08-05 15:59:54 -07:00
Wentao Ye
877975897d
[K3 Perf] Combine multiple all gather together for SP, 1.5~3x kernel level performance improvement ( #51070 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-05 17:42:41 -04:00
bnellnm
373fe8b83e
[MoE Refactor] Remove MoE legacy code ( #51078 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-08-05 14:35:36 -07:00
Netanel Haber
482c613423
[Bugfix][Humming] Preserve ModelOpt FP8 weight dimensions ( #51093 )
...
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
2026-08-05 14:08:55 -07:00
Wentao Ye
e76b71f659
[CI Bug] Fix pydantic_core._pydantic_core.ValidationError: Input should be a valid integer ( #51179 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-05 20:57:47 +00:00
Michael Goin and OpenAI Codex
8f158d0ee2
[MoE] Share apply_moe_activation support metadata ( #44359 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-05 12:05:47 -07:00
fxmarty-amd
bc37fc970e
Revert [Misc] Avoid importing nixl_ep on every vllm serve config ( #50879 ) ( #51176 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-08-05 11:04:40 -07:00
Wentao Ye
23c0f337b7
[CI bug] Fix Each KV cache group's real block_size must be divisible by has h_block_size ( #51180 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-05 14:03:17 -04:00
9833aa53d5
[Bugfix] Fail fast with a clear error when CPU offload region exceeds available space ( #50358 )
...
Signed-off-by: Alex <jihui.huang@daocloud.io >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-08-05 20:59:55 +03:00
Ronen Schaffer and Or Ozeri
8543522ca7
[KV Offload] Support out-of-tree secondary tier managers via module_path ( #51007 )
...
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-08-05 20:30:53 +03:00
zexplorerhj
d4da0c55af
[Model][Frontend] Add Ling 3.0 Flash BF16, MTP, and parser support ( #51045 )
...
Signed-off-by: zexplorerhj <zhjoneson@163.com >
2026-08-06 00:38:28 +08:00
66b3c0e61f
[Kernel][Model] Optimize FA4 mm_prefix range lookup ( #50294 )
...
Signed-off-by: Juqi Li <2223621784@qq.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 07:33:49 -07:00
Chauncey and Or Ozeri
62c5e21621
[KV Offloading] Support partial-tail prefix reuse with fine-grained prefix matching ( #50507 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-08-05 22:31:50 +08:00
Gabriel Wu and Pavani Majety
08b8613b7b
[Bugfix][Model] Fix MiniMax-M3 NVFP4 inference correctness ( #48929 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Pavani Majety <pmajety@nvidia.com >
2026-08-05 07:25:46 -07:00
cd930c8e78
[Bugfix] Fix MLA kv_b_proj activation dtype with Marlin FP8 ( #38771 )
...
Signed-off-by: Jacob Zhang <jaco8123@gmail.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 07:25:41 -07:00
397094da17
Resolve revision to commit_hash once per model load, via huggingface_hub's resolve_revision ( #49990 )
...
Signed-off-by: Wauplin <lucainp@gmail.com >
Signed-off-by: Lucain <lucainp@gmail.com >
Signed-off-by: Lucain Pouget <lucain@huggingface.co >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-05 13:49:56 +00:00
Ziming Huang
beca88e59e
[BugFix][K3] Skip moe_intermediate padding when EP is enabled ( #51131 )
2026-08-05 09:22:55 +00:00
fd859d2e06
[KV Connector][Mooncake] Add store group semantics ( #44956 )
...
Signed-off-by: Schatten <czhengt@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-08-05 08:25:38 +00:00
kliuae
f5cd862dbd
[ROCm][Bugfix] Kimi-K3 Fix KDA NaN on mixed batches and racy autotune config ( #50649 )
...
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com >
2026-08-05 08:07:21 +00:00
MINJUN GIL
b92352ca2c
[Perf][KV Offload] Avoid quadratic ARC batch eviction ( #50992 )
...
Signed-off-by: MINJUN GIL <alswnsrlf12@naver.com >
2026-08-05 10:58:36 +03:00
Sundaresan G and Kunshang Ji
999dd8b490
[XPU] Alias is_current_stream_capturing to XPU in cuda wrapper ( #50526 )
...
Signed-off-by: Sundaresan-G <sundaresan.g@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-05 15:10:17 +08:00
qizixi and Claude Opus 5
4719a9b8f5
[Bugfix][Spec Decode] Fix EAGLE3 DeepSeek draft crash on non-YaRN rope configs ( #51092 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-05 14:10:52 +08:00
Tianyu Guo
50c51682a1
[Bugfix][EC Connector] Don't stop an encoder-instance request before its images are encoded ( #50275 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-08-05 05:53:58 +00:00
Wei-Cheng (Wayne) Chiu and OpenAI Codex
7794b1e08b
[Bugfix] Skip Qwen3 deepstack buffers without vision ( #49397 )
...
Signed-off-by: WEI CHENG CHIU <waynehacking8@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-05 12:46:23 +08:00
TrainToGPB and Claude
eb3dce9757
[Bugfix][Model] Gemma3n/Gemma4: pad variable-length audio batches ( #50958 )
...
Signed-off-by: TrainToGPB <kaiba0514@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-05 12:46:00 +08:00
c25093a305
[Kernel] Batch invariant NVFP4 MoE using cutlass ( #40372 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-08-05 04:31:32 +00:00
41a7e7da0c
[KV Offload] Support partial secondary-tier load results ( #50321 )
...
Signed-off-by: Moein Khazraee <moein@nvidia.com >
Co-authored-by: Or Ozeri <or@ozery.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-08-04 20:48:58 -07:00
aoshen02 and yihengz
6153dbe036
[R3] Unify routed expert shape configuration ( #50940 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: yihengz <yihengz@nvidia.com >
2026-08-04 20:20:41 -07:00
Rehan Khan and Li, Jiang <jiang1.li@intel.com>
0187f4c88e
[CPU] Enable tcmalloc for s390x ( #50841 )
...
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-08-05 10:55:39 +08:00
Yejing Lai and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2cb3ff88cc
[BUGFIX][Quant]Fix test_kv_scale_reload failed ( #50405 )
...
Signed-off-by: Lai, Yejing <yejing.lai@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 10:50:54 +08:00
Dr Tasos Varoudis and Lucas Wilkinson
c416f15710
[Model] Fix Kimi-K3 MLA with disabled context parallelism ( #50404 )
...
Signed-off-by: Tasos Varoudis <varoudis@archtech.gr >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-08-05 02:33:14 +00:00
Andreas Karatzas
33c50587d2
[ROCm] Restore Inkling MTP backend parity ( #50806 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-08-05 02:02:57 +00:00
Rohan Potdar and Claude Opus 4.8
d0ce3dadb6
[ROCm]: Bump torch 2.12, triton 3.7, torchaudio, torchvision ( #50607 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-04 20:14:11 -05:00
Yifan Qiao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ffee32460d
[BugFix][Mooncake] Use global data_parallel_index for the DP engine index ( #48061 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-04 17:10:51 -07:00
