75231eff2f
[Bugfix][Parser] Prevent Inkling block-end leakage with tools ( #51391 )
...
Signed-off-by: cherry77-cloud <1615405@qq.com >
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-08 12:34:32 -04:00
58d3918e3e
[BugFix] Preserve divergent FA hits with external Mamba state ( #51468 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Jeff Ma <jeffjma@umich.edu >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-08 20:20:16 +08:00
Andreas Karatzas
6e18901195
[Bugfix][MM] Avoid device sync in FusedInputNorm initialization ( #51435 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-08-08 13:32:23 +08:00
Jiangyun Zhu
44351f81d5
[CI] Refresh hybrid Model Runner V2 coverage ( #51410 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-08-08 05:10:54 +00:00
Tianmu Li
700d39b558
[CPU] Optimize routed FP8/MXFP4 MoE GEMM dispatch ( #50949 )
...
Signed-off-by: Li, Tianmu <tianmu.li@intel.com >
2026-08-08 12:56:14 +08:00
Junpu Yu
d3621c1eb3
[Bugfix][Multimodal] Fix PyNvVideoCodec video backend returning NCHW instead of NHWC ( #51076 )
...
Signed-off-by: Junpu Yu <davidyu@nvidia.com >
2026-08-08 04:49:06 +00:00
aoshen02 and OpenAI Codex
22a175921c
[BugFix] Reject invalid data-parallel RPC ports ( #51469 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-08 12:23:26 +08:00
Gyula Chinoradszki and Cursor Agent
d21fed9522
[Bugfix][models_multimodal] Remote HF python code misses importing class ( #51427 )
...
Signed-off-by: Gyula Chinoradszki <gchinora@amd.com >
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2026-08-08 12:09:26 +08:00
music-dino
a828536fb5
[Bugfix][multi_modal] Fix pos_ids being unitialized for minicpmv2.6 in hf runner ( #51432 )
...
Signed-off-by: Dino Music <Dino.Music@amd.com >
2026-08-08 12:09:12 +08:00
Andreas Karatzas
12da9b23ae
[CI] Guard remote-code Transformers compatibility ( #51451 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-08-08 12:05:49 +08:00
f7ef489e93
[Bugfix] Fix ZMQ port TOCTOU race in shm_broadcast MessageQueue ( #50960 )
...
Signed-off-by: RTCartist <wangshengb@buaa.edu.cn >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: RTCartist <wangshengb@buaa.edu.cn >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-07 18:19:02 -07:00
Woosuk Kwon
e644c8cd8c
[Perf] Optimize DeepSeek V3.2 sequence parallelism ( #51434 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-08-07 16:51:01 -07:00
Wentao Ye
c39076feff
[CI Test] Add specific unit test for mrv2 offloading ( #51440 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-07 22:49:45 +00:00
Michael Goin and OpenAI Codex
c8fe1d5715
[Spec Decode] Register Qwen3.6 dSpark acceptance coverage ( #51310 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-07 17:07:55 -04:00
Wei-Ming Chen
1c94e8dc7d
Add NVFP4 KV 4-over-6 scale search ( #45187 )
...
Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com >
2026-08-07 13:22:27 -07:00
Walter Beller-Morales and Harry Mellor
70456e5e6f
[Bugfix] when loading weights skip empty expert bias if model does not support them ( #50937 )
...
Signed-off-by: walterbm <walter.beller.morales@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-07 12:45:52 -07:00
Nick Hill
ac70ce96e0
[Frontend] Disable uvicorn signal handlers instead of racing them ( #50916 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-07 13:40:24 -06:00
fxmarty-amd
3518110f2f
[Online quantization] Add online MXFP4 quantization support ( #49347 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-08-07 12:39:46 -07:00
Wentao Ye
a671679e9f
[MRv2 Feature] MR v2 weight offloading support ( #51413 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-07 13:20:21 -06:00
Lucas Wilkinson
021b7d985b
[Perf] Raise Blackwell CUDA graph capture default to 1024 ( #49390 )
2026-08-07 15:18:14 -04:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
56a4b63d44
[K3 Perf] Optimize k3 dspark fused kv, 4.5~4.6x kernel performance improvement ( #50585 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-07 12:16:11 -07:00
coltonottley and Colton Ottley
0df620d429
[Test] Add packed DeepSeek-V4 KV zeroer geometry regression ( #51288 )
...
Signed-off-by: Colton Ottley <colton@ottleyengineering.com >
Co-authored-by: Colton Ottley <colton@ottleyengineering.com >
2026-08-07 12:11:09 -07:00
Aarushi Jain and Aarushi Jain
a0056e103e
[Test] Add ROCm AITER MLA op registration and env gating tests ( #50930 )
...
Signed-off-by: Aarushi Jain <aarushi@amd.com >
Co-authored-by: Aarushi Jain <aarushi@amd.com >
2026-08-07 14:07:31 -05:00
Harry Mellor
45273b8dcb
[1/N] Harden Transformers modelling backend multi-modal path ( #51408 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-07 17:30:43 +00:00
Jinzhen Lin and OpenAI Codex
fcde8e1460
[Refactor] refactor humming linear and moe backends to use explicit layer configs ( #49610 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-07 10:03:19 -07:00
Jack Hu and Andreas Karatzas
34c1cd20a5
[Bugfix][ROCm] Fix ROCM_AITER_FA & ROCM_AITER_UNIFIED_ATTN QK-Norm+RoPE+KVCache fusion for the packed KV-cache [BLOCKS, HEADS, BLOCK_SIZE, 2*HEAD_DIM] layout ( #49373 )
...
Signed-off-by: Jack Hu <Jack.Hu@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-07 09:54:55 -07:00
448344c0e2
[Bugfix] Fix get_open_port() livelock on DP-reserved ports and cover get_open_ports_list ( #50965 )
...
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Chenglun Hu <chenglunhu@gmail.com >
Co-authored-by: ferkans-amir <amir.rezaei@tu-berlin.de >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-07 22:39:33 +08:00
47228db84c
[Bugfix][KV-transfer] MoRIIO: per-layer READ-completion barrier in wait_for_layer_load ( #48534 )
...
Signed-off-by: Edwin Lim <edwin.lim@mangoboost.io >
Signed-off-by: Redwan Khan <redwan.khan@mangoboost.io >
Signed-off-by: harishk-mangoboost <harish.kambhampaty@mangoboost.io >
Signed-off-by: limeward <32970461+edwinlim0919@users.noreply.github.com >
Signed-off-by: QinPR <1905873179@qq.com >
Co-authored-by: Jaeyoun Kim <jaeyoun.kim@mangoboost.io >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Redwan Khan <redwan.khan@mangoboost.io >
Co-authored-by: harishk-mangoboost <harish.kambhampaty@mangoboost.io >
Co-authored-by: rkhan055 <redwankhan055@gmail.com >
Co-authored-by: Peiran Qin <66068739+QinPR@users.noreply.github.com >
Co-authored-by: QinPR <1905873179@qq.com >
2026-08-07 14:23:33 +00:00
Nicolò Lucchesi
d4ecb75ba2
[PD][NixlPush][Bugfix] Fix prefix caching ( #48758 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-08-07 16:21:17 +02:00
music-dino and Harry Mellor
a231c5ceac
[Bugfix] Skip fetching revision for model when model and weights_model are different ( #51260 )
...
Signed-off-by: Dino Music <Dino.Music@amd.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-07 12:55:50 +00:00
Julien Debache
ae934ba8a5
feat: extended EPLB support for Mistral Large 3 and additional MoE backends ( #48355 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
2026-08-07 05:32:52 -07:00
Yan Ma
8d9b52f7c2
[XPU] quick fix online quantization UT break ( #51365 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-08-07 10:25:31 +00:00
pmanczak and pmanczak
6b5bec7bed
[Misc] Add and enable Triton kernel unit tests on XPU ( #45694 )
...
Signed-off-by: pmanczak <pmanczak@users.noreply.github.com >
Signed-off-by: pmanczak <pawel.manczak@intel.com >
Co-authored-by: pmanczak <pmanczak@users.noreply.github.com >
2026-08-07 02:53:03 -07:00
Nicolò Lucchesi
5ec47f3e48
[PD][PushConnector] Record last activity of remotes to allow clean up of stale ones ( #50234 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-08-07 09:43:29 +00:00
Sebastian Woo and Claude
4f76c8ad9d
[Bugfix][Platform] Stop re-initializing NVML on every device-capability check ( fixes #50381 ) ( #50393 )
...
Signed-off-by: seewoo <seewoo@ucsc.edu >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-07 08:30:00 +00:00
da788334bc
Support DeepSeek-V4 AMD Quark NVFP4 with emulation kernel ( #47972 )
...
Signed-off-by: Jimmy Adams <Adam_ji@outlook.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: huji <huidong.ji@amd.com >
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: huji <huidong.ji@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-08-07 01:19:31 -07:00
0de0362ea1
[ROCm][CI] Loosen block-FP8 fused MoE test tolerance for large-K shapes ( #48847 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-07 01:19:26 -07:00
Aaron Hao and Claude Opus 5
21ea5b4fa1
[rl] Stateful Trainer Send: NCCL + Sparse NCCL [3/N] ( #50902 )
...
Signed-off-by: haoaaron <ahao@anyscale.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-07 00:35:59 -07:00
kliuae
43d691ec6b
[ROCm][Perf] Kimi-K3 Shard Latent MoE up-projection for ROCm path ( #51253 )
...
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com >
2026-08-07 06:11:18 +00:00
Qianxu Wang and qianxu
c810e5ee99
[Bugfix] Fix Mamba all-mode CPU offload boundary alignment ( #51100 )
...
Signed-off-by: qianxu <admin@picasso-lab.com >
Co-authored-by: qianxu <admin@picasso-lab.com >
2026-08-07 07:37:48 +03:00
Tianyu Guo
dd856e48bb
[Bugfix][EPD][Model Runner V2] Skip gather mm embeddings for encoder only instance ( #51222 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-08-06 20:40:47 -07:00
Chauncey and Nick Hill
72c0d67657
[Feat] Support thinking_token_budget in Model Runner V2 ( #46727 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-07 03:36:18 +00:00
Artem Perevedentsev
5ac2684976
[CI] Re-enable FI autotune in GSM8K config for Qwen3.5-35B-A3B ( #51293 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-08-07 02:42:34 +00:00
e08111211b
[Kernel] Support Nvfp4 Cutedsl Moe Swiglu-oai and Relu2(non-gated) Activation ( #47106 )
...
Signed-off-by: Tiekai Bi <tiekaib@nvidia.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-08-06 17:42:45 -07:00
8170c23c4f
[Quantization] Share online weight scales across TP ( #49764 )
...
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Roger Wang <hey@rogerw.io >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-08-06 16:34:33 -07:00
Giancarlo Delfin and Nick Hill
27930df9c2
[Model Runner V2] Fix -1 placeholder draft token ids in rejection sam… ( #50939 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-06 16:32:00 -07:00
Taneem Ibrahim
9464529612
[ModelRunner v2] Enable decoder token-wise pooling ( #50931 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-08-06 22:21:52 +00:00
Tejas
4d341ca829
fix: resolve silent request skipping in PRIORITY scheduling ( #49206 )
...
Signed-off-by: Tejas-Raj01 <rajtejas.xyz@gmail.com >
2026-08-06 22:21:21 +00:00
wangxian001 and Nick Hill
d6af803f43
[Bugfix] Fix packed KV block zeroing stride ( #50276 )
...
Signed-off-by: wangxian001 <120719093+wangxian001@users.noreply.github.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-06 13:49:06 -07:00
d8eabdbfbe
[ROCm][MLA] Use asm decode for non-divisor small head counts ( #50578 )
...
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com >
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com >
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com >
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Markus Hartikainen <markus.hartikainen@amd.com >
Co-authored-by: Liuyinfeng01 <yinfeliu@amd.com >
2026-08-06 14:56:53 -04:00
