Commit Graph
18900 Commits
Author SHA1 Message Date
Sihan ChenandLi, Jiang <jiang1.li@intel.com> 4938d44a3b [CPU] fixes heterogeneous NIXL KV transfer into CPU_ATTN decode workers (#47871)
Signed-off-by: Spycsh <sihan.chen@intel.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-07-20 07:33:13 +00:00
Michał GanczarenkoandKunshang Ji 37bf988c2f [XPU][Bugfix] Fix GroupCoordinator device_index (#47295)
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-07-20 15:25:56 +08:00
aoshen02 9459fc6471 [Bugfix][RL] Set vLLM config during weight reload (#45989)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
2026-07-20 15:02:56 +08:00
Itay EtelisandItay Etelis 5245c80564 [Doc] Document blocks_per_chunk in the KV offloading guide (#49100)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
2026-07-20 09:48:43 +03:00
Yifan QiaoandOpenAI Codex 9bc266d923 [Bugfix][KV Offload] Propagate EAGLE mode to SimpleCPU coordinator (#49071)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-07-20 06:39:11 +00:00
5c9f6557d7 [Hardware][CPU] Enable granite-4 model on cpu (#47641)
Signed-off-by: Akash Kaothalkar <akashkaothalkar@akashs-mbp.bl1-in.ibm.com>
Signed-off-by: Akash Kaothalkar <akashkaothalkar@dhcp-9-123-5-76.bl1-in.ibm.com>
Signed-off-by: Akash Kaothalkar <akashkaothalkar@Akashs-MBP.lan>
Signed-off-by: Akash kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akashkaothalkar@dhcp-9-123-5-76.bl1-in.ibm.com>
Co-authored-by: Akash Kaothalkar <akashkaothalkar@Akashs-MBP.lan>
Co-authored-by: Akash Kaothalkar <akashkaothalkar@akashs-mbp.bl1-in.ibm.com>
Co-authored-by: Akash kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-07-20 06:15:16 +00:00
aoshen02mvanhornClaude Opus 4.6mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
dcfebf93f4 [Bugfix] Fix logprobs token-string collision from SentencePiece space… (#48674)
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: mvanhorn <mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-20 12:17:18 +08:00
Andreas KaratzasandOpenAI Codex 752bd10647 [ROCm][CI] Fix sparse MLA metadata sync fixture (#49128)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-07-19 23:02:03 -05:00
Thien Tran 2730b657c4 [Bugfix] Fix broken NVVM caused by CuteDSL 4.6.0 (#49108)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-07-19 19:45:56 -07:00
1dcbbd9cac [CI] Move compatible 1xL4 jobs to H200 35GB MIG (#43024)
Signed-off-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-07-19 19:21:25 -07:00
rasmithandAndreas Karatzas ace9fda495 [CI/Build][BugFix][The Rock][AMD] Add spawn method in vision examples to avoid reinitialization (#47932)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-07-19 13:41:52 -05:00
TJian ef0aa7ca2f [ROCm] [Release] [Per-commit] Reenable per commit rocm wheel (#49044)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
2026-07-19 13:38:04 -05:00
Taneem Ibrahim e6d1310b2a [Bugfix] Reject removed pooling parameters (#48984)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-07-19 05:18:03 -07:00
yzong-rh ac5f38a0f7 [Refactor] Extract StructuredOutputsParams creation logic from Request.to_sampling_params (#49003)
Signed-off-by: Yifan Zong <yzong@redhat.com>
2026-07-19 05:18:00 -07:00
b6ff8a2f50 [Core] Add MRV2 virtual-batch PCP for MLA (#46570)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
2026-07-19 02:53:15 +00:00
Isotr0pyandRoger Wang 9243e0124e [Multimodal] Automatically fallback to ViT DP when TP is unavailable (#49046)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
2026-07-18 14:41:04 -07:00
Andreas Karatzas df362b2d6d [ROCm][CI] Ensure sliding window tests release GPU memory (#49055)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-07-18 20:44:05 +00:00
SYLAR 7c2acd38b7 [Bugfix] Qwen3-VL/Qwen-Omni: honor max_pixels/min_pixels for video prompts (#49015) 2026-07-18 10:29:11 -07:00
yzong-rh a287eb163f [Front-end] [Messages] Populate num_cache_creation_tokens (#48535)
Signed-off-by: Yifan Zong <yzong@redhat.com>
2026-07-18 13:04:35 -04:00
frida-andersson e94243893d [ROCm][DSv3.2][Perf] Cap sparse MLA decode KV-splits with a work-per-split heuristic (#46832)
Signed-off-by: Frida Andersson <fanderss@amd.com>
2026-07-18 09:39:37 -07:00
29c0ec4d63 [ci] Move 3 entrypoints tests to h200_35gb queue (#43164)
Signed-off-by: Simon Mo <simon@inferact.ai>
Signed-off-by: Simon Mo <simon@simon-mac-mini-9.local>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-07-18 08:43:49 -07:00
Michael Goin c7ce03bcbd [Bugfix] Bump tml-fa4 for cutlass-dsl 4.6 API compatibility (#48988)
Signed-off-by: mgoin <mgoin64@gmail.com>
2026-07-18 05:59:33 -07:00
Harry Mellor c233d90aa8 Remove even more unnecessary load_weights methods (#48496)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-07-18 08:40:27 +00:00
d96aee0951 [Bugfix] Re-sync parameter tp_rank after process_weights_after_loading (fix replicated / disable_tp weight reload) (#48025)
Signed-off-by: Alex Xu <alexxu@roblox.com>
Co-authored-by: YQ-Wang <yiqingwang@roblox.com>
Co-authored-by: alexhxu <alex.xu1015@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-18 08:40:06 +00:00
Francesco Fusco c71a583aa9 [Perf][Hybrid] Vectorize _copy_mamba_state_block to uint64 for temporal (#48110) 2026-07-18 04:43:09 +00:00
xuebwang-amd f12b80c6ef [ROCm][Bugfix] Fix GPT-OSS Quark MXFP4 MoE loading - emulation buffer not block-aligned (#43979)
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
2026-07-18 03:49:41 +00:00
Jee Jee Li da64db78b9 [LoRA] Optimize TrtLlmLoRAExperts (#48759)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-07-18 10:26:14 +08:00
Michael Goinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick Hill
425c4eafb0 [Sampler] Stop upcasting logits to fp32 in apply_sampling_params (#48641)
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
2026-07-17 18:15:20 -07:00
Michael Goin 02c01f442b [Model] Use standard ModelOpt config for Inkling NVFP4 (#48990)
Signed-off-by: mgoin <mgoin64@gmail.com>
2026-07-17 18:13:14 -07:00
Wang Xingda王兴达mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
fae543015c [Frontend]Flatten beam-search beams with itertools.chain instead of sum (#48829)
Signed-off-by: Wang Xingda <wangxingda1993@126.com>
Co-authored-by: 王兴达 <wangxingda@360itdeMacBook-Pro.local>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-17 23:09:14 +01:00
Shangdi YuandClaude Opus 4.8 c9be3a8aa1 [Kernel][Helion] Disable warp specialization in rms_norm_per_block_quant B200 configs (#48797)
Signed-off-by: Shangdi Yu <shangdiy@meta.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 21:39:09 +00:00
41ea2dd44a [Bugfix][V1/V2] Fix prompt_logprobs to respect logprobs_mode (#47680)
Signed-off-by: Wojciech Wais <wojciech.wais@gmail.com>
Signed-off-by: Federico Kamelhar <209537060+fede-kamel@users.noreply.github.com>
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: Wojciech Wais <wojciech.wais@gmail.com>
Co-authored-by: Federico Kamelhar <209537060+fede-kamel@users.noreply.github.com>
2026-07-17 21:58:59 +01:00
Kevin H. LuuandCodex 088c0be268 [CI] Fix macOS wheel release annotation context (#48771)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Codex <codex@openai.com>
2026-07-17 13:44:48 -07:00
devalshahamdandDeval Shah fcd2255d16 [Hardware][GPU] Profiler config additional to increase it scope and annotation details (#37524)
Signed-off-by: devalshahamd <deval.shah@amd.com>
Signed-off-by: Deval Shah <devashah@amd.com>
Signed-off-by: Deval Shah <deval.shah@amd.com>
Co-authored-by: Deval Shah <devashah@amd.com>
2026-07-17 13:38:59 -07:00
Wentao Ye b5433b6f50 [Perf] Optimize dsv4 routing using specialized kernel, 2.94% E2E TPOT improvement (#48660)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-07-17 13:35:06 -07:00
Michael GoinandOpenAI Codex cc25f028b7 [Loader] Improve InstantTensor loading (#46868)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-07-17 16:30:02 -04:00
c4cd2bd544 [Bugfix] MoRIIO toy P/D proxy: fix DP-rank index aliasing + harden for high-concurrency bursts (#46115)
Signed-off-by: Edwin Lim <edwin.lim@mangoboost.io>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: QinPR <1905873179@qq.com>
Co-authored-by: Peiran Qin <66068739+QinPR@users.noreply.github.com>
2026-07-17 12:35:04 -07:00
Nicolò LucchesiandClaude Opus 4.8 5784507da4 [Attention] Allow selecting a different attention backend per KV-cache group (#48012)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 15:19:02 -04:00
labAxiaoming bf578e1abd [Bugfix][GLM4V] Fix video dummy profiling and memory usage (#48729)
Signed-off-by: xiaoming <1259730330@qq.com>
2026-07-18 01:44:45 +08:00
Fangzhou Ai efed8a1e83 [ROCm][Perf][DSV4] Improve sparse decode reduction occupancy on gfx950 (#48788)
Signed-off-by: fai <fangzhouai@gmail.com>
2026-07-17 10:24:59 -07:00
11d291511a [Bugfix][Tool Parser] Preserve whitespace in parameter values (MiniMax M2, Qwen3, MiniCPM5 XML) (#48846)
Signed-off-by: mosya415 <263250241+mosya415@users.noreply.github.com>
Signed-off-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
Co-authored-by: mosya415 <263250241+mosya415@users.noreply.github.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
2026-07-17 16:45:41 +00:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 877dae9c68 [Refactor] Remove deepseek dead code (#48780)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-17 14:57:13 +00:00
passtoor-agi c4dd6d78fd Fix: Restore data_parallel_size > 1 for use_sequence_parallel_moe (#48849)
Signed-off-by: passtoor-agi <305788622+passtoor-agi@users.noreply.github.com>
2026-07-17 10:27:50 -04:00
JooHo Lee ce2aecc4dc [Performance] Use CuTe-DSL for FlashInfer MXFP4 quantization (#48417)
Signed-off-by: BWAAEEEK <jooho414@gmail.com>
2026-07-17 06:53:48 -07:00
AlexHuangandOr Ozeri f38f3d11fb [Bugfix][KV Offloading] Offload last block at request finish and prevent reuse race (#48596)
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
2026-07-17 16:50:49 +03:00
Artur FierkaandClaude d4b4562917 [XPU] Bump vllm_xpu_kernels to v0.1.11.1 (#48942)
Signed-off-by: Artur Fierka <artur.fierka@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-07-17 20:43:23 +08:00
Yejing Laimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Harry Mellor
7b3192523e [Bugfix]Fix transformer backend failed: AttributeError: 'Parameter' object has no attribute 'weight_loader' (#48699)
Signed-off-by: Lai, Yejing <yejing.lai@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-07-17 12:43:44 +01:00
Yejing Lai 4c6e2e4b30 [XPU][UT]fix _POSSIBLE_KERNELS error on XPU (#47516)
Signed-off-by: Lai, Yejing <yejing.lai@intel.com>
2026-07-17 11:19:05 +00:00
liuzhenwei 8502958810 [XPU] support HND layout (#47975)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
2026-07-17 10:54:34 +00:00
Julien DenizeandNicolò Lucchesi ce4bdcbda4 [Bugfix] Enable FlashAttention MLA prefill for Mistral Small 4 head dims (#48855)
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
2026-07-17 18:07:00 +08:00