 Matthew Bonanni
|
20228cb851
|
[3/N][Attention] Move AttentionMetadata-related code from utils.py to backend.py (#32054)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-01-12 09:13:56 -08:00 |
|
|
|
49e6b86c91
|
[Feature] Support recording expert indices for rollout router replay (#28284)
Signed-off-by: xhx1022 <1737006628@qq.com>
Signed-off-by: Hongxin Xu <70438206+xhx1022@users.noreply.github.com>
Signed-off-by: arlenxu <arlenxu@tencent.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: arlenxu <arlenxu@tencent.com>
|
2026-01-12 06:23:04 -08:00 |
|
|
Or Ozeri
|
4c16ba617f
|
[KVConnector] OffloadingConnector: Fix bug in handling of preemptions (#29870)
Signed-off-by: Or Ozeri <oro@il.ibm.com>
|
2026-01-11 08:05:36 +00:00 |
|
|
zhrrr
|
97ba96fbe9
|
[perf][async] support non cpu sync get logprob tensors for spec (#31336)
Signed-off-by: izhuhaoran <izhuhaoran@qq.com>
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
|
2026-01-09 21:24:51 +00:00 |
|
|
Matthew Bonanni
|
2612ba9285
|
[1/N][Attention] Restructure attention: move files (#31916)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-01-09 13:10:24 -08:00 |
|
|
Nick Hill
|
29ce48221c
|
[Cleanup] Remove obsolete spec decoding compatibility logic (#32003)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-01-09 05:44:18 +00:00 |
|
|
zhrrrandNick Hill
|
8ff4a99566
|
[Async][Feat] support apply penalty or bad_words for async + spec (#30495)
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: izhuhaoran <izhuhaoran@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-01-09 02:31:50 +00:00 |
|
|
Nick Hill
|
a3d909ad2b
|
[Misc] Tidy up some spec decode logic in GPUModelRunner (#31591)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-01-08 09:10:07 -08:00 |
|
|
Nick Hill
|
287b37cda4
|
[BugFix] Fix spec decoding edge case bugs (#31944)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-01-08 15:31:03 +08:00 |
|
|
Ning Xie
|
c907d22158
|
[refactor] refactor memory constants usage (#31865)
Signed-off-by: Andy Xie <andy.xning@gmail.com>
|
2026-01-07 18:37:31 +00:00 |
|
|
Cyrus Leung
|
b7036c87a1
|
[Refactor] Clean up pooler modules (#31897)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-01-08 00:07:43 +08:00 |
|
|
tianshu-Michael-yu
|
efeaac92f2
|
[Bugfix] Fix race condition in async-scheduling for vlm model (#31841)
Signed-off-by: Tianshu Yu <tianshuyu.formal@gmail.com>
|
2026-01-07 06:45:10 +00:00 |
|
|
Benjamin ChislettandNick Hill
|
f7008ce1c4
|
[Perf] Async Scheduling + Speculative Decoding + Structured Outputs (#29821)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-01-06 18:50:37 +00:00 |
|
|
Nick Hill
|
43e3f8e4a9
|
[Misc] Various code simplifications (#31666)
Signed-off-by: njhill <nickhill123@gmail.com>
|
2026-01-04 18:35:56 -08:00 |
|
|
Nick Hill
|
da436f868a
|
[Minor] Small pooler output processing optimization (#31667)
Signed-off-by: njhill <nickhill123@gmail.com>
|
2026-01-04 18:33:12 -08:00 |
|
|
Nick Hill
|
6c2cfb62ff
|
[BugFix] Fix async scheduling for pooling models (#31584)
Signed-off-by: njhill <nickhill123@gmail.com>
|
2025-12-31 14:48:51 -08:00 |
|
|
maang-h
|
b4bb5f312f
|
[Core] Remove unused num_tokens parameter from _init_model_kwargs (#31517)
Signed-off-by: maang <maang_h@163.com>
|
2025-12-30 20:47:23 -08:00 |
|
|
vintipandey
|
04147dcfa7
|
[Bugfix]Fix pooling model always disabled due to incorrect PP rank check (#31505)
Signed-off-by: vintipandey <vinti.pandey@gmail.com>
|
2025-12-30 11:27:10 -08:00 |
|
|
Nick Hill
|
3b312fb792
|
[Minor] Various small code cleanups/simplifications (#31508)
Signed-off-by: njhill <nickhill123@gmail.com>
|
2025-12-29 22:42:06 -08:00 |
|
|
Nick Hill
|
094fcce250
|
[BugFix] Re-fix async multimodal cpu tensor race condition (#31373)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: njhill <nickhill123@gmail.com>
|
2025-12-28 03:05:08 -08:00 |
|
|
|
ce1eafd1a5
|
[Core] Initialize LoRA support for tower and connector in multi-modal models (#26674)
Signed-off-by: bk-201 <joy25810@foxmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: prashanth058 <prashanth.dannamaneni@uipath.com>
Co-authored-by: bk-201 <joy25810@foxmail.com>
Co-authored-by: prashanth058 <prashanth.dannamaneni@uipath.com>
Co-authored-by: Anexdeus <5142168@mail.ru>
|
2025-12-26 04:48:20 -08:00 |
|
|
Cyrus Leung
|
09dc7c690c
|
[Chore][1/2] Drop v0.14 deprecations (#31285)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-12-24 09:54:01 -08:00 |
|
|
Cyrus Leung
|
ca6a95ba25
|
[Chore] Simplify logic of _execute_mm_encoder (#31222)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-12-23 18:15:16 -08:00 |
|
|
Patrick von Platen
|
3faa8bee57
|
adapt voxtral (#31095)
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
|
2025-12-23 05:31:55 -08:00 |
|
|
Lucas Wilkinson
|
de71747655
|
[SpecDecode] Simplified alternative padded-speculation acceptance rate fix (#29845)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-12-22 13:06:10 -08:00 |
|
|
dengyunyang
|
8f8f469b1b
|
[BugFix] skip language model in Encoder (#30242)
Signed-off-by: dengyunyang <584797741@qq.com>
|
2025-12-22 05:25:59 -08:00 |
|
|
Lucas Wilkinson
|
5f6477d1d0
|
[BugFix] Fix TypeError: unhashable type: 'dict' when serving deepseek32 (#30924)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-12-19 16:07:54 -05:00 |
|
|
Nick Hill
|
b0b77c4655
|
[BugFix] Fix spec decode + structured outputs + preemption edge case (#30916)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-12-18 12:59:55 -08:00 |
|
|
Nick Hill
|
686cbaac64
|
[Cleanup] Remove unused ModelRunner V1 InputBatch.num_tokens field (#30218)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-12-18 09:17:00 -08:00 |
|
|
Jialin Ouyang
|
6e9dbcc50e
|
[Fix] uniform decode batch check (#30747)
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
|
2025-12-17 19:58:43 +08:00 |
|
|
Roger WangandSun Kim
|
f5f51e5931
|
[Core][MM] Optimize encoder cache manager by operating with embeddings only (#30475)
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Sun Kim <sunytokki@gmail.com>
|
2025-12-16 14:18:17 -08:00 |
|
|
Lucas WilkinsonandStanislaw Wozniak
|
9fec0e13d5
|
[Attention] Cache attention metadata builds across hybrid KV-cache groups (#29627)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Stanislaw Wozniak <stw@zurich.ibm.com>
|
2025-12-16 17:10:16 -05:00 |
|
|
jiangkuaixue123androot
|
b9ff4f2a8d
|
[feature] extend DBO to XBO (#30120)
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
|
2025-12-16 00:04:01 -05:00 |
|
|
Lucas Wilkinson
|
3e41992fec
|
[Attention] Use sparse prefill kernel for fp8 kv-cache in DeepSeek-v3.2 (#27532)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-12-12 05:57:47 -08:00 |
|
|
Lucas Wilkinson
|
042da73244
|
[Core] Refactor _build_attention_metadata (#29628)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-12-11 17:54:12 -08:00 |
|
|
Xingyu LiuandCyrus Leung
|
90d6cf921f
|
[BugFix][MM]support VLLM_RANDOMIZE_DP_DUMMY_INPUTS (#30472)
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2025-12-11 21:00:15 +00:00 |
|
|
QiuandCyrus Leung
|
a11f4a81e0
|
[Misc][PCP&DCP] relocate PCP feature check (#30050)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2025-12-11 03:36:18 -08:00 |
|
|
shivamprandChen Zhang
|
8580919ac3
|
[Bugfix] fix confusing OOM errors during v1 init (#28051)
Signed-off-by: Shivam <shivamprasad91@gmail.com>
Signed-off-by: shivampr <shivampr.dev@gmail.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
|
2025-12-10 23:17:41 +00:00 |
|
|
Nicolò Lucchesi
|
c756fb6781
|
[Core] Whisper enable FULL_DECODE_ONLY CudaGraph (#30072)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2025-12-10 06:14:24 -08:00 |
|
|
Wilson WuandWentao Ye
|
3bdd426636
|
Fix typos in comments across multiple files (#30345)
Signed-off-by: Wilson Wu <iwilsonwu@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2025-12-09 20:05:28 -08:00 |
|
|
Lucas WilkinsonandBenjamin Chislett
|
abe93bce59
|
[Attention] Make seq_lens_cpu optional in CommonAttentionMetadata to enable true async spec-decode (#29624)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
|
2025-12-09 17:18:10 -08:00 |
|
|
|
95501a70ec
|
[BugFix] Fix DeepSeek-R1 hang with DP and MTP (#30119)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2025-12-09 18:51:19 +00:00 |
|
|
Lucas Wilkinson
|
56037dfa2f
|
[BugFix] Fix assert batch_descriptor.num_tokens == num_tokens_padded (#30173)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-12-09 10:36:12 -05:00 |
|
|
Isotr0py
|
b952f4d3c3
|
[v1] Add PrefixLM support to FlexAttention backend (#27938)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2025-12-07 15:51:36 +00:00 |
|
|
Cyrus Leung
|
e83b7e379c
|
Revert "[Renderer] Separate out RendererConfig from ModelConfig (#30145)" (#30199)
|
2025-12-07 00:00:22 -08:00 |
|
|
Cyrus Leung
|
27f4c2fd46
|
[Renderer] Separate out RendererConfig from ModelConfig (#30145)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-12-06 23:15:42 -08:00 |
|
|
Cyrus Leung
|
671427efbf
|
[Model] Move multimodal_cpu_fields definition to field config (#30181)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-12-06 13:40:02 +00:00 |
|
|
Max HuandMax Hu
|
c2894d3883
|
[Feature] Add Layer-wise NVTX Support (#29990)
Signed-off-by: Max Hu <hyoung2991@gmail.com>
Signed-off-by: Max Hu <maxhu@nvidia.com>
Co-authored-by: Max Hu <maxhu@nvidia.com>
|
2025-12-05 11:20:07 +00:00 |
|
|
Lucas Wilkinson
|
c8ab988b15
|
[BugFix] Fix DBO assert assert B_block_table == B_q (#29933)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-12-04 14:48:54 -05:00 |
|
|
Cyrus Leung
|
b286a311c2
|
[Chore] Deprecate merge_by_field_config arg (#30035)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-12-04 17:21:24 +00:00 |
|