 Nicolò Lucchesi
|
54020c3c3e
|
[Engram] Drop redundant VLLM_PLE_CPU_OFFLOAD env var (#57937)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-21 20:16:02 +02:00 |
|
|
Tony Lin
|
dc479a2d83
|
[Bugfix] Fix external LB DP rank handling when replicas share nodes (#53743)
Signed-off-by: Tony Lin <tony.lin@intel.com>
|
2026-09-21 17:19:14 +02:00 |
|
|
|
76ffb0d374
|
[Model][Engram] Share host tables across co-located DP replicas by default (#57651)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
|
2026-09-21 08:26:56 +00:00 |
|
|
JohnQinAMD
|
6dfd87f596
|
[ROCm][DSv4.1] Keep the Engram tables in host memory on ROCm (#57491)
|
2026-09-21 10:17:11 +08:00 |
|
|
|
5302d1fe40
|
[Model Runner V2] Support custom logits processors (#56497)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-19 17:46:49 +00:00 |
|
|
|
a85dd55177
|
Fix Laguna patch mutating flat RoPE parameters (#57189)
Signed-off-by: Laura López Real <laulopezreal@users.noreply.github.com>
Co-authored-by: Laura López Real <laulopezreal@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-17 12:22:57 +00:00 |
|
|
Matt
|
2d8a5c5741
|
[ROCm][Bugfix] Reduce CUDA graph divergences (#57229)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
|
2026-09-17 05:12:13 +00:00 |
|
|
|
c58ff535f4
|
[Bugfix][Spec Decode] Fix EAGLE and dense draft startup with EP (#56930)
Signed-off-by: Kyle Liang <kyleliang-nv@users.noreply.github.com>
Co-authored-by: Kyle Liang <kyleliang-nv@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-16 17:03:36 -07:00 |
|
|
Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
|
Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
|
Nicolò Lucchesi
|
df42d112ee
|
[Config] Infer HiSparse attention config from HiSparseConnector (#57041)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-15 19:44:26 +00:00 |
|
|
|
78e84261ab
|
[Performance] Add Triton kernel for Gemma3n sparse GELU (#48498)
Signed-off-by: BWAAEEEK <jooho414@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-09-14 07:42:15 +00:00 |
|
|
|
d2d649e674
|
[DS V4.1][Engram] Support async prefetch for offloaded engram lookups and engram DP sharding (#56512)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: ywang96 <>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-13 02:44:23 -07:00 |
|
|
Markus HartikainenandJee Jee Li
|
c711f740b7
|
[ROCm] Auto-enable breakable CUDA graphs for DeepseekV41ForCausalLM (#56349)
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-09-12 20:47:07 -07:00 |
|
|
|
7ee8a6dd01
|
[watermarking] Dual-key gumbel-max watermarking for speculative decoding support (#56122)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
|
2026-09-12 18:42:01 +00:00 |
|
|
liuzhenwei
|
2f59050eda
|
[XPU][CI] Skip ROCm test on non-ROCm platforms (#56555)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
|
2026-09-12 21:26:55 +08:00 |
|
|
|
d43bb2f37f
|
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-09-12 00:36:47 -04:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
9dd969da09
|
[Model][ROCm] Enable DeepSeek V4 Vision (#55107)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-11 09:17:30 -07:00 |
|
|
Harry MellorandClaude Opus 5
|
c191787a68
|
[Bugfix] Align vLLM YaRN with Transformers and stop re-scaling max_model_len (#56446)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-11 13:01:38 +00:00 |
|
|
|
d0dfe587d5
|
[Bugfix] Fall back to full decode graphs for noncompiled models (#55095)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-09-11 01:56:05 -07:00 |
|
|
|
ea40bb9e90
|
[feature] Watermarked generation and detection (Gumbel-max algorithm) (#54053)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-09 21:00:15 -07:00 |
|
|
Harry Huangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
3116c5d06b
|
[Qwen4Exp] Support UVA PLE-offload and Engram tensor parallelism (#54371)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-09 07:32:34 -07:00 |
|
|
Andreas Karatzas
|
18615ad1ee
|
[Bugfix] Handle null RoPE parameters for NoPE layers (#55949)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-09-08 20:20:05 +00:00 |
|
|
Thien Tran
|
d9105ea800
|
[Qwen3.8-Flash-Next] Remove torch.compile for NVIDIA implementation (#55272)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-09-06 21:57:56 -07:00 |
|
|
|
560ef78bfe
|
[Perf][Model Runner V2] Compact sampling masks on GPU instead of unpacking the full-vocab bitmask on CPU (#54901)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: vx120 <893600387@qq.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: vx120 <893600387@qq.com>
|
2026-09-03 19:03:36 -07:00 |
|
|
Nicolò Lucchesiandcjackal
|
f81eb41934
|
[Bugfix] adjust_dcp_kv_cache_interleave_size for NixlConnector only (#54803)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
|
2026-09-02 12:31:14 +02:00 |
|
|
Eli Nabro Yeongmin OhandCodex
|
cdefd9d499
|
[Bugfix] Support Sentence Transformers 5.4+ serialized configs (#54533)
Signed-off-by: maireneu <maireneu@gmail.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-09-01 15:53:38 +00:00 |
|
|
Ziming HuangandOpenAI Codex
|
481839ad9e
|
[Feature][Spec] Support disabling trailing prefix-cache block dropping (#53388)
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com>
Signed-off-by: Ziming Huang <48115868+ZeldaHuang@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-01 19:12:36 +08:00 |
|
|
|
ce2e343be1
|
[ROCm] Keep GLM-5.2 on MRV1 and disable default breakable cudagraph (#53155)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-31 23:17:54 -05:00 |
|
|
Nicolò Lucchesi
|
7f4793eaa3
|
[Nixl][PD] DCP support for MLA models (#50611)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-08-29 16:03:53 +00:00 |
|
|
Nick Hill
|
4aab2b0ebe
|
[Model Runner V2] Use MRV2 for all models by default (#53183)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-27 11:59:34 -07:00 |
|
|
vllm-agentandKevin Luu
|
d1e5e66ee3
|
[Bugfix] Restore portable all2all backend default (#53952)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
|
2026-08-26 16:19:49 -07:00 |
|
|
|
a4d70bef37
|
[Model Runner V2] Reserve CUDA graph memory (#53306)
Signed-off-by: Anh Tran <anh.tran.3889@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Anh Tran <anh.tran.3889@gmail.com>
Co-authored-by: anhtra3889 <anhtra@nvidia.com>
|
2026-08-23 23:11:19 -07:00 |
|
|
Zhou248
|
2f55ef254c
|
[Model] Add Qwen3-Omni DSpark support (#52560)
Signed-off-by: Zhou248 <630563665@qq.com>
|
2026-08-22 15:35:22 -07:00 |
|
|
SoluMilken
|
27ec8ac626
|
[Bugfix] Fix MTP draft model using local cache path instead of S3 URL with runai_streamer (#42079)
Signed-off-by: SoluMilken <ypiheyn.imm02g@g2.nctu.edu.tw>
|
2026-08-21 09:07:08 -07:00 |
|
|
Zihan Zhang
|
b389ac2946
|
[Spec Decode] DFlash2: local convolution + candidate selector (#52816)
|
2026-08-20 22:27:22 -07:00 |
|
|
|
7cfb97e337
|
[Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915)
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-20 15:42:49 -07:00 |
|
|
|
01af92e175
|
[Feature][Model Runner V2] Support extract_hidden_states speculation (#49811)
Signed-off-by: Zupeng Wang <71580390+zupengwang@users.noreply.github.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI <support@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-20 11:13:43 -04:00 |
|
|
Harry Mellor
|
f0c14b4f77
|
Fix weight tying (#51665)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-08-20 10:31:55 +00:00 |
|
|
Woosuk Kwon
|
b09bd69b5b
|
[Model][NVIDIA] Route DSA models to the CUDA non-compiled path (#52861)
|
2026-08-19 00:13:52 -07:00 |
|
|
Taneem Ibrahim
|
08afae2786
|
[ModelRunner v2] Enable MRV2 for pooling models by default (#48290)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-08-18 21:36:05 -07:00 |
|
|
|
70afdedc10
|
[K3] support recoverssm for K3 (#51855)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-17 19:34:14 +08:00 |
|
|
Nick Hill
|
8efa13b700
|
[Bugfix] Pick the DeepSeek V4 eager cudagraph region per model runner (#52401)
|
2026-08-15 21:04:08 -07:00 |
|
|
kx
|
2ac1f683f1
|
[BugFix] Reserve the bonus query slot in DFlash scheduling budget (#51256)
Signed-off-by: HF-001 <1670186653@qq.com>
|
2026-08-13 03:50:23 +00:00 |
|
|
Woosuk KwonandOpenAI Codex
|
87668ab69b
|
[Bugfix] Guard DeepSeek V4 MRV1 piecewise CUDA graphs (#51768)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-11 07:34:44 -07:00 |
|
|
aoshen02andOpenAI Codex
|
22a175921c
|
[BugFix] Reject invalid data-parallel RPC ports (#51469)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-08 12:23:26 +08:00 |
|
|
music-dinoandHarry Mellor
|
a231c5ceac
|
[Bugfix] Skip fetching revision for model when model and weights_model are different (#51260)
Signed-off-by: Dino Music <Dino.Music@amd.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-08-07 12:55:50 +00:00 |
|
|
JooHo Lee
|
c0202c5603
|
[Bugfix][Spec Decode] Auto-enable async scheduling for draft models (#48341)
Signed-off-by: BWAAEEEK <jooho414@gmail.com>
|
2026-08-06 13:51:40 +08:00 |
|
|
aoshen02andAndy Lo
|
8cfa01cdd2
|
[BugFix] Dense multinode DP rescope with regression test (#49212)
Signed-off-by: Andy Lo <andy@mistral.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Andy Lo <andy@mistral.ai>
|
2026-08-05 18:12:03 -07:00 |
|
|
+13
|
aeeb36b1f1
|
[New model] Kimi K3 (#50000)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-07-30 03:49:58 -07:00 |
|