|
6849ca696a
|
[Quantization] Add ModelOpt IQ2_XS, IQ2_XXS, and Q8_0 support via b12x (#59604)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Luke Alonso <lalonso@gmail.com>
|
2026-10-13 09:06:33 -07:00 |
|
|
|
31e90ac370
|
[Attention] Enable FA4 sparse MQA decode on Blackwell (#55866)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-10-13 09:31:37 -04:00 |
|
|
Linze Shi
|
e6e84f350c
|
[Feature][Pooling] Add offline fixed-length late chunking (#60164)
Signed-off-by: Linze-Shi <linzeshi0@gmail.com>
|
2026-10-13 11:16:21 +00:00 |
|
|
Juan Pérez de AlgabaandOpenAI Codex
|
7173c64e2e
|
[Security] Bound decoded and downloaded media per request (#61226)
Signed-off-by: juanpas97 <juan.perezalgaba@gmail.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-10-13 10:17:14 +00:00 |
|
|
|
2cc2e68d1c
|
[Core][Mamba] Remove leftovers of mamba_cache_mode="all" (#60043)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-13 00:58:59 -07:00 |
|
|
|
380638afbb
|
[Model] Add Clef-Flash support (#61308)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-10-13 05:46:37 +00:00 |
|
|
Divakar VermaandClaude Sonnet 5
|
f7ba007b5e
|
[ROCm][MoE] Split AiterExperts by weight-quantization key (#60322)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
|
2026-10-13 00:13:33 -05:00 |
|
|
|
189bbfb9f4
|
Add MoNe experts pruning support (#50415)
Signed-off-by: Ilya Boytsov <ieboytsov@nebius.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Ilya Boytsov <ieboytsov@nebius.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-12 23:18:43 +00:00 |
|
|
Almog TavorandRobert Shaw
|
69d54eb078
|
[Docs] Fix broken llm-d.ai links on the llm-d integration page (#61022)
Signed-off-by: Almog Tavor <almogtavor@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-10-12 21:26:43 +00:00 |
|
|
Woosuk KwonandClaude Opus 5.5
|
e1a5086256
|
[Attention] Remove cascade attention (#61126)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-12 10:41:07 -07:00 |
|
|
|
a7b7bc1fbe
|
[Metrics] Count finished requests by watermark status (#58715)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-10-12 17:21:22 +00:00 |
|
|
|
fa39582756
|
[Frontend] Serve LiquidAI/d1-3B on /v1/systemone (#60676)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-12 13:00:47 -04:00 |
|
|
Misha GoinandClaude
|
d79b52b182
|
[Misc] Consolidate csrc shared headers and prune dead code (#60721)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-12 09:46:22 -07:00 |
|
|
|
e0317b28ce
|
[Feature] Support batch invariance on ROCm (#52231)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-10-12 10:53:26 -05:00 |
|
|
Hoai VoandClaude Opus 5.5
|
f3cb10b690
|
[Model][LoRA] Enable LoRA support for BERT and RoBERTa embedding models (#61033)
Signed-off-by: hoaivoke <vokehoai@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-12 14:09:53 +00:00 |
|
|
|
092c35dd15
|
[Model] Support Cloudflare/clef-omni on /v1/systemone (#61064)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-10-12 21:27:34 +08:00 |
|
|
Tianyu Guo
|
258b97ff6b
|
[Bugfix][EC Connector] Size transfers by measured encoder output widths (#57904)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
|
2026-10-12 12:32:39 +00:00 |
|
|
Jimmy WangandJuan Pérez de Algaba
|
2534a96fcf
|
[Docs] List additional unauthenticated endpoints (#61081)
Signed-off-by: Jimmy Wang <53066820+JimmyWang0417@users.noreply.github.com>
Co-authored-by: Juan Pérez de Algaba <124347725+jperezdealgaba@users.noreply.github.com>
|
2026-10-12 11:47:52 +02:00 |
|
|
Thuongvu HoandNicolò Lucchesi
|
99b44c7a42
|
[Metrics] Add admission rejection metrics (#55812)
Signed-off-by: Thuongvu Ho <thuongvuho@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
|
2026-10-12 08:31:07 +00:00 |
|
|
|
4163a80142
|
[Rust Frontend] Return weight-version spans on final outputs (#61070)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-12 15:56:59 +08:00 |
|
|
|
d4e92ccc37
|
[Bugfix][Frontend] Honor endpoint plugin route overrides (#58839)
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-12 03:58:15 +00:00 |
|
|
|
93d2cab35e
|
[Security] Fix P2P ZMQ socket exhaustion via unbounded peer sessions (#51504)
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
|
2026-10-11 11:39:11 -04:00 |
|
|
aoshen02andCodex
|
239a3fa3a8
|
[Sleep] Retain draft-private weights with existing parameter selector (#61046)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-11 23:30:56 +08:00 |
|
|
tc-mbandCyrus Leung
|
c85616efc6
|
[Model] Support MiniCPM-V 4.7 (#58674)
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-11 10:21:10 +00:00 |
|
|
aoshen02andOpenAI Codex
|
1789f09588
|
[Frontend] Explain disabled scale-out endpoints with opt-in hints (#61042)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-10-11 08:58:07 +00:00 |
|
|
Shuolei Wangandaoshen02
|
b89852aad0
|
[RL][Frontend] Return token-level weight-version spans on final outputs (#59582)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-11 11:33:29 +08:00 |
|
|
|
f2376cdc4d
|
[Fast Start] Preload the FlashInfer autotune table on the weight cache daemon (#60085)
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-10-10 15:48:06 +00:00 |
|
|
wang.yuqi
|
c1692c85f1
|
[Misc] mv chat_utils.py to renderers. (#60522)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-10-10 06:50:34 +00:00 |
|
|
Misha GoinandClaude Opus 5.5
|
29e4d2d42e
|
[Frontend][Build] Add vllm download-kernels for FlashInfer precompiled kernels (#58765)
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-09 21:22:35 -07:00 |
|
|
|
3c802af3cf
|
[Feat][Frontend] structured decisions: DiffusionGemma read strategy (#60466)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-10 09:33:22 +08:00 |
|
|
|
04b02fc8a5
|
[Core] shm tensor arena for efficient cpu->gpu worker broadcast (#51207)
Signed-off-by: Bolin Sun <bolins@nvidia.com>
Signed-off-by: <bolins@nvidia.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-09 16:13:24 -07:00 |
|
|
Matthew BonanniandClaude Opus 5.5
|
4c43254d52
|
[HiSparse] Report host-pool block residency as HiSparse metrics (#59485)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-09 16:45:34 -06:00 |
|
|
Giancarlo Delfin
|
ffdde24f54
|
[Docs] Document adaptive verification without a confidence head (#60913)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-10-09 13:54:40 -07:00 |
|
|
Taewoong KimandCyrus Leung
|
75e3b17c05
|
[Frontend] Add cache salt support to generative scoring (#60841)
Signed-off-by: Taewoong Kim <ktw2172@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-09 16:07:50 +00:00 |
|
|
|
90ba2f34b5
|
[Build] Find compatible published wheels for precompiled installs (#58871)
Signed-off-by: greg pereira <grpereir@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-09 08:05:54 -07:00 |
|
|
Abhijit Roy
|
970dc63478
|
[Docs] Add Giga-Embeddings bidirectional Qwen3 usage to embed docs (#60854)
Signed-off-by: Abhijit <abroy@redhat.com>
|
2026-10-09 14:41:38 +00:00 |
|
|
Praneeth Reddy AandLi, Jiang <jiang1.li@intel.com>
|
1c47ebfc57
|
[Docs] Fix VLLM_CPU_KVCACHE_SPACE default and units (#60053)
Signed-off-by: Praneeth <praneethreddyarikatla@gmail.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-10-09 19:43:55 +08:00 |
|
|
|
5cadabfafd
|
[Docs] Add Qwen2.5-Coder-7B-Instruct to batch invariance tested models (#46368)
Signed-off-by: Yuval Luria <yluria@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-10-09 10:41:29 +00:00 |
|
|
Chao Lei
|
bc713cf8cc
|
[Bugfix]MooncakeConnect add query timeout and retry (#55923)
|
2026-10-09 09:19:33 +00:00 |
|
|
Taewoong KimandChauncey
|
18c19eaf23
|
[Frontend] Add cache salt and tracing to structured decisions (#60786)
Signed-off-by: Taewoong Kim <ktw2172@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-10-09 08:53:50 +00:00 |
|
|
|
a6f43e57d6
|
[ROCm] Enable the cuMem CUDA-graph pool offload on ROCm (#59523)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: indianspeedster <cspandey016@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: indianspeedster <cspandey016@gmail.com>
|
2026-10-09 08:29:55 +00:00 |
|
|
Maroon AyoubandClaude
|
8a03ccf7b3
|
[Frontend] Add return_mm_kwargs to render requests (#60728)
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-09 08:14:13 +00:00 |
|
|
|
8cbd5d0300
|
[Frontend] structured decisions: noul and score question types (#60696)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: DeepSeek <noreply@deepseek.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-09 04:48:16 +00:00 |
|
|
Rajeev Warrier
|
81d9c45e4b
|
[Docs] [Multimodal] Note that per-request media_io_kwargs needs --trust-request-mm… (#60598)
Signed-off-by: Rajeev Warrier <warrier.rajeev.95@gmail.com>
|
2026-10-09 04:23:07 +00:00 |
|
|
Rémi Delhaye
|
a1cd0ec650
|
[Model] Add zerank-2 reranker (#60415)
Signed-off-by: rdlhio <contact@rdlh.io>
|
2026-10-09 03:01:51 +00:00 |
|
|
|
201e537d98
|
[Bugfix][LoRA] Validate incompatible MoE adapter formats (#59924)
Signed-off-by: Brian Weston <brianweston@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io>
|
2026-10-09 09:49:13 +08:00 |
|
|
sungbin1015andFlora Feng
|
2ddde19e8e
|
[Frontend] Support min_p in the Responses API (#48084)
Signed-off-by: sungbin1015 <sbin@solbox.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-08 20:38:17 +00:00 |
|
|
|
ae9adec905
|
[Frontend] Expose /v1/systemone without an opt-in flag (#60677)
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-08 20:21:05 +00:00 |
|
|
Doug Smith
|
242e4213fc
|
[Doc] Require GCC 13 for CUDA source builds (#60259)
Signed-off-by: Doug Smith <dosmith@redhat.com>
|
2026-10-08 09:43:39 -07:00 |
|
|
Woosuk Kwon
|
194bd72be7
|
[Compile] Remove torch.compile-based sequence parallelism and AsyncTP (#60517)
|
2026-10-08 08:28:23 -07:00 |
|