 Alexandre JUAN
|
2f6e6b33fb
|
[Bugfix] Fix isinstance check for tensor types in _load_prompt_embeds to use dtype comparison (#21612)
Signed-off-by: Alexandre Juan <a.juan@netheos.net>
|
2025-07-25 20:11:10 -07:00 |
|
|
Farzad AbdolhosseiniandPatrick Li
|
62965de5fe
|
[Model] Ultravox: Support Llama 4 and Gemma 3 backends (#17818)
Signed-off-by: Farzad Abdolhosseini <farzad@fixie.ai>
Signed-off-by: Patrick Li <patrick8289@gmail.com>
Co-authored-by: Patrick Li <patrick8289@gmail.com>
|
2025-07-25 18:12:31 -07:00 |
|
|
Alex Kogan
|
7ae75fa6d0
|
[Feature] Add support for MoE models in the calibration-free RTN-based quantization (#20766)
Signed-off-by: Alex Kogan <alex.kogan@oracle.com>
|
2025-07-25 18:09:34 -07:00 |
|
|
Rui Qiao
|
c7742d6113
|
[Bugfix] Always set RAY_ADDRESS for Ray actor before spawn (#21540)
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
|
2025-07-25 17:08:30 -07:00 |
|
|
Rui Qiao
|
cea96a0156
|
[Bugfix] Fix sync_and_slice_intermediate_tensors (#21537)
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
|
2025-07-25 17:07:58 -07:00 |
|
|
Wentao Ye
|
75d29cf4e1
|
[Perf] Cuda Kernel for Int8 Per Token Group Quant (#21476)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2025-07-25 17:07:07 -07:00 |
|
|
mgazzandCyrus Leung
|
e189b50f53
|
Add support for Prithvi in Online serving mode (#21518)
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2025-07-25 07:01:27 -07:00 |
|
|
who who whoandamd-ruitang3
|
b3caeb82e7
|
[ROCm][AITER] Enable fp8 kv cache on rocm aiter backend. (#20295)
Signed-off-by: fsx950223 <fsx950223@outlook.com>
Signed-off-by: amd-ruitang3 <Rui.Tang2@amd.com>
Co-authored-by: amd-ruitang3 <Rui.Tang2@amd.com>
|
2025-07-25 06:50:21 -07:00 |
|
|
Chih-Chieh YangandYu Chin Fabian Lim
|
eab2f3980c
|
[Model] Replace Mamba2 RMSNorm Gated with Fused Triton Kernel (#20839)
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
Signed-off-by: Yu Chin Fabian Lim <fabian.lim@gmail.com>
Signed-off-by: Chih-Chieh Yang <7364402+cyang49@users.noreply.github.com>
Co-authored-by: Yu Chin Fabian Lim <fabian.lim@gmail.com>
|
2025-07-25 06:49:36 -07:00 |
|
|
kourosh hakhamaneshi
|
9fe98d4250
|
[Frontend] Add request_id to the Request object so they can be controlled better via external load balancers (#21009)
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
|
2025-07-25 06:49:11 -07:00 |
|
|
bigshanedogg
|
29c6fbe58c
|
[MODEL] New model support for naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B (#20931)
Signed-off-by: bigshanedogg <bigshane319@gmail.com>
|
2025-07-25 06:05:42 -07:00 |
|
|
xyxinyangandzhouchong
|
c72f049cb4
|
[Model] Fix Ernie4.5MoE e_score_correction_bias parameter (#21586)
Signed-off-by: zhouchong <zhouchong03@baidu.com>
Co-authored-by: zhouchong <zhouchong03@baidu.com>
|
2025-07-25 06:02:53 -07:00 |
|
|
Mengqing Cao
|
f3a683b7c9
|
[Bugfix][Logprobs] Fix logprobs op to support more backend (#21591)
Signed-off-by: MengqingCao <cmq0113@163.com>
|
2025-07-25 05:53:07 -07:00 |
|
|
Cyrus Leung
|
46d81d6951
|
[V1] Get supported tasks from model runner instead of model config (#21585)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-07-25 05:36:45 -07:00 |
|
|
Jee Jee Li
|
5c3f2628d5
|
[Quantization] Enable BNB support for more MoE models (#21370)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2025-07-25 03:57:34 -07:00 |
|
|
Kebe
|
7311f74468
|
[Bugfix] GGUF: fix AttributeError: 'PosixPath' object has no attribute 'startswith' (#21579)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2025-07-25 03:42:23 -07:00 |
|
|
Xu Wenqing
|
8ed01e32f7
|
Add H20-3e fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct (#21598)
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
|
2025-07-25 02:36:55 -07:00 |
|
|
hfan
|
fde60ee775
|
[Model] Fix a check for None but the return value was empty list in Gemma3 MM vision_embeddings (#21479)
Signed-off-by: Hongmin Fan <fanhongmin@google.com>
|
2025-07-25 13:46:06 +08:00 |
|
|
Jason Gu
|
b38bc652ac
|
[Model] Support tensor parallel for timm ViT in Deepseek_vl2 (#21494)
Signed-off-by: wzqd <1057337859@qq.com>
|
2025-07-24 22:45:16 -07:00 |
|
|
Ning Xie
|
adaf2c6d4f
|
[Bugfix] fix modelscope snapshot_download serialization (#21536)
Signed-off-by: Andy Xie <andy.xning@gmail.com>
|
2025-07-24 22:44:38 -07:00 |
|
|
Benji Beck
|
965bc71b04
|
Integrate TensorSchema with shape validation for Phi3VImagePixelInputs (#21232)
Signed-off-by: Benji Beck <benjibeck@meta.com>
|
2025-07-24 21:43:52 -07:00 |
|
|
Nick Hill
|
9c8b2c2a8a
|
[DP] Support api-server-count > 0 in hybrid DP LB mode (#21510)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-07-24 20:18:16 -07:00 |
|
|
Varun Sundar RabindranathandVarun Sundar Rabindranath
|
2212cd6cfb
|
[Bugfix] DeepGemm utils : Fix hardcoded type-cast (#21517)
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
|
2025-07-24 20:17:29 -07:00 |
|
|
Burkhard RingleinandThomas Parnell
|
ce3a9b1378
|
[Kernel] adding fused_moe configs for upcoming granite4 (#21332)
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
|
2025-07-24 20:16:59 -07:00 |
|
|
Yuxuan Zhang
|
2ce90e5b01
|
Fix GLM-4 PP Missing Layer When using with PP. (#21531)
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
|
2025-07-24 20:07:38 -07:00 |
|
|
Wentao Ye
|
633f6e804b
|
[Bug] Fix DeepGemm Init Error (#21554)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2025-07-24 20:07:22 -07:00 |
|
|
Cyrus Leung
|
34ddcf9ff4
|
[Frontend] run-batch supports V1 (#21541)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-07-24 20:05:55 -07:00 |
|
|
Woosuk Kwon
|
fe56180c7f
|
[MoE] More balanced expert sharding (#21497)
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
|
2025-07-24 15:56:08 -07:00 |
|
|
weiliang
|
2dd72d23d9
|
update flashinfer to v0.2.9rc1 (#21485)
Signed-off-by: Weiliang Liu <weiliangl@nvidia.com>
|
2025-07-24 14:06:11 -07:00 |
|
|
|
6066284914
|
[P/D] Support CPU Transfer in NixlConnector (#18293)
Signed-off-by: Juncheng Gu <juncgu@gmail.com>
Signed-off-by: Richard Liu <ricliu@google.com>
Co-authored-by: Richard Liu <39319471+richardsliu@users.noreply.github.com>
Co-authored-by: Richard Liu <ricliu@google.com>
|
2025-07-24 17:58:42 +01:00 |
|
|
Rui Qiao
|
1e9ea8e69d
|
[P/D] Move FakeNixlWrapper to test dir (#21328)
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
|
2025-07-24 08:53:45 -07:00 |
|
|
Chaojun Zhang
|
d9f9a3fd96
|
[XPU] Conditionally import CUDA-specific passes to avoid import errors on xpu platform (#21036)
Signed-off-by: chzhang <chaojun.zhang@intel.com>
|
2025-07-24 23:23:36 +08:00 |
|
|
Shu Wang
|
1b25f1fe75
|
Update flashinfer CUTLASS MoE Kernel (#21408)
Signed-off-by: Shu Wang. <shuw@nvidia.com>
|
2025-07-24 08:13:31 -07:00 |
|
|
Harry Mellor
|
13abd0eaf9
|
[Model] Officially support Emu3 with Transformers backend (#21319)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-07-24 03:22:12 -07:00 |
|
|
Lucas Wilkinson
|
61b8cea3b4
|
[Attention] Optimize FlashInfer MetadataBuilder Build call (#21137)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-07-24 03:21:46 -07:00 |
|
|
Chauncey
|
6da0078523
|
[Feat] Allow custom naming of vLLM processes (#21445)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2025-07-24 03:15:23 -07:00 |
|
|
Rui Qiao
|
73e3949d07
|
[Misc] Improve comment for DPEngineCoreActor._set_cuda_visible_devices() (#21501)
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
|
2025-07-24 03:13:40 -07:00 |
|
|
Shintarou OkadaandHarry Mellor
|
6eca337ce0
|
Replace --expand-tools-even-if-tool-choice-none with --exclude-tools-when-tool-choice-none for v0.10.0 (#20544)
Signed-off-by: okada <kokuzen@gmail.com>
Signed-off-by: okada shintarou <okada@preferred.jp>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-07-24 02:56:36 -07:00 |
|
|
Yuxuan Zhang
|
85bda9e7d0
|
remove GLM-4.5 quantization wrong Code (#21435)
|
2025-07-24 01:52:43 -07:00 |
|
|
22quinn
|
610852a423
|
[Core] Support model loader plugins (#21067)
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
|
2025-07-24 01:49:44 -07:00 |
|
|
Nick Hill
|
f0f4de8f26
|
[Misc] Fix duplicate FusedMoEConfig debug messages (#21455)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-07-24 01:27:30 -07:00 |
|
|
Zhou Fang
|
fc5f756db4
|
[v1][Core] Clean up usages of SpecializedManager (#21407)
Signed-off-by: Zhou Fang <fang.github@gmail.com>
|
2025-07-24 00:40:11 -07:00 |
|
|
Chengji YaoandSimon Mo
|
e74bfc70e4
|
[TPU][Bugfix] fix moe layer (#21340)
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
|
2025-07-24 00:38:39 -07:00 |
|
|
Harry Mellor
|
dde295a934
|
Deduplicate Transformers backend code using inheritance (#21461)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-07-24 00:16:23 -07:00 |
|
|
Julien Denize
|
6d8d0a24c0
|
Add think chunk (#21333)
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
|
2025-07-23 21:51:32 -07:00 |
|
|
|
11ef7a611e
|
[BugFix] Set CUDA_VISIBLE_DEVICES before spawning the subprocesses (#21211)
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
|
2025-07-23 21:44:04 -07:00 |
|
|
Woosuk Kwon
|
dc2f159f8a
|
Dump input metadata on crash for async scheduling (#21258)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-07-23 21:10:30 -07:00 |
|
|
|
d5b981f8b1
|
[DP] Internal Load Balancing Per Node [one-pod-per-node] (#21238)
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2025-07-23 20:57:32 -07:00 |
|
|
Nick Hill
|
eec6942014
|
[BugFix] Fix KVConnector TP worker aggregation (#21473)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-07-23 20:56:49 -07:00 |
|
|
KazusatoOokoandKazusatoOko
|
fd48d99ffd
|
[BugFix]: Batch generation from prompt_embeds fails for long prompts (#21390)
Signed-off-by: KazusatoOko <kazusto.oko@sakana.ai>
Co-authored-by: KazusatoOko <kazusto.oko@sakana.ai>
|
2025-07-23 20:43:17 -07:00 |
|