Commit Graph
9 Commits
Author SHA1 Message Date
Taneem Ibrahim 0cf266a469 [Pooling] Support prompt embeddings in MRV2 decoder pooling (#56639)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-09-13 05:49:45 +00:00
Taneem Ibrahim 79bb395eea [Pooling] Improve BGE-M3 sync pooling throughput by up to 3.13% (#53464)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-08-24 08:33:32 -04:00
Taneem Ibrahim 9464529612 [ModelRunner v2] Enable decoder token-wise pooling (#50931)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-08-06 22:21:52 +00:00
Taneem Ibrahim 55c98e370a [Model Runner v2] Enable BGE M3 pooling embed token_classify (#50661)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-08-02 12:46:03 -04:00
Taneem Ibrahim 652ba59229 [Model Runner V2] Enable encoder token embedding (#50574)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-08-01 07:55:36 -04:00
Taneem Ibrahim 0f17394564 [Model Runner V2] Enable encoder token classification (#50293)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-07-31 04:15:36 +00:00
Taneem Ibrahim 43eaefba5a [ModelRunner V2] Enable sequence pooling for embedding and classification models (#48791)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-07-29 06:30:45 -07:00
Woosuk Kwon 2f8b4ce0c0 [Model Runner V2] Do not initialize sampler for non-last PP ranks (#36824)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-03-12 03:55:28 +00:00
Woosuk Kwon 86ac7bcf84 [Model Runner V2] Support pooling models (#35120)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-02-27 18:03:01 -08:00