iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
7,293 Commits 3 Branches 0 Tags
f59fc60fb317e7e04456de50b7abce99a9017225
Commit Graph
1973 Commits
Author SHA1 Message Date
Wentao Ye 879f69bed3 [Refactor] Remove duplicate ceil_div (#20023) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-25 05:19:09 +00:00
Isotr0py 2273ec322c Revert "Fix(models/siglip): Add compatibility for Gemma models quantized by llm-compressor" (#20030) 2025-06-25 11:23:29 +08:00
Wentao Ye a6c4b87fbc Revert "[Feature] Integrate new deepgemm (#19820)" (#20049) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-24 19:45:22 -07:00
Brayden Zhong 1afa9948f5 [Llama4] Update attn_temperature_tuning (#19997) 
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-06-24 22:42:53 -04:00
Boyuan Feng c01d1c5aba use .dev for version comparison with pytorch nightly release (#20031) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-06-24 21:52:16 +00:00
Wentao Ye c6e3bba8e6 [Feature] Integrate new deepgemm (#19820) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-24 12:51:56 -07:00
Vadim Gimpelson 9a3b88328f [PERF] Speedup of MRoPE prepare inputs (#19939) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@centml.ai>
 2025-06-23 23:01:26 -07:00
Jun-Howie dd2ccf8dde Feat Dynamic Quantization for MoE Layers in GPTQ Marlin Backend (#19395) 2025-06-24 07:23:28 +09:00
Isotr0py 61f4fc5dc6 [Bugfix][v1] Fix step pooler implementation and step pooling usage in v1 (#19956) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-06-23 18:38:06 +00:00
Tyler Michael SmithandVarun Sundar Rabindranath 68aaeb3749 [EP+DP] Optimize the little operations in the DeepGEMM + DeepEP low latency case (#19885) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-06-23 11:07:47 -07:00
Isotr0py 5111642a6f [Doc] Update V1 status for decoder-only embedding models (#19952) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-06-23 09:31:06 +00:00
Vensen 493c275352 Fix(models/siglip): Add compatibility for Gemma models quantized by llm-compressor (#19643) 
Signed-off-by: Vensenmu <vensenmu@gmail.com>
 2025-06-23 03:40:28 +00:00
Ye (Charlotte) Qi 33d51f599e [BugFix] Add an env to disable moe chunking to work around compile incompatibility (#19642) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-06-22 15:17:49 -07:00
Woosuk KwonandRoger Wang 2c5302fadd [Multimodal] Optimize Qwen2/2.5-VL startup time (#19756) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Roger Wang <hey@rogerw.me>
Co-authored-by: Roger Wang <hey@rogerw.me>
 2025-06-21 20:01:07 +00:00
汪志鹏 c3bf9bad11 [New model support]Support Tarsier2 (#19887) 
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2025-06-21 04:01:51 +00:00
Isotr0py 6f170f11dd [Bugfix] Fix bnb 8bit model weights loading (#19917) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-06-21 03:29:09 +00:00
Chendi.Xue 7e8977fcd4 [custom_op][vllm-plugin] update custom_op class to use op_registry (#19164) 
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
 2025-06-20 07:44:56 -07:00
Adrianandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> f1e840e842 [Model] GPT2ForSequenceClassification model (#19663) 
Signed-off-by: nie3e <adrcwiek@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-06-20 12:07:41 +00:00
qli88 e3a3e4db46 [Bugfix] Enable PP with AITER+V1 (#19822) 
Signed-off-by: Qiang Li <qiang.li2@amd.com>
 2025-06-20 12:43:20 +08:00
Xerxesandxinnan.hou e41bf15cd0 [Chore]: qwen3-moe-type-hints-mistake (#19860) 
Co-authored-by: xinnan.hou <hxn02029096@alibaba-inc.com>
 2025-06-19 21:43:07 -07:00
TJian 2de12be428 [ROCm] [AITER] [Bugfix] Patch for AITER commit 648764942e552a8bb5fe16026703716a81f05374 (#18990) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-06-18 22:56:31 -07:00
Yu-Hang "Maxin" Tang 83ca9ae47b Mark invariant normalizer in Gemma as non-persistent (#19788) 
Signed-off-by: Yu-Hang Tang <Tang.Maxin@gmail.com>
 2025-06-18 22:56:03 -07:00
Lu Fang b1098b4072 [Bugfix] Fix the linter (#19826) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-06-18 21:44:41 -07:00
Maximilien de Bayserand22quinn 799397ee4f Support embedding models in V1 (#16188) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-06-18 21:36:33 -07:00
Jee Jee Li 4959915089 [Quantization] Modify the logic of BNB double quantization (#19742) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-06-19 03:52:09 +00:00
Woosuk Kwon d49adea1f9 [Multimodal] Use fast processor for Qwen2/2.5-VL (#19789) 2025-06-18 15:49:40 -07:00
Russell Bryant 14fdd21d39 [Core] More fixes to MultiModalEmbeddings type handling (#19715) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-06-18 22:48:29 +00:00
Chen Zhang a89209b78d [v1] Support mamba2 (#19327) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-06-18 20:34:15 +00:00
Wentao Yeandmgoin ffb2cd6b54 [Perf] Optimize moe_align_block_size CUDA kernel (#19572) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-06-17 11:49:26 -07:00
Isotr0py ca94d7fa00 [Bugfix] Update multimodel models mapping to fit new checkpoint after Transformers v4.52 (#19151) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-06-17 15:58:38 +00:00
Di Liu 5b3ad5ecf2 [DOC] fix doc typos (#19600) 
Signed-off-by: Di Liu <liu-di@sjtu.edu.cn>
 2025-06-17 11:34:53 +08:00
Dipika Sikka 6bc7b57315 [Quantization] Remove FP4 emulation; Fall-back to marlin for device < 100 (#19563) 2025-06-16 17:33:51 -04:00
Russell Bryant 90f9c2eb5c [V1] Change return type on get_multimodal_embeddings() (#19446) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-06-16 13:32:15 -04:00
qscqesze 387bdf0ab9 [Model] Add support for MiniMaxM1ForCausalLM (shares architecture with MiniMaxText01ForCausalLM) (#19677) 
Signed-off-by: QscQ <qscqesze@gmail.com>
 2025-06-16 09:47:14 -07:00
bnellnm 5e5baa91aa [Kernels] Use empty for modular MoE workspaces (#19667) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-06-16 14:58:01 +00:00
Shawn Tan 4d5424029b [Feature]:Allow for Granite MoE Hybrid models with _only_ shared experts. (#19652) 
Signed-off-by: Shawn Tan <shawntan@ibm.com>
 2025-06-16 11:14:18 +00:00
Szymon Ożóg dec66d253b [Kernel] GGUF MMVQ kernel for multiple input vectors (#18754) 
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
 2025-06-16 17:33:26 +08:00
wang.yuqi f40f763f12 [CI] Add mteb testing for rerank models (#19344) 2025-06-16 01:36:43 -07:00
Lu Fang ee1531bc38 [Bugfix][2/n] Fix speculative decoding CI - Fix test_ngram_e2e_greedy_correctness (#19644) 2025-06-14 21:15:41 -07:00
Isotr0py 2db9044ab6 [Bugfix] Fix auto dtype casting for BatchFeature (#19316) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-06-14 15:13:08 +00:00
qscqesze a24cb91600 [Model] Fix minimax model cache & lm_head precision (#19592) 
Signed-off-by: qingjun <qingjun@minimaxi.com>
 2025-06-13 12:08:20 +00:00
youkaichao d70bc7c029 [torch.compile] reorganize the cache directory to support compiling multiple models (#19064) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-06-13 15:23:25 +08:00
Boyuan Feng ce688ad46e use base version for version comparison (#19587) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-06-13 15:09:34 +08:00
Varun Sundar RabindranathandVarun Sundar Rabindranath e3b12667d4 [BugFix] : Fix Batched DeepGemm Experts (#19515) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-06-12 20:43:02 -06:00
Varun Sundar Rabindranath 9d880f594d [Misc] Turn MOE_DP_CHUNK_SIZE into an env var (#19506) 2025-06-12 18:01:16 +00:00
mobicham 96846bb360 Fix TorchAOConfig skip layers (#19265) 
Signed-off-by: mobicham <hicham@mobiuslabs.com>
 2025-06-12 22:22:53 +08:00
Jee Jee Li 73e2e0118f [Quantization] Improve AWQ logic (#19431) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-06-12 11:02:11 +00:00
rasmith 2e090bd5df [AMD][Kernel][BugFix] fix test_rocm_compressed_tensors_w8a8 for rocm (#19509) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-06-12 07:14:24 +00:00
Brayden Zhong 3f6341bf7f Add Triton Fused MoE kernel config for E=16 on B200 (#19518) 
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-06-12 04:31:51 +00:00
Varun Sundar RabindranathandVarun Sundar Rabindranath e5d35d62f5 [BugFix] Force registration of w8a8_block_fp8_matmul_deepgemm via lazy import (#19514) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-06-12 04:28:12 +00:00