iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jee Jee Li 1e5ad9b74f [Bugfix] Fix Qwen3NextForCausalLM packed_modules_mapping (#35413) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-02-26 19:46:30 -08:00
Jee Jee Li 7291d1b288 [Bugfix] Fix kernel benchmark (#33752) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-02-22 21:18:08 -08:00
Jee Jee Li 978a37c823 [Model] GLM adaptation (#34124) 2026-02-09 17:32:52 +08:00
Jee Jee Li db4ede9743 [Model] Enable Step3p5ForCausalLM testing (#33755) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-02-07 05:25:24 -08:00
Jee Jee Li 73b243463b [BugFix] Add env variable to control PDL in LoRA (#32836) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-01-25 16:32:30 +08:00
Jee Jee Li 2636d76257 [Misc] Remove unused ModelKeys (#32608) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-01-19 17:34:59 +00:00
Jee Jee LiCyrus Leunggemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		05e8981234 [Doc] Improve LoRA docs (#32159) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-12 02:19:17 -08:00
Jee Jee Li 49568d5cf9 [Doc] Improve MM models LoRA notes (#31979) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-01-08 08:55:22 -08:00
Jee Jee Li 30399cc725 UX: add vLLM env info in '/server_info' (#31899) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-01-07 17:13:02 +00:00
Jee Jee Li cbd4690a03 [LoRA]Disable linear LoRA kernel PDL (#31777) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-01-06 23:12:25 +08:00
Jee Jee Li d5503ca7f9 [LoRA] LoRA PDL improvement (#31660) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-01-05 08:28:46 +00:00
Jee Jee Li f099cd557a [Bugfix] Fix AttributeError: 'Stream' object has no attribute 'dp_size' (#31663) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-01-05 02:31:31 +00:00
Jee Jee Libk-201prashanth058Anexdeus
 		ce1eafd1a5 [Core] Initialize LoRA support for tower and connector in multi-modal models (#26674) 
Signed-off-by: bk-201 <joy25810@foxmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: prashanth058 <prashanth.dannamaneni@uipath.com>
Co-authored-by: bk-201 <joy25810@foxmail.com>
Co-authored-by: prashanth058 <prashanth.dannamaneni@uipath.com>
Co-authored-by: Anexdeus <5142168@mail.ru>
 2025-12-26 04:48:20 -08:00
Jee Jee Li c3666f56fd [Misc] Fix Qwen2-MoE shared_expert_gate (#31339) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-26 05:10:39 +00:00
Jee Jee Li 27c6c2f98c [Bugfix] Fix MoE LoRA bin/pt loading (#31161) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-23 19:09:15 +08:00
Jee Jee Li 6b16fff01b [Bugfix] Fix Jais2ForCausalLM (#31198) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-23 07:44:01 +00:00
Jee Jee Li 097978a15d [Kernel] Enable fused_qknorm_rope_kernel supports partial rope (#30821) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-21 18:39:22 -08:00
Jee Jee Li 0e391e7570 [Bugfix] Fix RequestOutput miss lora_request (#30636) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-16 01:36:35 -08:00
Jee Jee Li a524d1ba0a [Bugfix] Fix deepseek_v32 tokenizer_mode (#30658) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-15 04:20:31 +00:00
Jee Jee Li 67312cad11 [Misc] Split the LoRA code (#30253) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-09 00:59:31 +08:00
Jee Jee Li 80433e225e [LoRA] Reduce the loading time of MoE LoRA (#30243) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-08 13:29:47 +00:00
Jee Jee Li b0f4866a77 [CI/Build]Temporary workaround for test_default_mm_loras timeout (#30202) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-07 20:27:11 +08:00
Jee Jee Li 652ba93da3 [Bugfix] Fix FP8 MoE LoRA (#29890) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-04 18:17:49 +00:00
Jee Jee Li dd38ba3a26 [Bugfix] Fix adapter_enabled IMA (#29977) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-04 12:51:15 +08:00
Jee Jee Li b9d0504a36 [Bugfix] Revert test_tokenization.py (#29729) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-29 16:35:15 +00:00
Jee Jee LiandCyrus Leung 39e63dec7c [LoRA] Cleanup LoRA unused code (#29611) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-11-28 22:52:58 -08:00
Jee Jee Li 2f5f9acd55 [LoRA] Continue optimizing MoE LoRA weight loading (#29322) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-27 05:56:28 -08:00
Jee Jee Li c069086b9c [Bugfix] Fix getting device for MoE LoRA (#29475) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-26 23:16:07 -08:00
Jee Jee Li 1073ba68b0 [LoRA] Optimize 3D MoE logic (#29222) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-24 10:27:23 +08:00
Jee Jee Li 1489902b53 [LoRA] Cleanup FusedMoEWithLoRA (#29187) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-22 04:01:30 +00:00
Jee Jee Li 9875be6431 [LoRA][2/2]Remove LoRA extra vocab (#28545) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-21 09:46:43 +08:00
Jee Jee Li 3380ed5e11 [Doc] Add llama4 LoRA tag (#28825) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-17 14:08:48 +08:00
Jee Jee Li a9d18b5107 [Bugfix] Fix gpt_oss packed_modules_mapping (#28536) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-12 21:02:06 +08:00
Jee Jee Li 9d1c474704 [LoRA][1/N]Remove LoRA extra vocab (#28382) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-11 11:06:21 -08:00
Jee Jee Li 21b82f4ea2 [Kernel] LoRA triton kernels support PDL (#27402) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-07 08:05:48 +00:00
Jee Jee Li 32257297dd [CI/Build] Remove the flaky gpt-oss lora test (#27966) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-03 16:50:06 +08:00
Jee Jee Li 3a5de7d2d6 [Bugfix] Fix KDA output (#27905) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-01 11:54:36 +08:00
Jee Jee Li bc4486d609 [Kernel] Enable FusedMoEModularKernel support bias (#27754) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-01 02:05:12 +00:00
Jee Jee Li 0384aa7150 [CI/Build] Add gpt-oss LoRA test (#27870) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-31 22:17:21 +08:00
Jee Jee Li f4e8154076 [Kernel] Enable moe LoRA kernel support FP16 (#27468) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-27 19:48:37 +08:00
Jee Jee LiandRoger Wang 2d631d28c6 [Doc] Slight improvement to M2 and beyond (#27554) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-10-27 09:02:10 +00:00
Jee Jee Li 9771e0b432 [Bugfix] Add missing 'is_internal_router' attribute to FusedMoEWithLoRA (#27351) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-22 08:19:12 -07:00
Jee Jee Li abf3db40ef [Core] Handle MoE LoRA edge cases (#27335) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-22 13:14:33 +00:00
Jee Jee Li daec4d2624 [Model]Improve Qwen3VLMoeForConditionalGeneration packed_modules_mapping (#27096) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-17 04:47:00 -07:00
Jee Jee Li fec2b341ad [Kernel] Lazy import FlashInfer (#26977) 2025-10-17 04:48:18 +00:00
Jee Jee Li 87bc0c492f [Bugfix] Fix ReplicatedLinearWithLoRA (#27065) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-17 04:43:16 +00:00
Jee Jee Li 9f4e30904b [Model] Fix Qwen3VL mm mapping (#27027) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-16 09:45:59 -07:00
Jee Jee Li fdd32750f0 [CI/Build] Cleanup LoRA test (#26752) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-14 12:06:35 +00:00
Jee Jee Li 98f30b8cba [Model] Fix Skywork R1V mlp (#26673) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-12 22:42:17 -07:00
Jee Jee Li f0a30a067b [Bugfix] Fix qwen-moe packed_modules_mapping (#26634) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-11 15:21:33 +00:00
Jee Jee Li 1b2c440cd6 [Core] Relax the LoRA max rank (#26461) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-08 23:47:14 -07:00
Jee Jee Li 0c52d6ef81 [Bugfix] Set the minimum python version for gpt-oss (#26392) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-08 20:35:49 -07:00
Jee Jee Li 67f3fb0844 [Bench] Add DeepSeekV32 to MoE benchmark (#25962) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-30 14:13:48 -07:00
Jee Jee Li e61eb5e09d [Model] Remove MotifForCausalLM (#25866) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-30 00:36:30 +08:00
Jee Jee Li 0fa673af4c [V0 deprecation] Clean up LoRA (#25686) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-25 18:12:33 +00:00
Jee Jee Li 52d0cb8458 [Model] Improve DotsOCRForCausalLM (#25466) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-25 07:58:08 +08:00
Jee Jee Li 5abb117901 [Core] Ensure LoRA linear respect the base_layer's tp_size and tp_rank (#25487) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-23 18:19:25 +00:00
Jee Jee Li 273690a50a [Core] Optimize LoRA weight loading (#25403) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-23 18:19:45 +08:00
Jee Jee Li 2821986450 [Core] Modify the initialization parameters of the lora manager (#25249) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-19 18:01:28 +00:00
Jee Jee Li 37970105fe [Model] Improve Pooling Model (#25149) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-18 11:04:21 +00:00
Jee Jee Li 04ad0dc275 [benchmark] Add triton version in the moe tuned config (#24769) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-16 14:10:54 +08:00
Jee Jee Li 99bfef841f [Bugfix] Fix GPUModelRunner has no attribute lora_manager (#24762) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-12 23:55:14 -07:00
Jee Jee Li 60a0951924 [Bugfix] Fix BNB name match (#24735) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-12 11:12:01 +00:00
Jee Jee Li 12a8414d81 [Qwen3-Next] MoE configs for H20 TP=1,2,4,8 (#24707) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-12 10:06:26 +08:00
Jee Jee Li 817beef7f3 [Bugifx] Fix qwen-next packed_modules_mapping (#24656) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-11 22:26:17 +08:00
Jee Jee Li d11ec124a0 [Bench] Add qwen-next in benchmark_moe.py (#24661) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-11 21:29:43 +08:00
Jee Jee Li bb3eb80d92 [Core] Split LoRA layers (#24574) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-10 07:47:51 -07:00
Jee Jee Li 9ad0688e43 [Bugfix] Fix hidden_size for multimodal classification model (#24501) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-09 10:37:25 -07:00
Jee Jee Li 8d7f39b48c [Model] Remove quantized mixtral (#24437) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-08 11:02:14 -07:00
Jee Jee Li 6f4a82f8b5 [Model] Enable BNB support for qwen2_5_omni_thinker (#24420) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-08 09:37:08 -07:00
Jee Jee Li 62f66be1f7 [Bugfix] Fix Qwen3-coder moe tuned config (#24072) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-07 05:19:46 +00:00
Jee Jee Li 7555d6b34a [Bugfix] Fix test_mixtral_moe (#24371) 2025-09-06 09:32:03 -07:00
Jee Jee Li 94866d7c93 [Misc] Slight improve deepgemm print (#24085) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-04 16:06:51 +00:00
Jee Jee LiandWentao Ye dc1a53186d [Kernel] Update DeepGEMM to latest commit (#23915) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-09-01 02:38:04 -07:00
Jee Jee Li 628d00cd7b [Bugfix] Fix test_lora_resolvers.py (#23984) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-30 11:16:11 +00:00
Jee Jee Li b4f9e9631c [CI/Build] Clean up LoRA test (#23890) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-28 23:28:35 -07:00
Jee Jee Li e03940762b [CI/Build] Reduce LoRA layer test cases (#23721) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-27 10:59:35 +00:00
Jee Jee Li 9de25c294b [CI/Build] Remove redundant LoRA model tests (#23706) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-27 05:51:50 +00:00
Jee Jee Li fdeb3dac13 [Model] fix DeepSeek e_score_correction_bias dtype to fp32 (#23640) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-26 20:09:47 +08:00
Jee Jee LiandMichael Goin 9b5f64238f [Bugfix] Fix Qwen25VL packed_modules_mapping (#23604) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-26 01:09:14 -07:00
Jee Jee Li 170e8ea9ea [Misc] Unified linear print info (#23516) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-24 20:13:51 -07:00
Jee Jee Li 65197a5fb3 [Misc] Modify CacheConfig import (#23459) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-23 06:05:27 +00:00
Jee Jee Li 285178b3b8 [V0 Deprecation] Remove V0 LoRA test (#23418) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-22 09:56:51 +00:00
Jee Jee Li c6d80a7a96 [Model] Improve olmo and olmo2 (#23228) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-20 12:47:05 +00:00
Jee Jee Li 4d9c61993a [Bugfix] Fix benchmark_moe.py (#23177) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-19 13:39:40 +00:00
Jee Jee Li 4d4061b6e7 [Kernel] Add cuda kernel for gpt_oss activation (#22951) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-17 05:03:24 +00:00
Jee Jee Li 6d3da472bc [Misc] Add --save-dir option to benchmark_moe (#23020) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-16 07:26:10 +00:00
Jee Jee Li 48f4636927 [Misc] Ignore ep_kernels_workspace (#22807) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-15 05:58:03 -07:00
Jee Jee Li 81f4b96481 [Kernel] Add cuda kernel for gpt_oss activation (#22538) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-14 17:21:29 -07:00
Jee Jee Li 92ff41abea [Model] Modify the gate implementation of glm4_moe (#22832) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-14 05:28:50 -07:00
Jee Jee Li fde0b611a3 [Model] Decouple glm4v (#22751) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-12 17:13:17 -07:00
Jee Jee Li b8a9d0e429 [Misc] remove GH discussions link (#22722) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-12 03:15:33 -07:00
Jee Jee Li 384a052971 [Misc] benchmark_moe supports expert parallel (#22251) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-11 00:13:27 -07:00
Jee Jee Li 0c5254b82a [oss] Init gpt-oss bf16 support (#22508) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-09 20:19:13 -07:00
Jee Jee Li 0edc0cd52b [Bugfix] Fix CI moe kernel failure (#22556) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-09 00:03:29 -07:00
Jee Jee Li 8e6c7e873f [Bugfix] Fix MoE BNB version (#22260) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-05 19:56:22 -07:00
Jee Jee Li a7b8788d2c [Misc] Modify the organization of GLM series (#22171) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-03 23:51:20 -07:00
Jee Jee Li 8d705996df [Misc] Minor enhancement of benchmark_moe (#22068) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-02 01:35:30 +08:00
Jee Jee Li 28b18cc741 [Quantization] Enable BNB support for InternS1 (#21953) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-01 11:09:54 +00:00
Jee Jee Li 0f7919fca0 [Misc] Expand SUPPORTED_HIDDEN_SIZES for DeepEP low-latency kernels (#21818) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-30 20:41:12 -07:00