Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

ad388d25a8 Type-fix: make execute_model output type optional (#12020) Keyun Tong 2025-01-15 01:44:56 -08:00
cbe94391eb Fix: cases with empty sparsity config (#12057) Rahul Tuli 2025-01-15 04:41:24 -05:00
994fc655b7 [V1][Prefix Cache] Move the logic of num_computed_tokens into KVCacheManager (#12003) Chen Zhang 2025-01-15 15:55:30 +08:00
3f9b7ab9f5 [Doc] Update examples to remove SparseAutoModelForCausalLM (#12062) Kyle Sayers 2025-01-15 01:36:01 -05:00
ad34c0df0f [core] platform agnostic executor via collective_rpc (#11256) youkaichao 2025-01-15 13:45:21 +08:00
f218f9c24d [core] Turn off GPU communication overlap for Ray executor (#12051) Rui Qiao 2025-01-14 21:19:55 -08:00
0794e7446e [Misc] Add multipstep chunked-prefill support for FlashInfer (#10467) Elfie Guo 2025-01-14 20:47:49 -08:00
b7ee940a82 [V1][BugFix] Fix edge case in VLM scheduling (#12065) Woosuk Kwon 2025-01-14 20:21:28 -08:00
9ddac56311 [Platform] move current_memory_usage() into platform (#11369) Shanshan Shen 2025-01-15 11:38:25 +08:00
1a51b9f872 [HPU][Bugfix] Don't use /dev/accel/accel0 for HPU autodetection in setup.py (#12046) Konrad Zawora 2025-01-15 03:59:18 +01:00
42f5e7c52a [Kernel] Support MulAndSilu (#11624) Jee Jee Li 2025-01-15 10:29:53 +08:00
a3a3ee4e6f [Misc] Merge bitsandbytes_stacked_params_mapping and packed_modules_mapping (#11924) Jee Jee Li 2025-01-15 07:49:49 +08:00
87054a57ab [Doc]: Update the Json Example of the Engine Arguments document (#12045) maang-h 2025-01-15 01:03:04 +08:00
c9d6ff530b Explain where the engine args go when using Docker (#12041) Harry Mellor 2025-01-14 16:05:50 +00:00
a2d2acb4c8 [Bugfix][Kernel] Give unique name to BlockSparseFlashAttention (#12040) Chen Zhang 2025-01-14 23:45:05 +08:00
2e0e017610 [Platform] Add output for Attention Backend (#11981) wangxiyuan 2025-01-14 21:27:04 +08:00
1f18adb245 [Kernel] Revert the API change of Attention.forward (#12038) Chen Zhang 2025-01-14 20:59:32 +08:00
bb354e6b2d [Bugfix] Fix various bugs in multi-modal processor (#12031) Cyrus Leung 2025-01-14 20:16:11 +08:00
ff39141a49 [HPU][misc] add comments for explanation (#12034) youkaichao 2025-01-14 19:24:06 +08:00
8a1f938e6f [Doc] Update Quantization Hardware Support Documentation (#12025) TJianandtjtanaa 2025-01-14 12:37:52 +08:00
078da31903 [HPU][Bugfix] set_forward_context and CI test execution (#12014) Konrad Zawora 2025-01-14 04:04:18 +01:00
1a401252b5 [Docs] Add Sky Computing Lab to project intro (#12019) Woosuk Kwon 2025-01-13 17:24:36 -08:00
f35ec461fc [Bugfix] Fix deepseekv3 gate bias error (#12002) Steve Luoandmgoin 2025-01-14 04:43:51 +08:00
289b5191d5 [Doc] Fix build from source and installation link in README.md (#12013) Yikun Jiang 2025-01-14 01:23:59 +08:00
c6db21313c bugfix: Fix signature mismatch in benchmark's get_tokenizer function (#11982) elijah 2025-01-13 23:22:07 +08:00
a7d59688fb [Platform] Move get_punica_wrapper() function to Platform (#11516) Shanshan Shen 2025-01-13 21:12:10 +08:00
458e63a2c6 [platform] add device_control env var (#12009) youkaichao 2025-01-13 20:59:09 +08:00
e8c23ff989 [Doc] Organise installation documentation into categories and tabs (#11935) Harry Mellor 2025-01-13 12:27:36 +00:00
cd8249903f [Doc][V1] Update model implementation guide for V1 support (#11998) Roger WangandCyrus Leung 2025-01-13 03:58:54 -08:00
0f8cafe2d1 [Kernel] unified_attention for Attention.forward (#11967) Chen Zhang 2025-01-13 19:28:53 +08:00
5340a30d01 Fix Max Token ID for Qwen-VL-Chat (#11980) Alex Brooks 2025-01-13 01:37:48 -07:00
89ce62a316 [platform] add ray_device_key (#11948) youkaichao 2025-01-13 16:20:52 +08:00
c3f05b09a0 [Misc]Minor Changes about Worker (#11555) Chenguang Li 2025-01-13 15:47:05 +08:00
cf6bbcb493 [Misc] Fix Deepseek V2 fp8 kv-scale remapping (#11947) Concurrensee 2025-01-13 01:05:06 -06:00
80ea3af1a0 [CI][Spec Decode] fix: broken test for EAGLE model (#11972) Sungjae Lee 2025-01-13 15:50:35 +09:00
9dd02d85ca [Bug] Fix usage of .transpose() and .view() consecutively. (#11979) Siyuan Li 2025-01-13 14:24:10 +08:00
f7b3ba82c3 [MISC] fix typo in kv transfer send recv test (#11983) Yangcheng Li 2025-01-13 13:07:48 +08:00
619ae268c3 [V1] [2/n] Logging and Metrics - OutputProcessor Abstraction (#11973) Robert Shaw 2025-01-12 23:54:10 -05:00
d14e98d924 [Model] Support GGUF models newly added in transformers 4.46.0 (#9685) Isotr0pyandCyrus Leung 2025-01-13 08:13:44 +08:00
9597a095f2 [V1][Core][1/n] Logging and Metrics (#11962) Robert Shaw 2025-01-12 16:02:02 -05:00
263a870ee1 [Hardware][TPU] workaround fix for MoE on TPU (#11764) Avshalom Manevich 2025-01-12 17:53:51 +02:00
8bddb73512 [Hardware][CPU] Multi-LoRA implementation for the CPU backend (#11100)
Akshat Tripathi Oleg Mosalov Jee Jee Li Isotr0py
2025-01-12 13:01:52 +00:00
f967e51f38 [Model] Initialize support for Deepseek-VL2 models (#11578) Isotr0pyandCyrus Leung 2025-01-12 16:17:24 +08:00
43f3d9e699 [CI/Build] Add markdown linter (#11857) Rafael Vasquez 2025-01-12 03:17:13 -05:00
b25cfab9a0 [V1] Avoid sending text prompt to core engine (#11963) Roger Wang 2025-01-11 22:36:38 -08:00
4b657d3292 [Model] Add cogagent model support vLLM (#11742) sixgodandIsotr0py 2025-01-12 03:05:56 +08:00
d697dc01b4 [Bugfix] Fix RobertaModel loading (#11940) Nicolò Lucchesi 2025-01-11 15:05:09 +01:00
a991f7d508 [Doc] Basic guide for writing unit tests for new models (#11951) Cyrus Leung 2025-01-11 21:27:24 +08:00
7a3a83e3b8 [CI/Build] Move model-specific multi-modal processing tests (#11934) Cyrus Leung 2025-01-11 13:50:05 +08:00
c32a7c7c0c [Bugfix] fused_experts_impl wrong compute type for float32 (#11921) shaochangxuandshaochangxu.scx 2025-01-11 13:49:39 +08:00
2118d0565c [Bugfix][SpecDecode] Adjust Eagle model architecture to align with intended design (#11672) Sungjae Lee 2025-01-11 13:49:38 +09:00
899136b857 [ci] fix broken distributed-tests-4-gpus (#11937) youkaichao 2025-01-11 09:07:24 +08:00
c9f09a4fe8 [mypy] Fix mypy warnings in api_server.py (#11941) Fred Reiss 2025-01-10 17:04:58 -08:00
d45cbe70f5 [Bugfix] Check that number of images matches number of <|image|> tokens with mllama (#11939) Travis Johnson 2025-01-10 16:26:00 -07:00
8a579408f3 [Misc] Update benchmark_prefix_caching.py fixed example usage (#11920) minminandRen MinMin 2025-01-11 04:39:22 +08:00
46fa98ccad [Misc] Clean up debug code in Deepseek-V3 (#11930) Isotr0py 2025-01-11 03:19:15 +08:00
aa1e77a19c [Hardware][CPU] Support MOE models on x86 CPU (#11831) Li, Jiang 2025-01-11 00:07:58 +08:00
5959564f94 Doc fix in benchmark_long_document_qa_throughput.py (#11933) Kuntai Du 2025-01-10 23:51:43 +08:00
f33e033e27 [Docs] Fix docstring in get_ip function (#11932) Kuntai Du 2025-01-10 23:51:02 +08:00
482cdc494e [Doc] Rename offline inference examples (#11927) Harry Mellor 2025-01-10 15:50:29 +00:00
20410b2fda [platform] support custom torch.compile backend key (#11318) wangxiyuanandyoukaichao 2025-01-10 23:46:51 +08:00
12664ddda5 [Doc] [1/N] Initial guide for merged multi-modal processor (#11925) Cyrus Leung 2025-01-10 22:30:25 +08:00
241ad7b301 [ci] Fix sampler tests (#11922) youkaichao 2025-01-10 20:45:33 +08:00
d85c47d6ad Replace "online inference" with "online serving" (#11923) Harry Mellor 2025-01-10 12:05:56 +00:00
ef725feafc [platform] support pytorch custom op pluggable (#11328) wangxiyuan 2025-01-10 18:02:38 +08:00
d907be7dc7 [misc] remove python function call for custom activation op (#11885) cennnandyoukaichao 2025-01-10 17:18:25 +08:00
d53575a5f0 [ci] fix gh200 tests (#11919) youkaichao 2025-01-10 16:25:17 +08:00
61af633256 [BUGFIX] Fix UnspecifiedPlatform package name (#11916) Kunshang Ji 2025-01-10 16:20:46 +08:00
ac2f3f7fee [Bugfix] Validate lora adapters to avoid crashing server (#11727) Joe RundeandJee Jee Li 2025-01-10 00:56:36 -07:00
cf5f000d21 [torch.compile] Hide KV cache behind torch.compile boundary (#11677) Chen Zhang 2025-01-10 13:14:42 +08:00
3de2b1eafb [Doc] Show default pooling method in a table (#11904) Cyrus Leung 2025-01-10 11:25:20 +08:00
b844b99ad3 [VLM] Enable tokenized inputs for merged multi-modal processor (#11900) Cyrus Leung 2025-01-10 11:24:00 +08:00
c3cf54dda4 [Doc][5/N] Move Community and API Reference to the bottom (#11896) Cyrus LeungandSimon Mo 2025-01-10 11:10:12 +08:00
36f5303578 [Docs] Add Modal to deployment frameworks (#11907) Charles Frye 2025-01-09 15:26:37 -08:00
9a228348d2 [Misc] Provide correct Pixtral-HF chat template (#11891) Cyrus Leung 2025-01-10 01:19:37 +08:00
bd82872211 [ci]try to fix flaky multi-step tests (#11894) youkaichao 2025-01-09 22:47:29 +08:00
405eb8e396 [platform] Allow platform specify attention backend (#11609) wangxiyuanandMengqing Cao 2025-01-09 21:46:50 +08:00
65097ca0af [Doc] Add model development API Reference (#11884) Cyrus Leung 2025-01-09 17:43:40 +08:00
1d967acb45 [Bugfix] fix beam search input errors and latency benchmark script (#11875) Ye (Charlotte) Qiandyeq 2025-01-09 01:36:39 -08:00
0bd1ff4346 [Bugfix] Override dunder methods of placeholder modules (#11882) Cyrus Leung 2025-01-09 17:02:53 +08:00
310aca88c9 [perf]fix current stream (#11870) youkaichao 2025-01-09 15:18:21 +08:00
a732900efc [Doc] Intended links Python multiprocessing library (#11878) Guspan Tanadi 2025-01-09 12:39:39 +07:00
d848800e88 [Misc] Move print_*_once from utils to logger (#11298) Cyrus LeungandMaxime Fournioux 2025-01-09 12:48:12 +08:00
730e9592e9 [Doc] Recommend uv and python 3.12 for quickstart guide (#11849) Michael Goin 2025-01-08 22:37:48 -05:00
1fe554bac3 treat do_lower_case in the same way as the sentence-transformers library (#11815) Maximilien de Bayser 2025-01-09 00:05:43 -03:00
615e4a5401 [CI] Turn on basic correctness tests for V1 (#10864) Tyler Michael Smith 2025-01-08 21:20:44 -05:00
3db0cafdf1 [Docs] Add Google Cloud Meetup (#11864) Simon Mo 2025-01-08 12:38:28 -08:00
526de822d5 [Kernel][Triton][AMD] Use block size heuristic for avg 2.8x speedup for int8 models (#11698) rasmith 2025-01-08 14:23:15 -06:00
56fe4c297c [TPU][Quantization] TPU W8A8 (#11785) Robert ShawandWoosuk Kwon 2025-01-08 14:33:29 -05:00
47de8821d3 [Misc]add some explanations for BlockHashType (#11847) WangErXiao 2025-01-09 02:21:30 +08:00
5984499e47 [Doc] Expand Multimodal API Reference (#11852) Cyrus Leung 2025-01-09 01:14:14 +08:00
ca47e176af [Misc] Move some model utils into vision file (#11848) Cyrus Leung 2025-01-09 01:04:46 +08:00
78f4590b60 [Bugfix][XPU] fix silu_and_mul (#11823) Yan Ma 2025-01-09 00:11:50 +08:00
2f7024987e [CI/Build][Bugfix] Fix CPU CI image clean up (#11836) Li, Jiang 2025-01-08 23:18:28 +08:00
6cd40a5bfe [Doc][4/N] Reorganize API Reference (#11843) Cyrus Leung 2025-01-08 21:34:44 +08:00
aba8d6ee00 [Doc] Move examples into categories (#11840) Harry Mellor 2025-01-08 13:09:53 +00:00
2a0596bc48 [VLM] Reorganize profiling/processing-related code (#11812) Cyrus Leung 2025-01-08 18:59:58 +08:00
f12141170a [torch.compile] consider relevant code in compilation cache (#11614) youkaichao 2025-01-08 18:46:43 +08:00
cfd3219f58 [Hardware][Apple] Native support for macOS Apple Silicon (#11696) Wallas HenriqueandMichael Goin 2025-01-08 05:35:49 -03:00
a1b2b8606e [Docs] Update sponsor name: 'Novita' to 'Novita AI' (#11833) Simon Mo 2025-01-07 23:05:46 -08:00

... 191 192 193 194 195 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86