Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

baed180aa0 [tech debt] Revisit lora request model checker (#20636) kourosh hakhamaneshi 2025-07-08 18:42:41 -07:00
0b407479ef [misc]refactor Platform.set_device method (#20262) Kunshang Ji 2025-07-09 09:39:47 +08:00
5eaf570050 Replace multiply_add with homogeneous_multiply_add to Address Clang Template Parameter Issue (#20142) Wenxin Cheng 2025-07-08 17:30:18 -07:00
d8ee5a2ca4 [TPU][Bugfix] disable phi-3 test (#20632) QiliangCui 2025-07-08 16:14:26 -07:00
b9fca83256 [Bugfix] Fix GLM-4.1-V video prompt update (#20635) Isotr0py 2025-07-09 07:13:58 +08:00
32dffc2772 [Core] Rename get_max_tokens_per_item for backward compatibility (#20630) Cyrus Leung 2025-07-09 07:11:30 +08:00
c438183e99 [Bugfix] Fix topk_ids indices_type for CUTLASS w8a8 FP8 MoE (#20166) Ming Yang 2025-07-08 16:10:57 -07:00
baba0389f7 [CI] Increase the threshold of the MTEB RERANK tests (#20615) wang.yuqi 2025-07-08 23:10:11 +08:00
c6c22f16d3 Revert invalid spellchecker fix on deepseek_vl2 (#20618) viravera 2025-07-08 08:07:14 -07:00
dd382e0fe3 [Model] Implement missing get_language_model for Keye-VL (#20631) Cyrus Leung 2025-07-08 22:47:46 +08:00
849590a2a7 Update torch/xla pin to 20250703 (#20589) XiongfeiWei 2025-07-08 07:44:02 -07:00
a4c23314c0 [xpu]feat: support multi-lora on xpu (#20616) Yan Ma 2025-07-08 22:07:10 +08:00
b942c094e3 Stop using title frontmatter and fix doc that can only be reached by search (#20623) Harry Mellor 2025-07-08 11:27:40 +01:00
b4bab81660 Remove unnecessary explicit title anchors and use relative links instead (#20620) Harry Mellor 2025-07-08 10:49:13 +01:00
b91cb3fa5c [Docs] Improve documentation for Deepseek R1 on Ray Serve LLM (#20601) Ricardo Decal 2025-07-08 02:09:06 -07:00
71d1d75b7a [PD][Nixl] Remote consumer READ timeout for clearing request blocks (#20139) Nicolò Lucchesi 2025-07-08 09:56:40 +02:00
72d14d0eed [Frontend] [Core] Integrate Tensorizer in to S3 loading machinery, allow passing arbitrary arguments during save/load (#19619) Sanger SteelandEta 2025-07-08 01:47:43 -04:00
e34d130c16 [TPU] Temporary fix vmem oom for long model len by reducing page size (#20278) Chenyaaang 2025-07-07 22:16:16 -07:00
7721ef1786 [CI/Build][CPU] Fix CPU CI and remove all CPU V0 files (#20560) Li, Jiang 2025-07-08 13:13:44 +08:00
8369b7c2a9 [Misc] improve error msg (#20604) Reid 2025-07-08 12:45:18 +08:00
3eb4ad53f3 [Docs] Add Anyscale to frameworks (#20590) Ricardo Decal 2025-07-07 20:09:13 -07:00
90a2769f20 [Docs] Add Ray Serve LLM section to openai compatible server guide (#20595) Ricardo Decal 2025-07-07 20:08:05 -07:00
e60d422f19 [Docs] Improve docstring for ray data llm example (#20597) Ricardo Decal 2025-07-07 20:06:26 -07:00
0d914c81a2 [Docs] Rewrite offline inference guide (#20594) Ricardo Decal 2025-07-07 20:06:02 -07:00
6e428cdd7a [Doc] Syntax highlight request responses as JSON instead of bash (#20582) Harry Mellor 2025-07-08 04:02:45 +01:00
93b9d9f499 [Bugfix]: Fix messy code when using logprobs (#19209) Chauncey 2025-07-08 11:02:15 +08:00
af107d5a0e Make distinct code and console admonitions so readers are less likely to miss them (#20585) Harry Mellor 2025-07-08 03:55:28 +01:00
31c5d0a1b7 [Optimize] Don't send token ids when kv connector is not used (#20586) Woosuk Kwon 2025-07-07 19:04:54 -07:00
afb7cff1b9 [Bugfix] Fix Maverick correctness by filling zero to cache space in cutlass_moe (#20167) Ming Yang 2025-07-07 18:07:22 -07:00
d2e841a10a [Misc] Improve logging for dynamic shape cache compilation (#20573) Kyle Yu 2025-07-07 20:48:09 -04:00
14601f5fba [Config] Refactor mistral configs (#20570) Patrick von Platen 2025-07-08 00:25:10 +02:00
042d131f39 Fix links in multi-modal model contributing page (#18615) Harry Mellor 2025-07-07 22:13:52 +01:00
8e807cdfa4 [Misc] feat output content in stream response (#19608) rongfu.leng 2025-07-08 04:45:10 +08:00
e601efcb10 [Misc] Add fully interleaved support for multimodal 'string' content format (#14047) Antonanddrobyshev.anton 2025-07-07 22:43:08 +03:00
22dd9c2730 [Kernel] Optimize Prefill Attention in Unified Triton Attention Kernel (#20308) jvlunteren 2025-07-07 21:08:12 +02:00
a6d795d593 [DP] Copy environment variables to Ray DPEngineCoreActors (#20344) Rui Qiao 2025-07-07 10:14:22 -07:00
a37d75bbec [Front-end] microbatch tokenization (#19334) ztang2370 2025-07-08 00:54:10 +08:00
edd270bc78 [Bugfix] Prevent IndexError for cached requests when pipeline parallelism is disabled (#20486) Peter Pan 2025-07-08 00:41:15 +08:00
110df74332 [Model][Last/4] Automatic conversion of CrossEncoding model (#19675) wang.yuqi 2025-07-07 22:46:04 +08:00
1ad69e8375 [Doc] Fix some MkDocs snippets used in the installation docs (#20572) Harry Mellor 2025-07-07 15:44:34 +01:00
b8a498c9b2 [Doc] Add outline for content tabs (#20571) Harry Mellor 2025-07-07 15:43:26 +01:00
923147b5e8 [Doc] Fix internal links so they don't always point to latest (#20563) Harry Mellor 2025-07-07 12:15:50 +01:00
45877ef740 [Doc] Use gh-pr and gh-issue everywhere we can in the docs (#20564) Harry Mellor 2025-07-07 11:54:22 +01:00
6e4bef1bea [Doc] Remove extra whitespace from CI failures doc (#20565) Harry Mellor 2025-07-07 11:35:47 +01:00
4ff79a136e [Misc] Set the minimum openai version (#20539) Jee Jee Li 2025-07-07 17:15:26 +08:00
448acad31e [Misc] remove unused jinaai_serving_reranking (#18878) Abirdcfly 2025-07-07 17:14:12 +08:00
eb0b2d2f08 [Docs] Clean up tables in supported_models.md (#20552) Michael Yao 2025-07-07 16:46:31 +08:00
3112271f6e [XPU] log clean up for XPU platform (#20553) Yan Ma 2025-07-07 16:38:22 +08:00
1fd471e957 Add docstrings to url_schemes.py to improve readability (#20545) Michael Yao 2025-07-07 16:31:49 +08:00
2c5ebec064 [XPU][CI] add v1/core test in xpu hardware ci (#20537) Liangliang Ma 2025-07-07 16:16:40 +08:00
2e610deb72 [CI/Build] Enable phi2 lora test (#20540) Jee Jee Li 2025-07-07 13:10:41 +08:00
6e2c19ce22 [Refactor]Abstract Platform Interface for Distributed Backend and Add xccl Support for Intel XPU (#19410) Yang YangandKunshang Ji 2025-07-07 12:32:32 +08:00
47db8c2c15 [Misc] add a tip for pre-commit (#20536) Reid 2025-07-07 10:42:06 +08:00
462b269280 Implement OpenAI Responses API [1/N] (#20504) Woosuk Kwon 2025-07-06 18:32:13 -07:00
c18b3b8e8b [Bugfix] Add use_cross_encoder flag to use correct activation in ClassifierPooler (#20527) Cyrus Leung 2025-07-07 05:01:48 +08:00
9528e3a05e [BugFix][Spec Decode] Fix spec token ids in model runner (#20530) Woosuk Kwon 2025-07-06 12:44:52 -07:00
9fb52e523a [V1] Support any head size for FlexAttention backend (#20467) Cyrus Leung 2025-07-07 00:54:36 +08:00
e202dd2736 [V0 deprecation] Remove V0 CPU/XPU/TPU backends (#20412) Woosuk KwonandLi, Jiang <jiang1.li@intel.com> 2025-07-06 08:48:13 -07:00
43813e6361 [Misc] call the pre-defined func (#20518) Reid 2025-07-06 18:25:29 +08:00
cede942b87 [Benchmark] Add support for multiple batch size benchmark through CLI in benchmark_moe.py (#20516) Brayden Zhong 2025-07-06 05:20:11 -04:00
fe1e924811 [Frontend] Support image object in llm.chat (#19635) Flora Feng 2025-07-05 23:47:13 -07:00
4548c03c50 [TPU][Bugfix] fix the MoE OOM issue (#20339) Chengji Yao 2025-07-05 21:19:09 -07:00
40b86aa05e [BugFix] Fix: ImportError when building on hopper systems (#20513) Lucas Wilkinson 2025-07-06 00:17:30 -04:00
432870829d [Bugfix] Fix missing per_act_token parameter in compressed_tensors_moe (#20509) Lucia Fang 2025-07-06 12:08:30 +08:00
f73d02aadc [BUG] Fix #20484. Support empty sequence in cuda penalty kernel (#20491) Vadim Gimpelson 2025-07-06 06:38:02 +04:00
c5ebe040ac test_attention compat with coming xformers change (#20487) Jeremy Reizensteinandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-07-06 03:37:59 +01:00
8d763cb891 [Misc] remove unused import (#20517) Reid 2025-07-06 10:17:06 +08:00
cf4cd53982 [Misc] Add logger.exception for TPU information collection failures (#20510) Reid 2025-07-05 22:24:32 +08:00
32c9be2200 [v1] Re-add fp32 support to v1 engine through FlexAttention (#19754) Isotr0py 2025-07-05 17:41:10 +08:00
8aeaa910a2 Fix unknown attribute of topk_indices_dtype in CompressedTensorsW8A8Fp8MoECutlassMethod (#20507) Lucia FangandLucia Fang 2025-07-05 14:03:20 +08:00
906e05d840 [Misc] Remove the unused LoRA test code (#20494) Jee Jee Li 2025-07-05 13:48:16 +08:00
ef9a2990ae [doc] small fix (#20506) Reid 2025-07-05 11:56:39 +08:00
7e90870491 [Misc] Add security warning for development mode endpoints (#20508) Reid 2025-07-05 11:52:13 +08:00
d3f05c9248 [Doc] fix mutltimodal_inputs.md gh examples link (#20497) Guy Stone 2025-07-05 00:41:35 +01:00
c108781c85 [CI Bugfix] Fix pre-commit failures on main (#20502) Michael Goin 2025-07-05 06:17:30 +09:00
3d184b95b8 [feat]: CUTLASS block scaled group gemm for SM100 (#19757) Duncan MossandDuncan Moss 2025-07-04 11:58:04 -07:00
2f35a022e6 Enable V1 for Hybrid SSM/Attention Models (#20016)
Thomas Parnell Stanislaw Wozniak Tyler Michael Smith Chen Zhang
2025-07-04 19:46:53 +02:00
ffe00ef77a [Misc] Small: Remove global media connector. Each test should have its own test connector object. (#20395) Chenheli Hua 2025-07-04 08:15:03 -07:00
5561681d04 [CI] add kvcache-connector dependency definition and add into CI build (#18193) Peter Pan 2025-07-04 21:49:18 +08:00
fbd62d8750 [Doc] Fix classification table in list of supported models (#20489) Cyrus Leung 2025-07-04 21:08:02 +08:00
2e26f9156a [Model][3/N] Automatic conversion of CrossEncoding model (#20168) wang.yuqi 2025-07-04 20:47:39 +08:00
9e5452ee34 [Bug][Frontend] Fix structure of transcription's decoder_prompt (#18809) sangbumlikeagod 2025-07-04 20:28:07 +09:00
0e3fe896e2 Support Llama 4 for fused_marlin_moe (#20457) Michael Goin 2025-07-04 16:55:10 +09:00
1caca5a589 [Misc] Add SPDX-FileCopyrightText (#20428) Jee Jee Li 2025-07-04 15:40:42 +08:00
783921d889 [Perf] Optimize Vectorization Utils for Int 8 Quantization Kernels (#20331) Wentao Ye 2025-07-04 03:06:24 -04:00
4a98edff1f [Structured Outputs][V1] Skipping with models doesn't contain tokenizers (#20365) Aaron PhamandNick Hill 2025-07-04 03:05:49 -04:00
a7bab0c9e5 [Misc] small update (#20462) Reid 2025-07-04 11:33:44 +08:00
25950dca9b Add ignore consolidated file in mistral example code (#20420) 汪志鹏 2025-07-04 10:55:07 +08:00
a4113b035c [Platform] Add custom default max tokens (#18557) Gabriel Marinho 2025-07-03 23:50:17 -03:00
7e1665b089 [Misc] Change warn_for_unimplemented_methods to debug (#20455) Michael Goin 2025-07-04 11:35:08 +09:00
8d1096e7db [Bugfix] Register reducer even if transformers_modules not available (#19510) Seiji Eicher 2025-07-03 15:08:12 -07:00
8d775dd30a [Misc] Fix Unable to detect current VLLM config. Defaulting to NHD kv cache layout warning (#20400) Nicolò Lucchesi 2025-07-03 23:56:09 +02:00
78fe77534b [Kernel] Enable fp8 support for pplx and BatchedTritonExperts. (#18864) bnellnm 2025-07-03 17:55:40 -04:00
2f2fcb31b8 [Misc] Remove _maybe_ignore_quant_config from GLM4.1v (#20432) Yuxuan Zhang 2025-07-04 05:41:13 +08:00
1dba2c4ebe [Misc] adjust for ipv6 for mookcacke url parse (#20107) Ning Xie 2025-07-04 04:27:17 +08:00
71d6de3a26 [Misc] Clean up InternVL family config registration (#19992) Isotr0py 2025-07-04 04:01:47 +08:00
536fd33003 [CI] Trimming some failing test groups from AMDPRODUCTION. (#20390) Alexei-V-Ivanov-AMD 2025-07-03 10:21:31 -05:00
619b9f5c7e [Frontend] fix duplicate output for bench subcmd (#20446) Reid 2025-07-03 23:02:06 +08:00
d1b689c445 [Bugfix] Fix flaky test_streaming_response test (#20363) Nicolò Lucchesi 2025-07-03 16:46:24 +02:00
9854dc9040 [Frontend] improve vllm bench <bench_type> --help display (#20430) Reid 2025-07-03 22:22:16 +08:00

... 157 158 159 160 161 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86