Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

387bdf0ab9 [Model] Add support for MiniMaxM1ForCausalLM (shares architecture with MiniMaxText01ForCausalLM) (#19677) qscqesze 2025-06-17 00:47:14 +08:00
5e5baa91aa [Kernels] Use empty for modular MoE workspaces (#19667) bnellnm 2025-06-16 10:58:01 -04:00
836d4ce140 [Bugfix] fix missing 'finish_reason': null in streaming chat (#19662) Chauncey 2025-06-16 22:10:39 +08:00
c3fec47bb7 [MISC] bump huggingface_hub pkg to 0.33.0 (#19547) Ning Xie 2025-06-16 20:22:28 +08:00
1173804dca [Bugfix] Fix TP inference for Flex attention backend (#19657) Isotr0py 2025-06-16 19:21:37 +08:00
4d5424029b [Feature]:Allow for Granite MoE Hybrid models with _only_ shared experts. (#19652) Shawn Tan 2025-06-16 07:14:18 -04:00
3e7506975c [DOC] Add reasoning capability to vLLM streamlit code (#19557) Navanit Dubey 2025-06-16 16:39:12 +05:30
ee35e96ac3 [BugFix] Don't catch BaseException when dumping execute_model errors (#19626) Nick Hill 2025-06-16 04:01:08 -07:00
dec66d253b [Kernel] GGUF MMVQ kernel for multiple input vectors (#18754) Szymon Ożóg 2025-06-16 11:33:26 +02:00
8d120701fd [Docs] Move multiproc doc to v1 dir (#19651) Russell Bryant 2025-06-16 05:10:12 -04:00
f40f763f12 [CI] Add mteb testing for rerank models (#19344) wang.yuqi 2025-06-16 16:36:43 +08:00
26bc46ef89 [MISC] typo fix (#19672) Ning Xie 2025-06-16 15:18:49 +08:00
a77aea59fd [TPU] support attention head dim smaller than 128 (#19620) Chengji Yaoandmgoin 2025-06-15 23:40:53 -07:00
b692e9cd07 [Misc] Fix skipped max-model-len validation when deriving max model length from tokenizer config (#19660) Ye (Charlotte) Qi 2025-06-15 23:30:29 -07:00
367871a469 [Misc][Frontend] passthrough bad_words (#19564)
Francesco Bertolotti Francesco Bertolotti Aaron Pham
2025-06-16 07:05:13 +02:00
92183b41f3 [Bugfix][Core] Prefix caching causes incorrect outputs due to outdated ComputedBlocksTracker (#18957) quanliuand刘全 2025-06-16 12:56:37 +08:00
c6703d1e0d [MISC] Remove unused variableds in C++ (#19609) Lu Fang 2025-06-16 11:05:28 +08:00
a5e7242d5f [Misc] Remove duplicate multiproc method setting for CPU platform (#19649) Isotr0py 2025-06-16 10:26:58 +08:00
91b2c17a55 [CI/Build] Fix torch nightly CI dependencies part 2 (#19589) Richard Zou 2025-06-15 08:01:10 -04:00
055915e6ce Enable prefix caching with full cuda graphs (#19617) Woosuk Kwon 2025-06-15 01:05:05 -07:00
3d330c4c09 [Benchmark] Refactor benchmark script for fp8 & int8 (#19627) Wentao Ye 2025-06-15 03:15:37 -04:00
0b73736a0d [Kernel] Raise verbose error and consolidate num_heads/num_kv_heads divisibility check (#19339) 22quinn 2025-06-14 22:43:48 -07:00
ee1531bc38 [Bugfix][2/n] Fix speculative decoding CI - Fix test_ngram_e2e_greedy_correctness (#19644) Lu Fang 2025-06-15 12:15:41 +08:00
e13945f9dd [Perf] Further tunings for SM100 FP8 CUTLASS kernel (#19566) Ilya Markov 2025-06-15 02:25:10 +02:00
08500011d3 [Fix] Convert kv_transfer_config from dict to KVTransferConfig (#19262) maobaolong 2025-06-15 03:32:07 +08:00
861a0a0a39 [Bugfix] Don't attempt to use triton if no driver is active (#19561) Konrad Zawora 2025-06-14 21:30:54 +02:00
bc956b38d0 Only build CUTLASS MoE kernels on Hopper (#19648) Huy Do 2025-06-14 11:44:15 -07:00
294fc1e2c9 [Hardware][NVIDIA][kernel] Fp4 MOE quant kernel optimization (#19500) jiahanc 2025-06-14 09:34:28 -07:00
2db9044ab6 [Bugfix] Fix auto dtype casting for BatchFeature (#19316) Isotr0py 2025-06-14 23:13:08 +08:00
6fa718a460 [Misc] Modularize CLI Argument Parsing in Benchmark Scripts (#19593) Reidandreidliu41 2025-06-14 16:54:52 +08:00
06be858828 [Bugfix] Fix the speculative decoding test by setting the target dtype (#19633) Lu Fang 2025-06-14 11:57:32 +08:00
d1e34cc9ac [V1][Metrics] Deprecate metrics with gpu_ prefix for non GPU specific metrics. (#18354) Saheli Bhattacharjee 2025-06-14 04:07:36 +01:00
bd517eb9fe [BugFix] Fix DP Coordinator incorrect debug log message (#19624) Nick Hill 2025-06-13 17:18:03 -07:00
d65668b4e8 Adding "AMD: Multi-step Tests" to amdproduction. (#19508)
Concurrenseegemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
2025-06-13 19:08:51 -05:00
aafbbd981f [torch.compile] Use custom ops when use_inductor=False (#19618) Woosuk Kwon 2025-06-13 15:05:54 -07:00
0f0874515a [Doc] Add troubleshooting section to k8s deployment (#19377) Anna Pendleton 2025-06-13 14:47:51 -07:00
3597b06a4f [CUDA] Enable full cudagraph for FlashMLA (#18581) Luka Govedič 2025-06-13 14:12:26 -04:00
1015296b79 [doc][mkdocs] fix the duplicate Supported features sections in GPU docs (#19606) Reidandreidliu41 2025-06-14 00:25:08 +08:00
ce9dc02c93 [Refactor] Remove unused variables in moe_permute_unpermute_kernel.inl (#19573) Wentao Ye 2025-06-13 09:12:15 -04:00
a24cb91600 [Model] Fix minimax model cache & lm_head precision (#19592) qscqesze 2025-06-13 20:08:20 +08:00
7e8d97dd3f [BugFix] Honor enable_caching in connector-delayed kvcache load case (#19435) Nick Hill 2025-06-13 02:46:32 -07:00
d70bc7c029 [torch.compile] reorganize the cache directory to support compiling multiple models (#19064) youkaichao 2025-06-13 15:23:25 +08:00
ce688ad46e use base version for version comparison (#19587) Boyuan Feng 2025-06-13 00:09:34 -07:00
cefdb9962d [Fix] The zip function in Python 3.9 does not have the strict argument (#19549) 汪志鹏 2025-06-13 14:57:48 +08:00
ace5cdaff0 [Fix] bump mistral common to support magistral (#19533) 汪志鹏 2025-06-13 13:28:12 +08:00
6458721108 [CPU] Refine default config for the CPU backend (#19539) Li, Jiang 2025-06-13 13:27:39 +08:00
bb4a0decef [Misc] Correct broken docs link (#19553) Hyogeun Oh (오효근) 2025-06-13 14:27:13 +09:00
c707cfc12e [doc] fix incorrect link (#19586) Reidandreidliu41 2025-06-13 12:26:09 +08:00
7b3c9ff91d [Doc] uses absolute links for structured outputs (#19582) Aaron Pham 2025-06-12 23:35:17 -04:00
c68698b326 [Bugfix] Fix EAGLE vocab embedding for multimodal target model (#19570) qizixi 2025-06-12 20:09:19 -07:00
e3b12667d4 [BugFix] : Fix Batched DeepGemm Experts (#19515) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-06-12 22:43:02 -04:00
e6aab5de29 Revert "[Build/CI] Add tracing deps to vllm container image (#15224)" (#19378) kourosh hakhamaneshi 2025-06-12 17:26:40 -07:00
c57bb199b3 [V1] Resolve failed concurrent structured output requests (#19565) Russell Bryant 2025-06-12 19:30:09 -04:00
dba68f9159 [Doc] Unify structured outputs examples (#18196) Aaron Pham 2025-06-12 18:50:31 -04:00
a3319f4f04 [Bugfix] Enforce contiguous input for dynamic_per_token FP8/INT8 quant (#19452) Michael Goin 2025-06-12 15:39:15 -04:00
9d880f594d [Misc] Turn MOE_DP_CHUNK_SIZE into an env var (#19506) Varun Sundar Rabindranath 2025-06-12 14:01:16 -04:00
017ef648e9 [Spec Decode][Benchmark] Generalize spec decode offline benchmark to more methods and datasets (#18847) Ekagra Ranjan 2025-06-12 13:30:56 -04:00
4b25ab14e2 [doc] Make top navigation sticky (#19540) Reidandreidliu41 2025-06-12 23:48:11 +08:00
f98548b9da [torch.compile][ROCm] Fuse quantization onto attention using a torch.compile pass (#16756) Luka GovedičandSage Moore 2025-06-12 11:31:04 -04:00
96846bb360 Fix TorchAOConfig skip layers (#19265) mobicham 2025-06-12 16:22:53 +02:00
b6efafd9e4 [Perf] Vectorize static / dynamic INT8 quant kernels (#19233) Wentao Ye 2025-06-12 09:51:41 -04:00
1129e2b1ab [V1][NixlConnector] Drop num_blocks check (#19532) Nicolò Lucchesi 2025-06-12 14:36:14 +02:00
c742438f8b [Doc] Add V1 column to supported models list (#19523) Cyrus Leung 2025-06-12 19:16:44 +08:00
73e2e0118f [Quantization] Improve AWQ logic (#19431) Jee Jee Li 2025-06-12 19:02:11 +08:00
c9280e6346 [Bugfix] Respect num-gpu-blocks-override in v1 (#19503) jmswen 2025-06-12 04:00:23 -07:00
af09b3f0a0 [Bugfix][V1] Allow manual FlashAttention for Blackwell (#19492) Michael Goin 2025-06-12 06:40:24 -04:00
4f6c42fa0a [Security] Prevent new imports of (cloud)pickle (#18018) Russell BryantandAaron Pham 2025-06-12 06:30:17 -04:00
dff680001d Fix typo (#19525) niu_he 2025-06-12 17:24:45 +08:00
2e090bd5df [AMD][Kernel][BugFix] fix test_rocm_compressed_tensors_w8a8 for rocm (#19509) rasmith 2025-06-12 02:14:24 -05:00
1b0b065eb5 [BugFix] Handle missing sep_token for Qwen3-Reranker in Score API (#19522) wonjun Jang 2025-06-12 16:00:47 +09:00
d5bdf899e4 [BugFix] Work-around incremental detokenization edge case error (#19449) Nick Hill 2025-06-11 23:43:20 -07:00
7e3e74c97c [Frontend] Improve error message in tool_choice validation (#19239) 22quinn 2025-06-11 23:13:00 -06:00
3f6341bf7f Add Triton Fused MoE kernel config for E=16 on B200 (#19518) Brayden Zhong 2025-06-12 00:31:51 -04:00
e5d35d62f5 [BugFix] Force registration of w8a8_block_fp8_matmul_deepgemm via lazy import (#19514) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-06-12 00:28:12 -04:00
2f1c19b245 [CI] change spell checker from codespell to typos (#18711) Ning Xie 2025-06-12 10:57:10 +08:00
42f52cc95b [CI/Build] Fix torch nightly CI dependencies (#19505) Richard Zou 2025-06-11 17:40:42 -04:00
97a9465bbc [UX] Add Feedback During CUDAGraph Capture (#19501) Robert Shaw 2025-06-11 14:09:05 -07:00
c7ea0b56cd [AMD] [Quantization] Add override flag for attention dtype instead of using kv_cache_dtype trigger (#17331) rasmith 2025-06-11 14:53:28 -05:00
29fa5cac1c [Kernels] Add activation chunking logic to FusedMoEModularKernel (#19168) bnellnm 2025-06-11 12:53:10 -04:00
b2d9be6f7d [Docs] Remove WIP features in V1 guide (#19498) Woosuk Kwon 2025-06-11 09:15:03 -07:00
04a55612dd [Misc] Fix misleading ROCm warning (#19486) Jee Jee Li 2025-06-12 00:12:10 +08:00
89b0f84e17 [doc] fix "Other AI accelerators" getting started page (#19457) David Xia 2025-06-11 12:11:17 -04:00
497a91e9f7 [CI] Update FlashInfer to 0.2.6.post1 (#19297) Michael Goin 2025-06-11 10:57:28 -04:00
943ffa5703 [Bugfix] Update the example code, make it work with the latest lmcache (#19453) runzhen 2025-06-11 05:42:20 -07:00
5c8d34a42c Support no privileged mode on CPU for docker and kubernetes deployments (#19241) Louie Tsai 2025-06-11 04:11:47 -07:00
3c8694eabe Fix some typo (#19475) Ximingwang-09andximing.wxm 2025-06-11 18:36:04 +08:00
7484e1fce2 Add cache to cuda get_device_capability (#19436) Michael Goin 2025-06-11 05:37:05 -04:00
a2142f0196 Support non-string values in JSON keys from CLI (#19471) Cyrus Leung 2025-06-11 17:34:04 +08:00
871d6b7c74 [Misc] Reduce warning message introduced in env_override (#19476) Lu Fang 2025-06-11 17:29:54 +08:00
29a38f0352 [Doc] Support "important" and "announcement" admonitions (#19479) Cyrus Leung 2025-06-11 16:39:58 +08:00
a5115f4ff5 [Doc] Fix quantization link titles (#19478) Cyrus Leung 2025-06-11 16:27:22 +08:00
68b4a26149 [Doc] Update V1 User Guide for Hardware and Models (#19474) Cyrus Leung 2025-06-11 15:49:06 +08:00
b8e809a057 [Kernel] Support deep_gemm for linear methods (#19085) artetaout 2025-06-11 15:14:45 +08:00
5039ec2336 [ROCm] Add rules to automatically label ROCm related PRs (#19405) Lu Fang 2025-06-11 15:09:18 +08:00
7c644ab6d5 Fix Typo in Documentation and Function Name (#19442) leopardracer 2025-06-11 08:44:11 +03:00
2d40665fe8 Add fused MOE config for Qwen3 30B A3B on B200 (#19455) Junhao Li 2025-06-11 01:43:46 -04:00
96ada386b7 [Misc] Remove unused MultiModalHasher.hash_prompt_mm_data (#19422) Lukas Geiger 2025-06-11 07:18:57 +02:00
1e473b3010 [CI] Disable failing GGUF model test (#19454) Michael Goin 2025-06-11 01:12:38 -04:00
2b1e2111b0 Fix test_max_model_len in tests/entrypoints/llm/test_generate.py (#19451) Lu Fang 2025-06-11 12:54:59 +08:00
a45b979d9f [BugFix] Fix docker build cpu-dev image error (#19394) niu_he 2025-06-11 11:56:40 +08:00

... 161 162 163 164 165 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86