iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
7,012 Commits 3 Branches 0 Tags
cda10fa3e2bb69ea276d663e5369ba16ec42cebb
Commit Graph
4744 Commits
Author SHA1 Message Date
jennyyyyzhenandyZhen cda10fa3e2 [Multi Modal] Add an env var for message queue max chunk bytes (#19242) 
Signed-off-by: yZhen <yZhen@fb.com>
Co-authored-by: yZhen <yZhen@fb.com>
 2025-06-08 21:39:12 +08:00
Dipika Sikka c123bc33f9 [Quantization] Add compressed-tensors NVFP4 support (#18312) 2025-06-08 09:05:55 -04:00
Xu Wenqing 989dcee981 Add H20-3e fused MoE kernel tuning configs for Qwen3-235B-A22B (#19315) 
Signed-off-by: Xu Wenqing <xuwq1993@qq.com>
 2025-06-08 16:07:02 +08:00
Richard Zou eaa2e51088 [Bugfix] Re-enable use_cudagraph in vLLM v1 (#19299) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2025-06-08 08:56:12 +08:00
Chauncey d77f7fb871 [Bugfix]: Fix TypeError: 'float' object cannot be interpreted as an integer (#19283) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-06-08 08:16:31 +08:00
Luka Govedič 2d8476e465 [BugFix][V1] Fix memory profiling bug (#18974) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-06-07 10:34:51 -07:00
Driss Guessous cf02f9b283 Add FlexAttention to V1 (#16078) 
Signed-off-by: drisspg <drisspguessous@gmail.com>
 2025-06-06 21:58:55 -07:00
ElizaWszolaandTyler Michael Smith 84166fee97 [Kernel] Integrate CUTLASS MoE kernel with PPLX (#18762) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-06-06 18:26:11 -07:00
Nick Hill 46ecc57973 [BugFix] Fix tpu_model_runner block_id concatenation (#19228) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-06-06 16:28:17 -07:00
Nicolò LucchesiandNick Hill b6a3a9f76d [Core] Fix abrupt request abort (#18485) 
Signed-off-by: nicklucche <nlucches@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>

Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-06-06 16:27:59 -07:00
Adolfo VictoriaandAdolfo Victoria ca27f0f9c1 [Bugfix][Core] Update cancellation logic in generate() to handle Generator exits (#19225) 
Co-authored-by: Adolfo Victoria <adovi@meta.com>
 2025-06-06 20:17:54 +00:00
Nick Hill aad30bd306 [BugFix] Fix MultiConnector test after HMA changes (#19291) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-06-06 20:16:24 +00:00
Yu Guo 8267f9916f improve logits bias (#19041) 2025-06-06 19:59:25 +08:00
jmswen 7353492a47 [Core] Raise when non-multi-instance DP clients target a DP rank (#19227) 
Signed-off-by: Jon Swenson <jmswen@gmail.com>
 2025-06-06 19:03:01 +08:00
Jee Jee Li 7661e92ef8 [Model] Optimize nemotron_h implementation (#19249) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-06-06 10:05:14 +00:00
Richard Zou da511d54d8 Fix CompilationConfig repr (#19091) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-06-06 16:23:35 +08:00
Nick Hill 65c69444b1 [Docs] Improve V1 KVConnector interface documentation (#19172) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-06-06 16:22:45 +08:00
Dipika Sikka 94870359cd [Quantization] Bump compressed-tensors version; update NVFP4A16 test model (#19224) 
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
 2025-06-06 01:21:54 -07:00
Chengji Yao 0d49483ea9 [TPU] fix kv cache dtype in model runner (#19244) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-06-06 16:20:16 +08:00
Jinghui Zhangandjinghui 90b78ec5f9 [v1][P/D] Fix a edge case in kv cache schedule (#19182) 
Co-authored-by: jinghui <jinghui@fb.com>
 2025-06-05 23:32:55 -07:00
Xu Song 3da2313d78 Support allowed_token_ids in ChatCompletionRequest (#19143) 
Signed-off-by: Xu Song <xusong.vip@gmail.com>
 2025-06-06 05:06:48 +00:00
Chengji Yao b61dc5f972 [TPU] update torch_xla pin (#19231) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-06-06 04:27:38 +00:00
Chen Zhang f8a1a2d108 [v1] Hybrid Memory Allocator (#17996) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-06-05 20:47:09 -07:00
Benjamin Chislett 3465b87ef8 [Bugfix] Fix EAGLE vocab embedding construction for Llama 70B (#19033) 
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
 2025-06-05 19:10:08 -07:00
Jerry Zhang c8134bea15 Fix AOPerModuleConfig name changes (#18869) 
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
 2025-06-05 18:51:32 -07:00
Luis VegaandLuis Vega cb6d572e85 [Model] NemotronH support (#18863) 
Signed-off-by: Luis Vega <2478335+vegaluisjose@users.noreply.github.com>
Co-authored-by: Luis Vega <2478335+vegaluisjose@users.noreply.github.com>
 2025-06-05 21:29:28 +00:00
Michael Goin 87360308b7 [V1] Use FlashInfer by default on Blackwell GPUs (#19118) 2025-06-05 15:40:39 -04:00
Nicolò Lucchesi 9ef9173cfa [P/D][NixlConnector] Enable FlashInfer backend (#19090) 2025-06-05 17:10:15 +00:00
Povilas Kanapickas 85e2b7bb13 [MISC][Bugfix] Use less CPU when message queue has been empty for some time (#16226) 
Signed-off-by: Povilas Kanapickas <povilas@radix.lt>
 2025-06-05 16:53:08 +00:00
Chiyue WeiandChiyue Wei 61059bee40 [Hardware][NVIDIA] FP4 MoE kernel optimization (#19110) 
Signed-off-by: Chiyue Wei <chiyuew@nvidia.com>
Co-authored-by: Chiyue Wei <chiyuew@nvidia.com>
 2025-06-05 09:48:26 -07:00
Xu Wenqing ec89524f50 Add H20-3e fused MoE kernel tuning configs for DeepSeek-R1/V3 (#19205) 2025-06-05 16:38:54 +00:00
Patrick von Platen f20f9f063b [mistral_common] Add v11 tokenizer (#19193) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2025-06-05 08:27:41 -07:00
Guillaume Calmettes 9bc8bb07cf [Bugfix] properly catch PIL-related errors for vision models when incorrect data urls are provided (#19202) 
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2025-06-05 12:59:28 +00:00
Reidandreidliu41 1aeb925f34 [Frontend] improve vllm run-batch --help display (#19187) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-06-05 11:16:25 +00:00
22quinn 188a4590d8 [Misc] Do not override NCCL_CUMEM_ENABLE if set explicitly (#19105) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-06-05 11:14:32 +00:00
vllmellm 18093084be [Misc] Remove unnecessary fallback to prefill-decode attention (#19138) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-06-05 16:08:26 +08:00
Chauncey 8fc57501d3 [Bugfix]: Fix the incompatibility issue with stream when Thinking is disabled (#19135) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-06-05 06:24:24 +00:00
Woosuk Kwon af7fc84fd2 [BugFix][Minor] Fix full cuda graph bug when max_num_seqs < 512 (#19171) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-05 13:41:25 +08:00
Michael Goin a408820f2f [Bugfix] Fix port handling in make_zmq_path (#19117) 2025-06-04 21:00:59 -06:00
Robert ShawandNick Hill c56ed8bb0e [Bugfix][Nixl] Fix full prefix cache hit bug (#18632) 
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-06-05 02:07:32 +00:00
Nicolò Lucchesi b2fac67130 [P/D] Heterogeneous TP (#18833) 
Signed-off-by: nicklucche <nlucches@redhat.com>
 2025-06-04 23:25:34 +00:00
CYJiang 23027e2daf [Misc] refactor: simplify EngineCoreClient.make_async_mp_client in AsyncLLM (#18817) 
Signed-off-by: googs1025 <googs1025@gmail.com>
 2025-06-04 15:37:25 -07:00
Varun Sundar RabindranathandVarun c3fd4d669a [Kernel] Integrate batched/masked deepgemm kernel (#19111) 
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
 2025-06-04 21:59:18 +00:00
Kebe ef3f98b59f [Bugfix] fix v1 cpu worker fails on macOS (#19121) 2025-06-04 20:17:38 +00:00
Tyler Michael Smith d459fae0a2 [Bugfix][EP+DP] Fix internode check (#19112) 
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
 2025-06-04 23:39:23 +08:00
jmswen c8dcc15921 Allow AsyncLLMEngine.generate to target a specific DP rank (#19102) 
Signed-off-by: Jon Swenson <jmswen@gmail.com>
 2025-06-04 08:26:47 -07:00
Lain 5f2cd251d2 Sm100 blockwise fp8 swap ab (#18564) 2025-06-04 07:48:45 -07:00
Cyrus Leung 01dc9a76db [CI/Build][Bugfix] Ensure compatibility with transformers 4.52 (#18678) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-06-04 04:49:20 -07:00
wang.yuqi 35cf32df30 Improve the output precision of embedding models (#19092) 2025-06-04 11:48:57 +00:00
Isotr0py 8711bc5e68 [Misc] Add packages for benchmark as extra dependency (#19089) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-06-04 04:18:48 -07:00