Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• b57296bb9a [Docs] Fix site_url for RunLLM (#21564) Harry Mellor 2025-07-25 04:05:58 +01:00
• 34ddcf9ff4 [Frontend] run-batch supports V1 (#21541) Cyrus Leung 2025-07-25 11:05:55 +08:00
• fe56180c7f [MoE] More balanced expert sharding (#21497) Woosuk Kwon 2025-07-24 15:56:08 -07:00
• 07d80d7b0e [TPU][TEST] HF_HUB_DISABLE_XET=1 the test 3. (#21539) QiliangCui 2025-07-24 15:33:04 -07:00
• 2dd72d23d9 update flashinfer to v0.2.9rc1 (#21485) weiliang 2025-07-25 05:06:11 +08:00
• a6c7fb8cff [Docs] Add Expert Parallelism Initial Documentation (#21373) Simon MoandHarry Mellor 2025-07-24 12:36:06 -07:00
• a7272c23d0 [Docs][minor] Fix broken gh-file link in distributed serving docs (#21543) Ricardo Decal 2025-07-24 10:36:56 -07:00
• 6066284914 [P/D] Support CPU Transfer in NixlConnector (#18293) 3 people 2025-07-24 09:58:42 -07:00
• 1e9ea8e69d [P/D] Move FakeNixlWrapper to test dir (#21328) Rui Qiao 2025-07-24 08:53:45 -07:00
• d9f9a3fd96 [XPU] Conditionally import CUDA-specific passes to avoid import errors on xpu platform (#21036) Chaojun Zhang 2025-07-24 23:23:36 +08:00
• 1b25f1fe75 Update flashinfer CUTLASS MoE Kernel (#21408) Shu Wang 2025-07-24 10:13:31 -05:00
• e8cb0d0495 [Bug] Fix Compressed Tensor NVFP4 cutlass_fp4_group_mm illegal memory access (#21465) Wentao Ye 2025-07-24 11:13:24 -04:00
• 684174115d [Docs] Rewrite Distributed Inference and Serving guide (#20593) 3 people 2025-07-24 08:13:05 -07:00
• cdb79ee63d [Docs] Update Tensorizer usage documentation (#21190) Sanger SteelandWilliam Goldby 2025-07-24 09:56:18 -04:00
• 5a19a6c670 [Fix] Update mamba_ssm to 2.2.5 (#21421) elvischenv 2025-07-24 18:25:41 +08:00
• 2ded067fd2 [Bugfix] Fix CUDA arch flags for MoE permute (#21426) Ming Yang 2025-07-24 03:23:59 -07:00
• 13abd0eaf9 [Model] Officially support Emu3 with Transformers backend (#21319) Harry Mellor 2025-07-24 11:22:12 +01:00
• 61b8cea3b4 [Attention] Optimize FlashInfer MetadataBuilder Build call (#21137) Lucas Wilkinson 2025-07-24 06:21:46 -04:00
• 526078a96c bump flashinfer to v0.2.8 (#21385) cjackal 2025-07-24 19:20:38 +09:00
• 6da0078523 [Feat] Allow custom naming of vLLM processes (#21445) Chauncey 2025-07-24 18:15:23 +08:00
• 73e3949d07 [Misc] Improve comment for DPEngineCoreActor._set_cuda_visible_devices() (#21501) Rui Qiao 2025-07-24 03:13:40 -07:00
• 6eca337ce0 Replace --expand-tools-even-if-tool-choice-none with --exclude-tools-when-tool-choice-none for v0.10.0 (#20544) Shintarou OkadaandHarry Mellor 2025-07-24 18:56:36 +09:00
• 85bda9e7d0 remove GLM-4.5 quantization wrong Code (#21435) Yuxuan Zhang 2025-07-24 16:52:43 +08:00
• 610852a423 [Core] Support model loader plugins (#21067) 22quinn 2025-07-24 01:49:44 -07:00
• f0f4de8f26 [Misc] Fix duplicate FusedMoEConfig debug messages (#21455) Nick Hill 2025-07-24 09:27:30 +01:00
• fc5f756db4 [v1][Core] Clean up usages of SpecializedManager (#21407) Zhou Fang 2025-07-24 00:40:11 -07:00
• e74bfc70e4 [TPU][Bugfix] fix moe layer (#21340) Chengji YaoandSimon Mo 2025-07-24 00:38:39 -07:00
• 90eeea8f85 [Bugfix][ROCm] Fix for warp_size uses on host (#21205) Gregory Shtrasberg 2025-07-24 03:37:19 -04:00
• dde295a934 Deduplicate Transformers backend code using inheritance (#21461) Harry Mellor 2025-07-24 08:16:23 +01:00
• 6d8d0a24c0 Add think chunk (#21333) Julien Denize 2025-07-24 06:51:32 +02:00
• 11ef7a611e [BugFix] Set CUDA_VISIBLE_DEVICES before spawning the subprocesses (#21211) 3 people 2025-07-23 21:44:04 -07:00
• dc2f159f8a Dump input metadata on crash for async scheduling (#21258) Woosuk Kwon 2025-07-23 21:10:30 -07:00
• d5b981f8b1 [DP] Internal Load Balancing Per Node [one-pod-per-node] (#21238) 4 people 2025-07-23 23:57:32 -04:00
• eec6942014 [BugFix] Fix KVConnector TP worker aggregation (#21473) Nick Hill 2025-07-24 04:56:49 +01:00
• fd48d99ffd [BugFix]: Batch generation from prompt_embeds fails for long prompts (#21390) KazusatoOokoandKazusatoOko 2025-07-23 20:43:17 -07:00
• f8c15c4efb [Bugfix] Fix example disagg_example_p2p_nccl_xpyd.sh zombie process (#21437) WeiQing Chen 2025-07-24 11:42:11 +08:00
• aa08a954f9 [Bugfix] Fix casing warning (#21468) Matthew Bonanni 2025-07-23 23:41:23 -04:00
• 13e4ee1dc3 [XPU][UT] increase intel xpu CI test scope (#21492) Liangliang Ma 2025-07-24 11:24:04 +08:00
• 772ce5af97 [Misc] Add dummy maverick test to CI (#21324) Ming YangandCyrus Leung 2025-07-23 20:22:42 -07:00
• 63d92abb7c [Frontend] Set MAX_AUDIO_CLIP_FILESIZE_MB via env var instead of hardcoding (#21374) deven-labovitch 2025-07-23 23:22:19 -04:00
• 11599b0e1f feat(gguf_loader): accept HF repo paths & URLs for GGUF (#20793) 3 people 2025-07-23 20:21:02 -07:00
• f3137cdd81 [Core] Freeze gc during cuda graph capture to speed up init (#21146) Michael Goin 2025-07-23 20:20:14 -04:00
• 82ec66f514 [V0 Deprecation] Remove Prompt Adapters (#20588) Michael Goin 2025-07-23 19:36:48 -04:00
• 78c13e30e1 [V1] Fix local chunked attention always disabled (#21419) Yong Hoon Shin 2025-07-23 15:59:30 -07:00
• 5c9b807b34 [Core] Add reload_weights RPC method (#20096) 22quinn 2025-07-23 14:24:52 -07:00
• 14bf19e39f [TPU][TEST] Fix the downloading issue in TPU v1 test 11. (#21418) QiliangCui 2025-07-23 11:29:36 -07:00
• 4ac7713e32 Add test case for compiling multiple graphs (#21044) Yong Hoon Shin 2025-07-23 11:00:47 -07:00
• 8560a5b258 [Core][Model] PrithviMAE Enablement on vLLM v1 engine (#20577) Christian Pinto 2025-07-23 19:00:23 +01:00
• 316b1bf706 [Tests] Add tests for headless internal DP LB (#21450) Nick Hill 2025-07-23 15:49:25 +01:00
• 7c734ee09b [Bugfix][Qwen][DCA] fixes bug in dual-chunk-flash-attn backend for qwen 1m models. (#21364) Tao He 2025-07-23 21:34:37 +08:00
• f59ec35b7f [V1] Check all pooling tasks during profiling (#21299) Cyrus Leung 2025-07-23 20:53:26 +08:00
• 2671334d45 [Model] add Hunyuan V1 Dense Model support. (#21368) Asher 2025-07-23 18:54:08 +08:00
• 2cc5016a19 [Docs] Clean up v1/metrics.md (#21449) Michael Yao 2025-07-23 18:37:25 +08:00
• 6929f8b437 [Misc] fixed nvfp4_moe test failures due to invalid kwargs (#21246) Yang Chen 2025-07-23 01:41:43 -07:00
• 32ec9e2f2a Mamba V2 Test not Asserting Failures. (#21379) Yu Chin Fabian Lim 2025-07-23 04:40:27 -04:00
• accac82928 [Sampler] Introduce logprobs mode for logging (#21398) Lu Fang 2025-07-23 01:39:25 -07:00
• 23637dcdef [Docs] Fix bullets and grammars in tool_calling.md (#21440) Michael Yao 2025-07-23 16:23:20 +08:00
• 6364af92f8 Fixed typo in profiling logs (#21441) Sergio Paniego Blanco 2025-07-23 10:18:54 +02:00
• 7aaa2bd5a8 [Bugfix] ensure tool_choice is popped when tool_choice:null is passed in json payload (#19679) Guillaume Calmettes 2025-07-23 09:30:05 +02:00
• 2f5c14de6a add clear messages for deprecated models (#21424) youkaichao 2025-07-23 15:03:16 +08:00
• f002e9a870 [Cleanup] Only log MoE DP setup warning if DP is enabled (#21315) Michael Goin 2025-07-23 03:02:48 -04:00
• a1f3610fc6 [Core] Add basic unit test for maybe_evict_cached_block (#21400) Jialin Ouyang 2025-07-23 00:02:02 -07:00
• 4ecedd1806 [Bugfix] Fix nightly transformers CI failure (#21427) Isotr0py 2025-07-23 15:01:01 +08:00
• 107111a859 Changing "amdproduction" allocation. (#21409) Alexei-V-Ivanov-AMD 2025-07-22 22:48:31 -05:00
• 2dec7c1a5d [Bugfix][CUDA] fixes CUDA FP8 kv cache dtype supported (#21420) elvischenv 2025-07-23 11:34:50 +08:00
• 08d2bd78da [BUGFIX] deepseek-v2-lite failed due to fused_qkv_a_proj name update (#21414) Chendi.Xue 2025-07-22 22:33:57 -05:00
• 4f76a05f4f [BugFix] Update python to python3 calls for image; fix prefix & input calculations. (#21391) ericehanleyandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-07-22 22:33:00 -05:00
• f154bb9ff0 Simplify weight loading in Transformers backend (#21382) Harry Mellor 2025-07-23 04:29:43 +01:00
• 3ec7170ff1 [Bugfix][ROCm][Build] Fix build regression on ROCm (#21393) Gregory Shtrasberg 2025-07-22 23:27:41 -04:00
• c401c64b4c [CI/Build] Fix model executor tests (#21387) Cyrus Leung 2025-07-23 11:25:37 +08:00
• b77c7d327f [BugFix] Fix ray import error mem cleanup bug (#21381) Joe RundeandTravis Johnson 2025-07-22 17:19:55 -06:00
• 35bc8bd5fb [Misc] Copy HF_TOKEN env var to Ray workers (#21406) Rui Qiao 2025-07-22 16:18:42 -07:00
• 4594fc3b28 [Model] Add Qwen3CoderToolParser (#21396) Yiheng Xuandsimon-mo 2025-07-22 15:05:57 -07:00
• ae268b6326 Fix Flashinfer Allreduce+Norm enable disable calculation based on fi_allreduce_fusion_max_token_num (#21325) Xin Li 2025-07-22 15:42:31 -04:00
• 35366ae57c [CI/Build] Fix test failure due to updated model repo (#21375) Cyrus Leung 2025-07-22 23:39:35 +08:00
• 2226d5bd85 [Bugfix] Decode Tokenized IDs to Strings for hf_processor in llm.chat() with model_impl=transformers (#21353) Aritra Roy Gosthipaty 2025-07-22 20:57:28 +05:30
• 44554a0068 Add tokenization_kwargs to encode for embedding model truncation (#21033) Wang Yijun 2025-07-22 23:24:00 +08:00
• 226b452a20 Revert "[Refactor] Fix Compile Warning #1444-D (#21208)" (#21384) Wentao Ye 2025-07-22 11:22:10 -04:00
• f38ee34a0a [feat] Enable mm caching for transformers backend (#21358) Raushan Turganbay 2025-07-22 17:18:46 +02:00
• b194557a6c Adds parallel model weight loading for runai_streamer (#21330) Benjamin BartelsandCyrus Leung 2025-07-22 16:15:53 +01:00
• 774d0c014b [Perf] Cuda Kernel for Per Token Group Quant (#21083) Wentao Ye 2025-07-22 10:27:15 -04:00
• 2c8db17cfd [feat]: add SM100 support for cutlass FP8 groupGEMM (#20447) 3 people 2025-07-22 07:27:12 -07:00
• 4fb56914c5 [perf] Add fused MLA QKV + strided layernorm (#21116) Mickaël Seznecandmgoin 2025-07-22 16:07:44 +02:00
• 0df4d9b06b [Misc] unify variable for LLM instance v2 (#21356) Ning Xie 2025-07-22 21:32:36 +08:00
• ed25054577 [Core] Introduce popleft_n and append_n in FreeKVCacheBlockQueue to further optimize block_pool (#21222) Jialin Ouyang 2025-07-22 06:17:47 -07:00
• 10904e6d75 [benchmark] Port benchmark request sent optimization to benchmark_serving (#21209) Jialin Ouyang 2025-07-22 05:28:00 -07:00
• a32237665d [Core] Optimize update checks in LogitsProcessor (#21245) Jialin Ouyang 2025-07-22 05:27:18 -07:00
• bc8a8ce5ec [Misc] Remove deprecated args in v0.10 (#21349) Kebe 2025-07-22 20:26:39 +08:00
• 32142b3c62 [Bugfix] Fix eviction cached blocked logic (#21357) Simon Mo 2025-07-22 01:18:40 -07:00
• 82b8027be6 Add arcee model (#21296) Raghav RavishankarandJee Jee Li 2025-07-22 13:27:43 +05:30
• 3779eb8c81 [Feature][eplb] add verify ep or tp or dp (#21102) rongfu.leng 2025-07-22 14:41:14 +08:00
• 9e23ad9655 Update fp4 quantize API (#21327) Shu Wang 2025-07-22 01:40:21 -05:00
• e69a92a1ce [Bug] DeepGemm: Fix Cuda Init Error (#21312) Wentao Ye 2025-07-22 02:36:18 -04:00
• 8425f785ad [Misc] DeepEPHighThroughtput - Enable Inductor pass (#21311) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-07-22 12:05:45 +05:30
• c17231e827 Fix kv_cache_dtype handling for out-of-tree HPU plugin (#21302) Konrad ZaworaandChendi.Xue 2025-07-22 08:35:14 +02:00
• 6e5b5ca580 [Refactor] Fix Compile Warning #1444-D (#21208) Wentao Ye 2025-07-22 02:33:51 -04:00
• 488d8a986a [V1] [Hybrid] Add new test to verify that hybrid views into KVCacheTensor are compatible (#21300) Thomas Parnell 2025-07-22 08:31:18 +02:00
• af376ca19d [Core] Minimize number of dict lookup in _maybe_evict_cached_block (#21281) Jialin Ouyang 2025-07-21 22:37:34 -07:00
• e7b2042681 Revert "[Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) (#21334) Ming Yang 2025-07-21 21:49:01 -07:00
• 90f1e55421 [Intel GPU] Ray Compiled Graph avoid NCCL for Intel GPU (#21338) Ratnam Parikh 2025-07-21 21:48:27 -07:00