Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

47b7af0d87 [Feat] Enable CompressedTensorW4A8Int for XPU (#37207) Tianmu Li 2026-03-19 19:34:28 -07:00
269bf46d99 fix: disambiguate multimodal prefix cache keys (#36708) tianshu-Michael-yu 2026-03-19 19:33:20 -07:00
e5a77a5015 [CI] Update mergify tool-calling label paths (#37478) Flora Feng 2026-03-19 22:22:23 -04:00
ca1ac1a4b4 Fix DP coordinator ZMQ port TOCTOU (#37452) Itay Alroy 2026-03-20 02:58:31 +02:00
4ca3fa6bb4 [ROCm][Bugfix] fix cache block size mismatch for aiter unified attention (#37606) Divakar Verma 2026-03-19 20:00:08 -04:00
be12afd284 [Bugfix] Fix Deepseekv32 tool parser when stream interval > 1 (#36056) Flora Feng 2026-03-19 19:51:25 -04:00
df3c0291a3 [Bug] Fix EmbedIOprocessor "classify" <-> "embed" (#37573) Wentao Ye 2026-03-19 19:40:10 -04:00
2be1a0f74b [Refactor] Remove dead code in pooling model (#37572) Wentao Ye 2026-03-19 19:39:43 -04:00
4120a05ff1 Fix AttributeError in Qwen3.5 GDN layers with quantized models (#37448)
Jim Smith mgoin Xin Yang
2026-03-19 19:21:14 -04:00
98ff042917 [CI][BugFix][AMD] Don't set VLLM_ROCM_USE_AITER anymore in test_rocm_aiter_topk since its not necessary (#36996) rasmith 2026-03-19 18:12:45 -05:00
b55156eae9 [Performance] Enable Triton autotuning disk cache by default (#37188) Artem Perevedentsev 2026-03-19 23:36:28 +02:00
112944fab9 test Qwen/Qwen3-4B-Instruct-2507 for unbacked (#36064) Laith Sakka 2026-03-19 14:28:45 -07:00
91be5f9be3 [MoE Refactor] Rename "naive" all2all backend (#36294) bnellnm 2026-03-19 15:50:34 -04:00
4ee847e400 Comment fix for async rl example (#35244) Aaron Hao 2026-03-19 12:46:07 -07:00
040a505ff5 [ROCm][CI] Cleaning and restructuring amd-ci legacy pipeline (#34839) Andreas Karatzas 2026-03-19 14:30:58 -05:00
9279c59a0e [MoE Refactor] DefaultMoERunner simplifcation (#33049) bnellnm 2026-03-19 15:07:44 -04:00
7454096199 [Log] Log once in local node by default (#37568) Wentao Ye 2026-03-19 15:04:59 -04:00
fb8b5e05fc [CI] Add retry with 4x backoff to HTTP fetches for transient failures (#37218) Andreas Karatzas 2026-03-19 14:00:20 -05:00
e5d96dc8fc Fix SpeculatorsConfig now that PreTrainedConfig is a dataclass in Transformers (#37574) Harry Mellor 2026-03-19 18:04:40 +00:00
daa05bf340 [Bugfix] Fix AttributeError when serving MXFP8 models with DeepGEMM installed (#37358) EdalatiAliandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-03-19 13:58:33 -04:00
7769b58307 [torch.compile][BE][Multimodal] Remove requirement to set_model_tag to avoid cache conflict (#37345) Lucas Kabela 2026-03-19 10:26:12 -07:00
2f9f946b22 [P/D] AnthropicMessages add kv_transfer_params for PD disaggregation (#37535) Chauncey 2026-03-20 00:41:20 +08:00
2890aecce5 [CPU][UX] Do not crash when tcmalloc/libiomp are not ldpreloaded (#37561) Fadi Arafeh 2026-03-19 16:35:45 +00:00
34f093b417 [CI] Gate pre-commit on ready label or number of contributions (#37544) Harry Mellor 2026-03-19 16:21:57 +00:00
4dce8321a9 Run MacOS smoke test on daily cron job instead of every commit (#37567) Harry Mellor 2026-03-19 16:19:50 +00:00
657855ab41 [Misc] Cleanup more configs and processors (#37560) Cyrus Leung 2026-03-19 23:45:23 +08:00
e27b8ba3d1 [Bug] Fix fp8 trtllm MoE modular kernel supported routing methods (#37346) Wei Zhao 2026-03-19 11:43:06 -04:00
40b8363b45 [MRV2] Use fp32 for draft logits (#37526) Woosuk Kwon 2026-03-19 08:41:21 -07:00
8b10e4fb31 [1/n] Migrate permute_cols to libtorch stable ABI (#31509) mikaylagawarecki 2026-03-19 11:27:26 -04:00
104605cbf2 Remove deprecated reasoning_content message field(part-2) (#37480) +2
Ifta khairul Alam Adil JartX Netanel Haber Wentao Ye Philip Ottesen Woosuk Kwon Michael Goin Giancarlo Delfin Andy Lo Thillai Chithambaram sihao_li Harry Mellor
2026-03-19 16:20:08 +01:00
96266f119b [LoRA] Minor improvements to LoRA log (#37557) Jee Jee LiandCyrus Leung 2026-03-19 23:18:06 +08:00
7c0cf3bcd0 Cap the number of API servers to 1 when using Elastic EP. (#37466) Sage Moore 2026-03-19 07:42:57 -07:00
572b432913 Stop bench CLI from recursively casting all configs to dict (#37559) Harry Mellor 2026-03-19 14:04:03 +00:00
9515c20868 [Misc] Clean up processing logic (#37541) Cyrus Leung 2026-03-19 21:30:20 +08:00
c63ca2b2e6 [Bugfix] Add Kimi-K2.5 reasoning/tool parser aliases and tool_call_id support (#37438) DorBernsohn 2026-03-19 15:08:00 +02:00
a32eaf5bb2 [CI] Merge cleanup_pr_body.yml and reminder_comment.yml (#37552) Harry Mellor 2026-03-19 12:55:07 +00:00
e390742c59 Fix KV Offloading + MLA AssertionError by using num_kv_heads=1 in cpu… (#37536) XueLiang Yangandxueliangyang-oeuler 2026-03-19 20:05:07 +08:00
7a6ebcbfcf [Model] Remove unnecessary get_language_model (#37545) Cyrus Leung 2026-03-19 20:00:36 +08:00
c7bc12c20f [CI/Build] Split out MM pooling tests (#37542) Cyrus Leung 2026-03-19 19:36:11 +08:00
f9e2a38386 [Docs] Reorganize pooling docs. (#35592)
wang.yuqi Cyrus Leung Harry Mellor
2026-03-19 19:25:47 +08:00
4426447bba Don't log exc_info when vLLM tries to doenload a file that doesn't exist (#37458) Harry Mellor 2026-03-19 10:38:29 +00:00
3322e26420 [Bugfix] Avoid more OpenMP thread reallocation in CPU torch compile (#37538) Li, Jiang 2026-03-19 18:24:39 +08:00
765e461065 [Bugfix] Fix Nemotron Parse loading (#37407) Cyrus Leung 2026-03-19 17:55:29 +08:00
6a9cceb219 [Bugfix][ROCm] Fix MoRI + AITER FP8 dispatch compatibility for defer_input_quant (#37418) Duyi-Wang 2026-03-19 17:49:27 +08:00
199f914183 fix(cpu): add null check for aligned_alloc in ScratchPadManager (#37369) yassha 2026-03-19 10:45:06 +01:00
ca21483bf9 [MISC] fix pin_memory=torch.cuda.is_available(), use is_pin_memory_available (#37415) Kunshang Ji 2026-03-19 17:23:24 +08:00
da70c87e81 [CI] Fix wrong path test file, missing rlhf_async_new_apis.py (#37532) TJian 2026-03-19 17:21:55 +08:00
0b6d52629f Support temporal compression for Nemotron-3-VL videos (#36808) Collin McCarthy 2026-03-19 01:02:19 -07:00
d3cc379567 [Perf] Fix slow hasattr in CUDAGraphWrapper.__getattr__ (#37425) Ziming Huang 2026-03-19 15:43:48 +08:00
354cd580d5 fix(anthropic): remove non-standard 'data: [DONE]' from Anthropic streaming (#37510) cdpath 2026-03-19 15:23:35 +08:00
d49f273144 [SSM/Mamba] Follow-up: N-1 prefill for P/D disaggregation (#37310) zhanqiuhu 2026-03-19 03:22:00 -04:00
b21d384304 [Refactor] Relocate endpoint tests to mirror serving code directory structure (#37504) Flora Feng 2026-03-19 03:19:36 -04:00
e3126cd107 [ROCm] issue management - request information for bug issues on ROCm (#37009) Hongxia Yang 2026-03-18 23:51:29 -04:00
e37ff5b5c8 [Perf] Optimize token_embed for pooling models, 1.0% token throughput improvement (#37347) Wentao Ye 2026-03-18 22:27:51 -04:00
6accb21f2a [bug] Fix deadlock with pause resume and collective_rpc (#37024) Aaron Hao 2026-03-18 18:49:02 -07:00
053f3b6309 [Model Runner V2] Spec decode rejection sampler logprobs support (#37237) Giancarlo Delfin 2026-03-18 18:36:27 -07:00
5f82706a21 [BUG] Exclude SKIP_TENSORS from get_layer_size() + new weight sync example for dpep (#37334) Aaron Hao 2026-03-18 17:45:10 -07:00
c32a58cc2a [EPLB] Simplify EPLB rearrange by only returning one map (#36267) Sage MooreandTyler Michael Smith 2026-03-18 17:34:00 -07:00
ef2c4f778d [Bugfix] Zero-init MLA attention output buffers to prevent NaN from CUDA graph padding (#37442) Elvir CrnčevićandMatthew Bonanni 2026-03-19 01:28:37 +01:00
9dade5da3a [XPU]Unify xpu test dependencies in dockerfile.xpu (#36477) sihao_li 2026-03-19 08:12:07 +08:00
828f862acb [Bugfix] Expand quantization method support in perf metrics (#37231) Thillai Chithambaram 2026-03-18 19:54:19 -04:00
577df69b26 [Bugfix] Fix KV scales inconsistency in fp8 MLA & FlashInfer kv_cache_dtype "auto" leading to gibberish (#37054) Andy Lo 2026-03-18 23:07:29 +00:00
04244fd0e1 [Model Runner V2] Spec decode rejection sampler greedy support (#37238) Giancarlo Delfin 2026-03-18 15:59:03 -07:00
9482b0b085 [Bugfix] Remove assertion for NVFP4 scale dynamic range (#37465) Michael Goin 2026-03-18 23:37:49 +01:00
5bc1da147f [LoRA][BugFix] Fix skipped LoRA adapters for Mistral3 (#36928) Woosuk Kwon 2026-03-18 15:34:19 -07:00
0091017188 fix(worker): optimize swap_states to copy only active token prefixes (#34733) Philip Ottesen 2026-03-18 22:59:27 +01:00
0d81a1fe61 [V0 Deprecation] Deprecate virtual engine (#37195) Wentao Ye 2026-03-18 17:30:14 -04:00
6ae4c8d6fc chunk parakeet into 30s clips to prevent OOMs on long audios (#36671) Netanel Haber 2026-03-18 23:22:24 +02:00
a913b612d8 [Bugfix] Fix ROCm crash in qwen3_next multi-stream events (#36795) (#37427) JartX 2026-03-18 21:06:31 +01:00
5ce2d10e4a Fix models which use layer_type_validation for Transformers v5 (#37398) Harry Mellor 2026-03-18 18:41:51 +00:00
738d0a281f [Bugfix] Fix incorrect use of merge_size in Qwen3-VL video timestamp calculation (#37439) Chengyu Fang 2026-03-19 02:36:34 +08:00
70b81c4f3d [bugfix][async scheduling] fix extra cuda context in device 0 with EP/DP (#37449) youkaichao 2026-03-19 02:32:30 +08:00
7476d148db [Model] Remove unnecessary processor definition for Nemotron Parse (#37456) Cyrus Leung 2026-03-19 02:25:13 +08:00
f3732bd931 [Misc] Clean up model registry (#37457) Cyrus Leung 2026-03-19 02:24:44 +08:00
0ef7f79054 [Perf] Add tuned triton moe config for Qwen3.5 H200, 9.9% E2E throughput improvement (#37340) Wentao Ye 2026-03-18 14:18:34 -04:00
5dd8df0701 [kv_offload+HMA][2/N]: Support multiple KV groups in GPULoadStoreSpec (#36642) Or Ozeri 2026-03-18 19:26:40 +02:00
39bfb57b7c Add API docs link if the CLI arg is a config class (#37432) Harry Mellor 2026-03-18 17:19:35 +00:00
c9d838fc33 Adding deterministic lora benchmarking to vLLM Bench (#36057) RonaldBXu 2026-03-18 09:02:03 -07:00
b1169d7be8 [Kernel] Add gpt-oss Router GEMM kernel (#37205) Xin Yang 2026-03-18 08:15:56 -07:00
17808394bc standardize load_weights using AutoWeightsLoader for kimi_linear and minimax_text_01 (#37371) XLiu-2000andXuLiu 2026-03-18 23:05:37 +08:00
296839a1b0 [Perf] Eliminate padding and slicing op for GPT-OSS with Flashinfer MXFP4 MXFP8 MoE (#30647) elvischenv 2026-03-18 23:01:26 +08:00
c373b5c00d [Log] Reduce duplicate log (#37313) Wentao Ye 2026-03-18 10:57:44 -04:00
de1a86b7de elastic_ep: Fix stateless group port races (#36330) Itay Alroy 2026-03-18 16:36:18 +02:00
99267c23ca [2/3] Refactor InternVL-based processors (#37324) Cyrus Leung 2026-03-18 22:22:19 +08:00
525f2eeb0b [kv_offload+HMA][6/N]: Split offloading_connector.py (#37405) Or Ozeri 2026-03-18 15:42:46 +02:00
918b7890a1 [Bugfix] Fix base64 JPEG video frames returning empty metadata (#37301)
Yufeng He Yufeng He Isotr0py
2026-03-18 21:40:03 +08:00
98b09ddc27 [NIXL][Bugfix] metrics & testing minor bug (#36051) Andy Lo 2026-03-18 13:39:14 +00:00
cef1f302d2 [Model] Enable LoRA support for tower and connector in H2OVL (#31696) Shwetha Poojary 2026-03-18 18:56:47 +05:30
17c47fb869 [Bugfix] Fix EP weight filter breaking EPLB and NVFP4 accuracy (#37322)
Elvir Crnčević Claude Opus 4.6 Kevin H. Luu
2026-03-18 11:30:29 +01:00
b322b197f1 [Build] Bump python openai version (#32316) Chauncey 2026-03-18 18:20:10 +08:00
eaf7c9b976 [CI] Fix PaddleOCR-VL HF test failure due to create_causal_mask API rename (#37328) Andreas Karatzas 2026-03-18 04:44:12 -05:00
47a1f11bff [docs] Add docs for new RL flows (#36188) Aaron HaoandHarry Mellor 2026-03-18 02:04:26 -07:00
fad09e8a1f fix(glm47): improve tool call parsing and content normalization (#37386) Karan Bansalandkaranb192 2026-03-18 13:42:21 +05:30
8c31f47c63 [LoRA] Make LoRA respect language_model_only (#37375) Jee Jee Li 2026-03-18 15:53:34 +08:00
261801242f [Bugfix] Avoid OpenMP thread reallocation in CPU torch compile (#37391) Li, Jiang 2026-03-18 15:51:39 +08:00
fcf0687b27 [kv_offload+HMA][0/N]: Support block-level preemption handling (#34805) Or OzeriandNicolò Lucchesi 2026-03-18 08:49:53 +02:00
86b7e3c95a [XPU] skip unsupported ut and update test_nixl_connector (#37179) liuzhenweiandKunshang Ji 2026-03-18 13:32:59 +08:00
0e95916155 [responsesAPI] parser.extract_response_outputs can take in token IDs (#37130) Andrew Xia 2026-03-17 22:31:31 -07:00
ce2ef42fd3 [CI] Stabilize test_cpu_offloading by waiting for async offload before cache reset (#37335) Andreas Karatzas 2026-03-18 00:26:20 -05:00
8b6325758c [ROCm][CI] Add ROCM_EXTRA_ARGS to audio_in_video test server fixture (#37349) Andreas Karatzas 2026-03-17 23:55:40 -05:00

... 82 83 84 85 86 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86