Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

a0dd1995c7 [Hardware][TPU] Add supports_async_scheduling() method to Executor interface so that it can be extended for Executor implementations. (#36924) gxd3 2026-03-17 21:53:28 -07:00
f1740006e4 [Perf] Enable dual stream execution of input projection for Qwen3 (#36795) Xin Yang 2026-03-17 20:13:27 -07:00
58cde5c026 [ROCm][CI] Skip trtllm kvfp8 dequant tests on ROCm (#37330) Andreas Karatzas 2026-03-17 22:12:26 -05:00
761e0aa7a0 [Performance] Add --enable-ep-weight-filter CLI option (#37351) Roy WangandClaude Opus 4.6 2026-03-18 09:36:55 +08:00
ff9fbc9aff [Kernel][Helion] [16/N] Refactor register_kernel API to be more Dynamo-friendly (#36705) Yanan CaoandClaude Opus 4.6 2026-03-17 18:23:35 -07:00
e6c4797704 [ROCm][Quantization] add fp8xfp8 attn support for rocm_aiter_unified_attn (#36927) Divakar Verma 2026-03-17 20:49:32 -04:00
09e4576f65 [Kernel] Add non-gated support for NVFP4 CUTLASS MoE (#37320) Michael Goin 2026-03-17 23:12:04 +01:00
3ed7b1e6e0 [ROCm] Validate block_size for explicitly selected attention backends (#36846) Andreas Karatzas 2026-03-17 17:04:40 -05:00
e8f9dbc369 [Bugfix][ROCm] Fix worker startup OOM on ROCm by skipping unreliable cudagraph memory profiling (#36720) JartX 2026-03-17 22:55:34 +01:00
de35c06c66 Make KV connector metadata build overridable via plugin (#37336) Yong Hoon Shin 2026-03-17 14:29:06 -07:00
c0745a851a [Model] Add ColQwen3.5 4.5B support (#36887) Athrael Sojuandwang.yuqi 2026-03-17 21:17:02 +00:00
b5ca9c3557 [Models] Cohere ASR (#35809) Ekagra Ranjan 2026-03-17 17:04:17 -04:00
245758992e [Bugfix] Rescale NVFP4 weight scales to fix BF16 dequant underflow (#34577) Chao-Ju ChenandMichael Goin 2026-03-18 04:48:42 +08:00
1204cf0a9d [Bugfix] Fix mock.patch resolution failure for standalone_compile.FakeTensorMode on Python <= 3.10 (#37158) Dimitrios BariamisandDimitrios Bariamis 2026-03-17 21:13:06 +01:00
b36adfa349 [Perf] Set Flashinfer sparse MLA as default backend for FP8 kv cache (#37252) Wei Zhao 2026-03-17 16:09:20 -04:00
e78821b438 [Deprecation] Deprecate --calculate-kv-scales option (#37201) Michael Goin 2026-03-17 20:57:24 +01:00
51f0acda79 [Model] Remove unused handle_oov_mm_token (#37321) Cyrus Leung 2026-03-18 03:44:52 +08:00
fa75204b16 bump compressed-tensors version to 0.14.0.1 (#36988) Brian DellabettaandDipika Sikka 2026-03-17 15:36:19 -04:00
bdb903bb5f [Bug] Fix FlashInfer MNNVL socket collisions under concurrent vLLM jobs (#36674) Wentao Ye 2026-03-17 15:19:52 -04:00
68f783a727 [Torch 2.11] Guard torch._C._cpu attribute checks for forward compatibility (#35673) Andrey Talman 2026-03-17 14:47:59 -04:00
c5030c439d [CI] Split Distributed Tests (4 GPUs) and Kernel MoE tests (#37100)
Avinash Singh Claude Opus 4.6 Kevin H. Luu
2026-03-18 00:14:55 +05:30
51b2333be1 [Perf] Optimize top-k search in apply_top_k_top_p_triton sampler (#37225) Michael Goin 2026-03-17 19:35:17 +01:00
4ed51308c8 [CI] Fix GPU memory leak when RemoteOpenAIServer fails to start in __init__ (#37230) Andreas Karatzas 2026-03-17 11:08:08 -05:00
c781fbbab3 [Bugfix] Standardize custom HF Processor init (#37289) Cyrus Leung 2026-03-17 23:38:55 +08:00
979ff44cea [BugFix] PyTorch Compilation Tests should error if any test fails (#37300) Richard Zou 2026-03-17 11:26:38 -04:00
f63ed7b5ac [Bugfix] Fix DP MTP Dummy Run (#35243) Benjamin Chislett 2026-03-17 11:16:48 -04:00
c9e5096256 [openapi] remove redundant exception stack trace[4/N] (#37157) Ning Xie 2026-03-17 23:06:25 +08:00
2ff0ad9694 [UltraVox] Fix output type (#37224) Anton VlasjukandHarry Mellor 2026-03-17 15:51:17 +01:00
a836524d20 [Chore] Replace all base64 usages with faster pybase64 package (#37290) Isotr0py 2026-03-17 22:44:19 +08:00
3717a4dd47 [Misc][LoRA] Add --lora-target-modules to restrict LoRA to specific modules (#34984) BhoomitandHarry Mellor 2026-03-17 07:36:41 -07:00
ecfcdd2ce4 Fix Phi3 test that fails with Transformers v5 (#37298) Harry Mellor 2026-03-17 14:29:24 +00:00
c25dbc2d27 [Bugfix] Fix unclean shutdown crash with AllReduce Fusion workspace (#36955) Siew's Capital Jarvis 2026-03-17 22:22:09 +08:00
77d2a5f17b pick up tuned prefill configs for FP8 FA3 (#36265) Jonas M. Kübler 2026-03-17 15:00:26 +01:00
59192dfd39 [Frontend] Complete OpenAI render delegation (#37287) Sage 2026-03-17 15:53:55 +02:00
56cb1baa66 [Misc] Use VLLMValidationError in batch, pooling, and tokenize protocol validators (#36256) Umut Polat 2026-03-17 16:52:30 +03:00
f340324335 [1/2] Move InternVL-based processors (#37260) Cyrus Leung 2026-03-17 21:50:56 +08:00
2660b9289c Bugfix for offloading+prefetch for GLM-4.7-FP8 (#37178) sfbemerkandBenjamin Merkel 2026-03-17 14:22:09 +01:00
293f036e6d Add gigachat 3.1 tool parser + fix gigachat3 tool parser (#36664) Viacheslav 2026-03-17 15:03:20 +03:00
0fb142a454 [perf][connector] optimize build_connector_meta when host buffer transfer is not used (#37165) youkaichao 2026-03-17 19:59:35 +08:00
00f8e0d211 [Frontend] Delegate tokenization serving preprocessing to OpenAIServingRender (#37266) Sage 2026-03-17 13:22:54 +02:00
4af9ed21cb [Bugfix](xpu): prevent “selected index k out of range” in TP decode path (#37259) zhao, zhenhui 2026-03-17 19:14:07 +08:00
9c7cab5ebb [Feature]: Support for multiple embedding types in a single inference call (#35829) Augusto Yao 2026-03-17 17:05:42 +08:00
132bfd45b6 [Bugfix][ResponsesAPI] Fix crash when tool_choice=required exceeds max_output_tokens (#37258) Chauncey 2026-03-17 16:54:52 +08:00
24b4272a8c Fix infinite recursive search issue in quark.py (#32779)
xiao-llm Yanwen Lin Kim Hee Su
2026-03-17 03:19:15 -04:00
8a680463fa [Bugfix] Fix NemotronH MTP + Chunked Prefill (#35447) Benjamin Chislett 2026-03-17 02:07:33 -04:00
20b14095a4 [Bugfix] Fix loading Music Flamingo (#35535) Nick Cao 2026-03-17 01:24:40 -04:00
17c1bdf371 [Bugfix] dtype mismatch in ngram gpu propose (#37246) PatchyTISandPatchouliTaisa 2026-03-17 13:19:55 +08:00
3e3d320c1b [Refactor] Relocate responses API tests (#37241) Flora Feng 2026-03-17 01:14:52 -04:00
54a62a79f7 [ROCm] Fix AttributeError for torch.compiler.skip_all_guards_unsafe on older PyTorch (#37219) Andreas Karatzas 2026-03-16 22:34:49 -05:00
384dc7f77b [Refactor] Relocate completion and chat completion tests (#37125) Flora Feng 2026-03-16 23:31:23 -04:00
f04d5226f8 [CI] Fix flaky tool_use chat completion tests with deterministic seed (#37027) Flora Feng 2026-03-16 23:24:34 -04:00
0a0a1a198b Add ability to replace oot ops when using lora (#37181) Kyuyeun Kim 2026-03-16 18:04:15 -07:00
6c1cfbad32 Support non-contiguous KV cache in TRTLLM fp8 dequant kernel (#36867) Vadim GimpelsonandPavani Majety 2026-03-17 04:48:42 +04:00
45f526d652 [BugFix] Correct max memory usage for multiple KV-cache groups (#36030) Harry Huang 2026-03-17 08:38:52 +08:00
5db91f0aaf Fix some Mistral parser issues (#37209) Julien Denize 2026-03-17 01:08:56 +01:00
061980c36a [Feature][Frontend] add support for Cohere Embed v2 API (#37074) Walter Beller-Morales 2026-03-16 19:55:53 -04:00
7a49742b88 [CI/Build] Add common tool call parser test suite (#27599) Ben Browning 2026-03-16 19:46:20 -04:00
3e6a1e1686 [Custom Ops] Add functional + out variant for scaled_fp4_quant (#34389) Terry Gao 2026-03-16 15:51:46 -07:00
7961486a9b Fix EagleMistralLarge3Model initialization (#37232) Julien Denize 2026-03-16 23:41:00 +01:00
4f9b14c21c [CI] Stabilize multinode DP internal LB completion tests (#36356) Andreas Karatzas 2026-03-16 17:40:23 -05:00
31a458c091 [Doc] Clarify schema enforcement behavior for tool_choice modes (#37064) Yuchen Fama 2026-03-16 18:27:42 -04:00
a3a51d20e7 [Benchmark] Improvements to attention benchmark script (#37115) Wei Zhao 2026-03-16 18:22:40 -04:00
e5b807607c [Quant][Feature] Support online MXFP8 quantization for MoE and dense models (#35448) EdalatiAli 2026-03-16 18:07:39 -04:00
fd4d96302a Fix eplb nvfp4 experts hook (#37217)
Elvir Crnčević Tyler Michael Smith Claude Opus 4.6
2026-03-16 23:03:54 +01:00
c0f011918d [Bugfix] opcheck false mutation error in rms_norm_per_block_quant (#36688) (#36779) Krish Gupta 2026-03-17 02:41:33 +05:30
e6ae4b1be1 [compile] Enable mega aot artifact for torch 2.12+. (#37198) Zhengxu Chen 2026-03-16 17:05:51 -04:00
2dccb38f73 [Bugfix][MultiConnector] Fix MultiConnector for SupportsHMA sub-connectors (#36549) zhanqiuhu 2026-03-16 16:51:04 -04:00
d157216093 [BUGFIX][Mamba] Use uint64 for address in KVBlockZeroer (#37197) Kunshang Ji 2026-03-17 04:39:56 +08:00
93f3c8e531 [Misc] Add float16 to CacheDType (#37199) Matthew Bonanni 2026-03-16 16:24:48 -04:00
2cc26c3a99 [CI][BugFix][MORI][AMD] Add transfer_id to kv transfer params for test (#37213) rasmith 2026-03-16 15:22:57 -05:00
dfa8852db2 [Refactor] Consolidate GPT-OSS reasoning parser tests (#36915) Flora Fengandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-03-16 15:53:07 -04:00
714c6e0eab [torch.compile][BE] Modify cudagraph callable to check for is_forward_context_set (#36288) Lucas Kabela 2026-03-16 12:42:34 -07:00
0fefd00e6c [Bugfix] Fix render server crash for quantized models on CPU-only hosts (#37215) Sage 2026-03-16 20:59:01 +02:00
f5c081d432 [PD][Nixl] Add support for hybrid SSM-FA models (#36687) Nicolò Lucchesi 2026-03-16 19:58:06 +01:00
c88ea8338b [MTP][Sparse MLA] Take advantage of native MTP support in indexer when possible (#36982) Matthew Bonanni 2026-03-16 13:51:21 -04:00
9f9ecff4cd Add simple granite4 tool parser (#36827) Max de Bayser 2026-03-16 14:49:09 -03:00
ca1954d58c [Bugfix] Disable cross-layer KV cache for MLA attention backends (#37090) haosdentandOr Ozeri 2026-03-17 01:03:10 +08:00
55e6d3d5c0 [Bugfix] Make siglip/clip compatible with transformers v5 (#37200) Raushan Turganbay 2026-03-16 17:48:18 +01:00
6682c231fa [Bugfix] Add error handling for FINISHED_ERROR in OpenAIServing (#37148) Chauncey 2026-03-17 00:27:47 +08:00
5ae685c1c8 [Bugfix] Relax TRTLLM KV cache contiguity assertion for cross-layer layout (#34158) Itay EtelisandItay Etelis 2026-03-16 17:20:51 +02:00
ce8cf9161d [Compile] Fix compile warning st256_cs in cuda_vec_utils.cuh (#36693) Wentao Ye 2026-03-16 11:12:15 -04:00
18be11fd59 [BUGFIX]fix CUDA OOM ERROR : invalid argument at cumem_allocator.cpp:119 (#35594) xjx 2026-03-16 23:10:42 +08:00
8d8855fdae [Bugfix] Add safety check and fallback for null scaling factor (#36106) Yuanheng ZhaoandHarry Mellor 2026-03-16 22:27:29 +08:00
e855d380fa [Compile] Fix compile warning in moe_permute (#36529) Wentao Ye 2026-03-16 10:16:14 -04:00
0e5a9382af [Bugfix] accept redacted thinking blocks in Anthropic messages (#36992) Benjamin BartelsandBenjamin Bartels 2026-03-16 14:01:57 +00:00
04bf5a35fa [Spec Decode] Update extract_hidden_states to use deferred kv_connector clear (#37013) Fynn Schmitt-Ulms 2026-03-16 09:53:45 -04:00
43a73f853b Remove unused EVS functions in qwen3_vl.py (#37183) Tianyu Guo 2026-03-16 21:09:09 +08:00
ffbc2e5bdb Patch Mistral config (#37104) Julien Denize 2026-03-16 13:22:18 +01:00
f9e6db3034 [Models][Qwen3 ViT] Keep max_seqlen on CPU to prevent D2H sync (#37139) Lukas GeigerandIsotr0py 2026-03-16 12:11:59 +00:00
d61d2b08e9 [Build] Fix API rate limit exceeded when using VLLM_USE_PRECOMPILED=1 (#36229) elvischenvandHarry Mellor 2026-03-16 20:09:27 +08:00
f5e59ee7a6 [Performance] Add prefetch for checkpoints to OS page cache (#36012) Artem Perevedentsev 2026-03-16 13:32:02 +02:00
9b005edc48 [Docs] Make the link to hardware plugins clearer (#37174) Harry Mellor 2026-03-16 11:12:58 +00:00
bf9a185395 GLM4 tool parser: fix streaming mode (#35208) Robin NabelandChauncey 2026-03-16 11:48:52 +01:00
ad041c79db Fix text only inputs for MRoPE models with the Transformers modelling backend (#37055) Harry Mellor 2026-03-16 10:31:16 +00:00
747b068136 [Hardware] Replace memory related torch.cuda APIs (#37031) Kunshang Ji 2026-03-16 18:24:48 +08:00
122f75d939 Fix pipeline parallel with multimodal models with the Transformers modelling backend (#37057) Harry Mellor 2026-03-16 10:20:37 +00:00
d8f8a7aad2 [Misc] Sync pre-commit to 4.5.1 in workflows and docs (#36675) SoluMilkenandHarry Mellor 2026-03-16 18:03:21 +08:00
0115e957d4 [Frontend][Misc] Remove unused log in /is_sleeping (#37093) Roy Wang 2026-03-16 17:46:28 +08:00
116ed130f4 [Bugfix] Fix GDN attention crash with mixed decode/spec-decode batches (#34871) haosdent 2026-03-16 17:30:23 +08:00
8374387bd8 [FlashInfer] Revert block_size 16 + head_size 256 workaround on Blackwell (#36987) Vadim Gimpelson 2026-03-16 13:04:29 +04:00

... 83 84 85 86 87 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86