iHore/vLLM

Fork 0

Files

T

History

41b8c15809 [Feature][MoRIIO] Support hybrid WRITE transfers with DSpark (#60544 )

Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-10-17 04:57:58 +00:00

quantization

[Online quantization] Extend re-quantization API to MOE (showcase NVFP4 modelopt/CT/quark to any online MOE method) (#60276 )

2026-10-14 15:49:37 +08:00

speculative_decoding

[MRV2][Spec Decode] Quantized draft lm_head for drafters that share the target head (#59973 )

2026-10-15 22:06:40 +00:00

automatic_prefix_caching.md

[Core][Mamba] Remove leftovers of mamba_cache_mode="all" (#60043 )

2026-10-13 00:58:59 -07:00

batch_invariance.md

[Feature] Support batch invariance on ROCm (#52231 )

2026-10-12 10:53:26 -05:00

context_extension.md

[Examples] Resettle features examples. (#40995 )

2026-04-28 00:33:41 -07:00

cross_encoder_cache.md

[Feature][Multimodal] Add cross-encoder caching to Mooncake P2P (#56242 )

2026-09-17 13:16:41 +08:00

custom_arguments.md

Forward Anthropic vllm_xargs to sampling params (#53308 )

2026-08-21 15:23:21 -07:00

custom_logitsprocs.md

[Model Runner V2] Support custom logits processors (#56497 )

2026-09-19 17:46:49 +00:00

disagg_encoder.md

[Doc] Document ECMooncakeConnector for EPD disaggregation (#59196 )

2026-10-02 12:10:33 +08:00

disagg_prefill.md

[Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) (#58588 )

2026-10-03 19:38:25 +08:00

ec_cpu_connector.md

[Bugfix][EC Connector] Size transfers by measured encoder output widths (#57904 )

2026-10-12 12:32:39 +00:00

engram.md

[Docs] Remove references to removed env vars (#59530 )

2026-10-01 08:26:14 +00:00

index_cache.md

[Feature]: IndexCache support for DSA models (#37735 )

2026-04-29 15:15:35 -04:00

initialized_snapshots.md

[Bugfix] Avoid InfiniBand state in TP1 snapshots (#59699 )

2026-10-03 03:59:17 +00:00

interleaved_thinking.md

Allow markdownlint to run locally (#36398 )

2026-03-08 20:05:24 -07:00

kv_offloading_usage.md

[Bugfix][KV Offload] Allow explicit SimpleCPUOffload block counts (#61674 )

2026-10-17 02:11:12 +00:00

lora.md

[Bugfix][LoRA] Validate incompatible MoE adapter formats (#59924 )

2026-10-09 09:49:13 +08:00

mooncake_connector_usage.md

[Bugfix]MooncakeConnect add query timeout and retry (#55923 )

2026-10-09 09:19:33 +00:00

mooncake_ec_connector_usage.md

[Frontend] Advertise --enable-x/--disable-x for boolean CLI flags (#61479 )

2026-10-15 01:08:02 -07:00

mooncake_store_connector_usage.md

[KV Connector] Support heterogeneous TP sharing in Mooncake Store Connector (#53129 )

2026-09-01 10:18:06 -07:00

moriio_connector_usage.md

[Feature][MoRIIO] Support hybrid WRITE transfers with DSpark (#60544 )

2026-10-17 04:57:58 +00:00

multimodal_inputs.md

[Bugfix][Multimodal] Honor RGB PNG transparency when loading images (#61986 )

2026-10-16 23:09:12 +00:00

nixl_connector_compatibility.md

[KVConnector][NIXL] Support packed MLA KV layouts in pipeline-parallel push prefill (#50499 )

2026-09-28 18:50:46 +02:00

nixl_connector_usage.md

[KV Connector] Support sleep mode with MooncakeConnector over RDMA (#59625 )

2026-10-14 04:18:47 +00:00

per_request_metrics.md

[Frontend] Add per-request metrics to Responses API (#55084 )

2026-09-16 22:52:05 +00:00

preload.md

[Frontend] Advertise --enable-x/--disable-x for boolean CLI flags (#61479 )

2026-10-15 01:08:02 -07:00

prompt_embeds.md

feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API (#40720 )

2026-05-01 10:05:55 +08:00

README.md

[CI][CPU] Add speculative-decoding coverage to CPU CI (#54934 )

2026-09-14 19:51:38 +08:00

reasoning_outputs.md

[Frontend] Port Step-3.5 parsers to the streaming parser engine (#59321 )

2026-10-01 22:18:37 +00:00

sleep_mode.md

[Sleep] Retain draft-private weights with existing parameter selector (#61046 )

2026-10-11 23:30:56 +08:00

structured_outputs.md

[Docs] Reorganize online serving docs. (#41907 )

2026-05-19 14:43:18 +08:00

tool_calling.md

[Frontend] Add the Beam reasoning and tool parsers (#62052 )

2026-10-16 14:08:29 -07:00

watermarking.md

[watermarking] Add DFlash support to dual-key gumbel-max watermarking (#59554 )

2026-10-15 22:06:06 +00:00

README.md

Features

Compatibility Matrix

The tables below show mutually exclusive features and the support on some hardware.

The symbols used have the following meanings:

✅ = Full compatibility
🟠 = Partial compatibility
❌ = No compatibility
❔ = Unknown or TBD

!!! note Check the ❌ or 🟠 with links to see tracking issue for unsupported feature/hardware combination.

Feature x Feature

Feature	CP	APC	LoRA	SD	CUDA graph	pooling	enc-dec	logP	prmpt logP	async output	multi-step	mm	best-of	beam-search	prompt-embeds
CP	✅
APC	✅	✅
LoRA	✅	✅	✅
SD	✅	✅	❌	✅
CUDA graph	✅	✅	✅	✅	✅
pooling	🟠*	🟠*	✅	❌	✅	✅
enc-dec	❌	❌	❌	❌	✅	✅	✅
logP	✅	✅	✅	✅	✅	❌	✅	✅
prmpt logP	✅	✅	✅	✅	✅	❌	✅	✅	✅
async output	✅	✅	✅	❌	✅	❌	❌	✅	✅	✅
multi-step	❌	✅	❌	❌	✅	❌	❌	✅	✅	✅	✅
mm	✅	✅	🟠^{^}	❔	✅	✅	✅	✅	✅	✅	❔	✅
best-of	✅	✅	✅	❌	✅	❌	✅	✅	✅	❔	❌	✅	✅
beam-search	✅	✅	✅	❌	✅	❌	✅	✅	✅	❔	❌	❔	✅	✅
prompt-embeds	✅	✅	✅	❌	✅	❌	❌	✅	❌	❔	❔	✅	❔	❔	✅

* Chunked prefill and prefix caching are only applicable to last-token or all pooling with causal attention.
^{^} LoRA is only applicable to the language backbone of multimodal models.

Feature x Hardware

Feature	Volta	Turing	Ampere	Ada	Hopper	CPU	AMD	Intel GPU
CP	❌	✅	✅	✅	✅	✅	✅	✅
APC	❌	✅	✅	✅	✅	✅	✅	✅
LoRA	✅	✅	✅	✅	✅	✅	✅	✅
SD	✅	✅	✅	✅	✅	✅	✅	✅
CUDA graph	✅	✅	✅	✅	✅	❌	✅	❌
pooling	✅	✅	✅	✅	✅	✅	✅	✅
enc-dec	✅	✅	✅	✅	✅	✅	❌	✅
mm	✅	✅	✅	✅	✅	✅	✅	✅
prompt-embeds	✅	✅	✅	✅	✅	✅	❔	✅
logP	✅	✅	✅	✅	✅	✅	✅	✅
prmpt logP	✅	✅	✅	✅	✅	✅	✅	✅
async output	✅	✅	✅	✅	✅	❌	❌	✅
multi-step	✅	✅	✅	✅	✅	❌	✅	✅
best-of	✅	✅	✅	✅	✅	✅	✅	✅
beam-search	✅	✅	✅	✅	✅	✅	✅	✅

!!! note For information on feature support on Google TPU, please refer to the TPU-Inference Recommended Models and Features documentation.