16814 Commits
Author SHA1 Message Date
MOVZXandHinata 61fbe0b865 [Mini-VLLM] Remove unreferenced definitions found by static scan
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.

- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
  reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
  unused BailingLinearAttentionBackend, and the sparse flash-attn entry
  points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
  cpu_mla_extend, cpu_mla_get_scheduler_metadata,
  gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
  gptq_quantize_weights, get_and_maybe_dequant_weights,
  marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
  ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
  input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
  run_dp_sharded_vision_model, fast_topk, get_layer_index,
  is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
  selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
  split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
  is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
  create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
  vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
  neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.

Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 21:39:56 +07:00
MOVZXandHinata 5aa2ac64f8 [Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.

- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
  scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
  CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
  CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
  render routes, and vllm/config/speech_to_text.py. GenerationTask is
  generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
  Multi, Offloading and Example connectors stay; production uses the
  CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
  arch gate, declarations and op registrations. The CUDA build never
  defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
  fail with "The model ... does not exist. Valid aliases: ...".

Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 21:06:19 +07:00
MOVZXandHinata 5f3c7bd8ab [Mini-VLLM] Remove the ROCm device path, debug routes and arch-gated tests
- Delete csrc/rocm, hipify, the _rocm_C extension and the setup.py rocm
  target path; drop the rocm requirement files
- Delete the aiter, rdna3, skinny-gemm and ROCm fused-MoE kernel modules
  and retarget the linear kernel registries away from them
- Reduce platforms/rocm.py to the GCN architecture predicates that live
  code still imports, and delete the RocmPlatform class and plugin
- Drop the VLLM_SERVER_DEV_MODE debug surface: reset, collective_rpc,
  server_info and sleep routes plus the log_error_stack flag
- Delete lm-format-enforcer, lark and otel-semantic-conventions-ai pins;
  the live server process maps none of them
- Delete tests that only build architectures this fork no longer
  registers, and test directories left without any test file
- Restore the tests/v1/kv_connector mock helpers that scheduler tests
  import at module level

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 19:31:07 +07:00
MOVZXandHinata 2066d3143f [Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
  rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
  select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
  quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
  requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
  fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
  dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
  mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
  cut deleted while keeping the model tree

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 17:10:56 +07:00
MOVZXandHinata bd70a61c20 [Mini-VLLM] Remove orphan subsystems: snapshot, CPU workers, dead decoders
- Delete the snapshot package, CLI command, and its launch tests
- Delete CPU workers, cpu_communicator, and the hw_agnostic layer package
- Delete the EC CPU connector, keeping the EC base glue intact
- Delete dead composite/triton_flashinfer backends and third_party/flashmla
- Delete dead video decoders, video-bound tests, and orphan utils
- Remove the deep_gemm_mega_moe option and dead allreduce fusion pair

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 09:27:06 +07:00
MOVZXandHinata 2b544b7ba0 [Mini-VLLM] Cut dead subsystems across serving, connectors, parsers
- Remove P/D transfer connectors: nixl, nixl_chat, mooncake x2, hf3fs,
  moriio, hisparse, example and hidden-states connectors
- Reduce weight_transfer to the base API used by model loading
- Remove kv_offload tiering, simple_kv_offload, ec cpu/example,
  snapshot command, engine snapshot server
- Remove transformers model fallback, dsv4 tokenizers, dead platforms
  cpu/tpu/xpu, flashmla third_party, dsa, megamoe/flydsl, dead attn
  ops, dead layers (short_conv, fused_embed_norm, indexer_topk,
  relu2 gemma_ar, lightning_attn, minimax_rms_norm), gemm_rs_ar
- Keep MoE LoRA op (imported eagerly by punica_gpu); drop fp8 variant
- Rebuild parser hubs around Qwen3 adapters only (36 dead adapters)
- Remove dead tests with the code they covered

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 02:19:03 +07:00
MOVZXandHinata 54a396586e [Mini-VLLM] Repair test suite after SM86 cut
- Restore example connector, test helpers, and dropped patch tests
- Guard MLA metadata import in the example connector
- Point the PLE test at common.ops.ple after upstream #60021 move
- Cut tests for removed features: tools, DRY, epd proxy, STT batch,
  diffusion scheduler, INC/Quark PLE cases, mamba-2 mixer

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 01:06:30 +07:00
MOVZXandHinata 8dfaffb700 [Bugfix] Restore xpu_communicator for CUDA graph capture
parallel_state.graph_capture() imports xpu_communicator on CUDA too.
The SM86 cut removed the file and capture crashed at startup.

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 00:33:19 +07:00
MOVZXandHinata 81074e94a3 [Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 00:20:48 +07:00
MOVZXandHinata 6a95f62e2f [Bugfix] Load checkpoint KV-cache scales in the Qwen3.5 family
Checkpoints that ship calibrated k_scale/v_scale tensors (e.g.
unsloth/Qwen3.8-27B-NVFP4, which carries one pair per full-attention
layer) name them ...self_attn.{k,v}_scale, but the runtime parameters
live on the inner attention layer (...self_attn.attn.{k,v}_scale).
Qwen3_5Model.load_weights hands the iterable straight to
AutoWeightsLoader, which performs no scale-name remapping, so the
scales are silently dropped and --kv-cache-dtype fp8 runs at scale 1.0
with only a warning_once that fires even on success (the MTP draft
layer legitimately has no scales).

Route scale-suffixed names through maybe_remap_kv_scale_name first,
mirroring gemma4/gpt_oss. No behavior change for checkpoints without
scales.

Measured on RTX 5090 / Qwen3.8-27B NVFP4 / fp8 KV / MTP k=3 (GSM8K
greedy, n=300): 92.3% at scale 1.0 vs 93.7% with scales loaded
(bf16-KV control 94.5%).

Fixes #54623

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-18 17:19:34 +00:00
MOVZXandHinata f36850e389 [Mini-VLLM] Restore FlashInfer backend for FP8 KV & only probe quantization overrides that exist
FlashInfer is the only backend serving fp8_e4m3 KV cache on compute
capability 8.6; the SM86 cut removed it and the Triton fallback
refuses FP8 KV, crashing worker startup. Restore flashinfer.py and
the triton_flashinfer shim.

The SM86 cut removed the inc and deepseek_v4_fp8 loaders but
ModelConfig still probed them, crashing compressed-tensors
checkpoints with "Invalid quantization method: inc". Filter the
override probe list to the supported methods.

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-18 23:31:30 +07:00
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-18 19:19:56 +07:00
MOVZX c5ab42bc0f Merge branch 'main' into patches 2026-10-18 16:04:36 +07:00
7c9042c05a [Bugfix][MRV2] Fix speculative warmup deadlock with pipeline parallelism (#62129)
Signed-off-by: chenjr97 <chenjr97@mail2.sysu.edu.cn>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
2026-10-17 23:51:32 -07:00
b2456c622b [ROCm][CI] Make ROCm 10.0 the default image; keep ROCm 7.2 as -rocm72 (#58761)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-10-17 21:43:59 -05:00
djramicandAndreas Karatzas 9c778e9ff8 [Bugfix][EPLB] Wait for pending expert buffer copies before NIXL READs (#62081)
Signed-off-by: Djordje Ramic <djoramic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-10-17 18:18:02 -05:00
7879f3687b [NIXL] Drop unreachable producer-side notif counting in push mode (#62210)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
2026-10-17 21:24:35 +00:00
03e8300d2c [Bugfix] Enable inductor batch-invariant mode under VLLM_BATCH_INVARIANT (#61118)
Signed-off-by: Yifan Feng <evanfengff@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-10-17 14:53:06 -04:00
Afonso RibeiroandFlora Feng 5c1c903501 [Perf][Parser] Stream Qwen3 tool argument values incrementally (12 KB call: 2.1 s to 11 ms) (#62027)
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
2026-10-17 18:49:06 +00:00
Tyrone bafb263664 [Bugfix][Benchmarks] Name the real cause when the /tokenize alignment probe fails (#59888) 2026-10-17 14:47:52 -04:00
Nick HillandClaude Opus 5.5 0c5dd18e7a [MRV2] Collect KV transfer results before the drafter runs (#62041)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-17 10:27:57 -07:00
Ting SUNmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Wentao Ye
763e3f7c91 [Bugfix][Spec Decode] Disable dynamic speculative decoding for fixed-K proposers instead of crashing (#46399)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
2026-10-17 12:41:04 -04:00
c0b6f74bee [Benchmark] Add per-case kernel profiles to vllm bench forward (#61692)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
2026-10-17 08:27:11 -07:00
siyuandXun Sun c61a1e73c6 [Feature] Skip MLA weight absorption when weights are pre-processed (#57320)
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
2026-10-17 23:10:04 +08:00
837e87e1c5 [Feature][LoRA][MoE] Expert LoRA for FlashInfer TRTLLM MXFP4 experts (#61979)
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 20:17:56 +08:00
liuzhenweiandKunshang Ji 4e964b2623 [XPU] Fix Event init failure w/ external (#61743)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-10-17 19:10:39 +08:00
Vadim GimpelsonandClaude ec8b20fbf0 [Bugfix] Fix Gemma 4 encoder CUDA graph capture OOM at startup (#61365)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-10-17 11:04:49 +00:00
2b85d9cb65 [Perf][Bugfix] Prefer MNNVL for single-node FlashInfer allreduce again (#61989)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
2026-10-17 02:55:28 -07:00
026f6ab268 [Model] DeepSeek-V4/V4.1 VL: stream language-model weights instead of sorting the checkpoint (#58857)
Signed-off-by: git bisector <gitbisector@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
2026-10-17 09:37:56 +00:00
f98ae6a9fd [Bugfix][Core] Keep resumable sessions that stop while preempted (#58778)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Zong <yzong@redhat.com>
2026-10-17 09:08:14 +00:00
2067297b2f [ROCm][Perf] Use AITER tuned configs to select a4w4 gemm kernel instead of env var (#61278)
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Signed-off-by: afriedri <afriedri@amd.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-10-17 04:00:28 -05:00
Jee Jee Li e56b1603af [Kernel]Add norm fused kernel for Step-5 Preview and Muse Spark Open (#62132)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-10-17 01:53:00 -07:00
Ting SUN eb8de8d1a1 [Bugfix][OpenAI] Respect allowed_token_ids in beam search (#47630)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
2026-10-17 08:50:40 +00:00
dskwe b05ede5616 [Bugfix][Frontend] Preserve token IDs in completion derender responses (#62114)
Signed-off-by: Yuan Li <dskwelmcy@163.com>
2026-10-17 08:27:48 +00:00
61bb590877 [Bugfix][Spec Decode] Clear stale shared block-table rows (#61174)
Signed-off-by: Cheng Rui <286040359@qq.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
2026-10-17 01:25:02 -07:00
c3de0b37e2 [Core][Spec Decode] Replace EAGLE group heuristics with explicit draft-layer tags (#61455)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 08:22:33 +00:00
MOVZX e0fe5df731 Merge branch 'main' into patches 2026-10-17 08:06:56 +00:00
znzrybandWoosuk Kwon ad2688ab30 [Bugfix] Validate draft-token capacity in forward benchmark (#61809)
Signed-off-by: znzryb <znzryb34@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
2026-10-17 00:51:36 -07:00
JovanandOpenAI Codex 5be4266f5c [Bugfix][Frontend] Preserve Cohere and Anthropic streaming errors (#61947)
Signed-off-by: Kevin <jia__kuan@163.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-10-17 06:10:16 +00:00
Wei Zhao 90d6d871d6 [Attention] Add CuTe DSL backend selection for MLA context prefill (#62069)
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
2026-10-17 01:21:55 -04:00
Cy ChengandCy d74f4180cd [Bugfix][Frontend] Detokenize echoed prompts after truncation (#61876)
Signed-off-by: Cy <15118312+cycancode@users.noreply.github.com>
Co-authored-by: Cy <15118312+cycancode@users.noreply.github.com>
2026-10-17 05:20:38 +00:00
Jee Jee Li 39983a5e7a [Kernel] Add FP32 router GEMM support for Step-5 Preview and Muse Spark Open (#62126)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-10-16 22:17:55 -07:00
41b8c15809 [Feature][MoRIIO] Support hybrid WRITE transfers with DSpark (#60544)
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-17 04:57:58 +00:00
Micah WilliamsonandCursor a0e7137c4b [CI][ROCm][Bugfix] DeepSeek V4.1: don't pass expert intermediate padding to AITER (#61960)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-17 04:25:39 +00:00
7b6310c109 [Perf][Sampler] Speed up the small-batch split top-p pipeline (#61996)
Signed-off-by: Bowen Wang <bowen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
2026-10-16 21:22:11 -07:00
Fadi Arafehandvllm-agent 556ac15eb9 [Bugfix] Remove extra args from CPU's compute slot mapping kernel (#61992)
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
2026-10-17 12:11:33 +08:00
Gabriel WuandClaude Opus 5.5 dcd07967ca [CPU] Keep the hybrid block size across the engine core's config rerun (#61805)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 12:06:03 +08:00
ce49c47cbb [Bugfix][Sampler] Carry the boundary tie count across blocks in the split top-p mask kernel (#62046)
Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: Bowen Wang <bowen@inferact.ai>
2026-10-17 02:26:57 +00:00
Sherif Walyandvllm-agent ce8853d27a [Bugfix][KV Offload] Allow explicit SimpleCPUOffload block counts (#61674)
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai>
Co-authored-by: vllm-agent <claw@inferact.ai>
2026-10-17 02:11:12 +00:00
71c20f73f2 [FlashInfer] Migrate autotune warmup to flashinfer.autotune_v2 managed cache (#55671)
Signed-off-by: Yang Xu <yanxu@nvidia.com>
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Signed-off-by: YangXu <yanxu@nvidia.com>
Co-authored-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
2026-10-17 02:08:28 +00:00