Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

276b837dc4 [ModelRunner V2][BugFix] Free all model refs on shutdown (#47483) Nick Hill 2026-07-03 03:32:48 +01:00
34bf7b45a0 [CI] intel CI: add quantization and awq case for xpu (#46456) wenjun liuandzengxian 2026-07-03 09:51:56 +08:00
4c3c64fcf7 Add Laguna XS.2.1 DFlash drafter support (#46853) adamkbaranowski 2026-07-03 03:09:27 +02:00
442ccc6098 [ROCm][CI] Adding extract hs 2gpu (#47482) Andreas Karatzas 2026-07-02 19:59:38 -05:00
6768fbc76f [ROCm][CI] Adding qwen3 dp4 eplb (#47480) Andreas Karatzas 2026-07-02 19:58:56 -05:00
407f406300 [ROCm][CI] Adding metadata (#47477) Andreas Karatzas 2026-07-02 19:45:23 -05:00
e24d1b24fe Fix Transformers modeling backend usage stats (#47472) Harry Mellor 2026-07-02 20:51:23 +01:00
d29125c085 Xqa decode kernels (#43232) Dan BlanaruandMatthew Bonanni 2026-07-02 21:32:05 +02:00
d715b3aa1e Delete PagedAttention (#47361) Michael Goin 2026-07-02 15:31:26 -04:00
258f8de91f [Bugfix][Tool Parser] poolside_v1: accept tool calls without newline after function name (#47311) Joe Rowell 2026-07-02 21:08:37 +02:00
e392bf7a68 [BugFix][MRV2] Ensure all req slots are accounted for when scheduling (#46974) Nick Hill 2026-07-02 18:19:24 +01:00
443e68cfa6 [Bugfix] Fix pooled Whisper encoder sliding-window kernel size (#47437) Nick Hill 2026-07-02 18:19:11 +01:00
320ee285c9 [Model Runner V2][Perf] Warm up GLM-5.2 DSA indexer prefill metadata kernel (#47285) Chauncey 2026-07-03 00:31:31 +08:00
ec0ffaacc8 [Rust Frontend] Improve scheduler stats logging parity (#47435) Bugen Zhao 2026-07-02 22:55:25 +08:00
178fd56094 support GLM-5.2 gate use FP32 (#47410) Yuxuan Zhang 2026-07-02 22:45:39 +08:00
a47f38f825 [Bugfix][Model Runner V2][Spec Decode] Fix int32 offset overflow in block verification kernels (#47383) Woosuk KwonandClaude Fable 5 2026-07-02 07:32:38 -07:00
3e158ae62d [ModelRunner V2] Fix Mamba2 crash on non-spec-decode (#47428) Nick Hill 2026-07-02 15:05:16 +01:00
a2f713002d [ModelRunner V2] Enable by default for all dense models (#44443) Wentao YeandNick Hill 2026-07-02 06:48:57 -04:00
de2a8fc042 [ROCm] [PyTorch] Move to stable abi since ROCm upgraded to torch 2.11 (#47128) TJian 2026-07-02 18:34:07 +08:00
84b9c2762f Update DeepGEMM tag to point to latest nv-dev branch for sm120 support (#47304) Michael Goin 2026-07-02 06:33:44 -04:00
25fcb65d51 [Rust Frontend] Use enum-backed domain types for engine outputs and structured outputs (#47283) Bugen Zhao 2026-07-02 17:41:46 +08:00
08a8a4af3f feat(rust): expose profiler control routes in Rust frontend (#46306) Pranav ThakurandBugen Zhao 2026-07-02 14:16:07 +05:30
b0b8a286dd [Model] Add LLaVA-OneVision-2 (LlavaOnevision2ForConditionalGeneration) (#44785)
chengzheng345 chengzheng345mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-02 16:41:49 +08:00
3af8789559 [Feature] Universal speculative decoding for heterogeneous vocabularies (TLI) (#38174)
Wonderful Wan_DF Benjamin Chislett Benjamin Chislett
2026-07-02 16:34:20 +08:00
8357226f4f [XPU][CI] Split test_punica_ops into separate pytest invocations for stability (#47376) Chaojun Zhang 2026-07-02 15:50:55 +08:00
2665ed704b [Bugfix][Kernel] Correct FlashInfer CUTLASS MoE tuning token bound (#46838) Hiki 2026-07-02 13:11:00 +08:00
09663abde0 [ROCm][MLA] Fuse MLA q/kv RMSNorm + FP8 per-token quant in the FP8 attention path (#44977) xaguilar-amd 2026-07-02 07:00:41 +02:00
d63c8e9444 [BugFix][Spec Decode] Compact shared topk indices buffer after first MTP draft step (#47238) Giancarlo Delfin 2026-07-01 23:38:51 -05:00
1360c42fe6 [UX] Include NVTX in cuda.txt (#47319) Jee Jee Li 2026-07-02 10:38:50 +08:00
d0a2584773 [Misc] Use functions instead of PTX for the PDL instruction (#46984) Jee Jee Liandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-07-02 10:38:35 +08:00
7fe7fa9cda [CI][Bugfix] Rerun test_engine_log_metrics_ray on Ray GCS startup timeout (#47208) peizhang56andClaude 2026-07-01 19:32:09 -07:00
2b753ad200 [Spec Decode] DSpark speculators checkpoint support (#47093) Michael Goin 2026-07-01 20:32:27 -04:00
e196268bad [Docker] Remove unused Dockerfile.nightly_torch (#47338) Andrey TalmanandAndrey Talman 2026-07-01 19:19:42 -04:00
e91f5f8439 [CI] Remove torch_nightly mirror tags (superseded by TORCH_NIGHTLY full-nightly build) (#47342) Andrey TalmanandAndrey Talman 2026-07-01 19:19:06 -04:00
fa248139a0 [MoE] Plumb gemm1_alpha/beta/clamp_limit into TRT-LLM FP8 MoE (#45723) Yongye ZhuandClaude Opus 4.8 2026-07-01 14:34:05 -07:00
d3229431f9 [DSV4] Better MXFP8 quantization kernel (#47229) Yongye Zhu 2026-07-01 14:33:51 -07:00
4787f2dd1b [Bugfix] Don't read KV cache past seq_len in triton paged attn kernels (#47305) Nick Hill 2026-07-01 20:43:00 +01:00
8cfeb84dba [ModelRunner V2] Warmup cross-attn properly in encoder-decoder case (#47308) Nick Hill 2026-07-01 20:36:48 +01:00
5fd442187c [ROCm][P/D] MoRIIO toy proxy: support JSON Content-Type for OpenAI clients. (#46482) Chaitanya Sri Krishna Lolla 2026-07-02 00:47:05 +05:30
00eb7cefa3 [Bugfix] Prevent padding placeholders from reaching embeddings (#47029) Yiliu DongandNick Hill 2026-07-02 00:26:03 +08:00
c8bdcc0116 [Bench][BugFix] Fix empty decoder prompt for Cohere ASR in throughput benchmark (#47135) Michał Ganczarenko 2026-07-01 18:42:27 +03:00
f5a8d73377 [Spec Decode] DSpark (#46995)
Benjamin Chislett Giancarlo Delfin mgoin
2026-07-01 11:30:24 -04:00
63fcce4de1 [Bugfix] Fix GraniteMoeShared weight loading broken by #41184 (#47031) Michał GanczarenkoandKunshang Ji 2026-07-01 17:39:12 +03:00
c638f9216a [Rust Frontend] Split engine core DTOs into separate modules (#47265) Bugen Zhao 2026-07-01 22:28:21 +08:00
13c49f9845 [xpu][lora]: Align LoRA implementation with Punica GPU: fix _apply_expand rank mismatch, add_inputs hardcode, and MoE EP (#45368) Chaojun Zhang 2026-07-01 22:14:04 +08:00
f1cf6b0086 [CI] Fix segfault in tracing test (#47299) Nick Hill 2026-07-01 15:00:37 +01:00
a78c15616f Migrate GPTBigCode and Starcoder2 to the Transformers modeling backend (#30966) Harry Mellor 2026-07-01 14:41:36 +01:00
5c4db60f01 docs(security): document gRPC interface as insecure for private use only (#45903)
Juan Pérez de Algaba Russell Bryant Russell Bryant
2026-07-01 14:39:57 +02:00
4e5ca89cfe [ROCm][MiniMax-M3] Cross-layer lightning-indexer top-k sharing (#47269) Fangzhou AiandClaude 2026-07-01 06:50:09 -04:00
a22e0dfc69 [Model] Remove AyaVision, MusicFlamingo (#47263) Harry Mellor 2026-07-01 11:39:33 +01:00
cc56379e28 [Model] Support Hy3 token suffix and JSON Schema array types (#47192) stevenkuang 2026-07-01 18:16:07 +08:00
024b06b0dc [Bugfix] Expose usage field in GenerateResponse for disaggregated serving (#42748) Aleksei IvashovandNickLucche 2026-07-01 13:00:19 +03:00
e7d0fcbc09 [CI] Fix various failures on main (#47197) Harry Mellor 2026-07-01 10:35:34 +01:00
aa8bb5562e [ROCm][Perf][Bugfix] DSv4 indexer: use platform FP8 dtype (fnuz) for Q-quant on gfx942 (#46730) akii96 2026-07-01 12:33:55 +03:00
fa4bec9056 [Bugfix] Fix pooled Whisper sliding-window KV sizing (#47071) Andy Lo 2026-07-01 10:33:19 +01:00
dee5da1dec [Test] Run SageMaker handler-override tests in-process via TestClient (#47250) Jyothirmai KottuandClaude 2026-07-01 02:14:00 -07:00
ed41aa270a [ROCm][DSV4] Use aiter mHC pre/post as the default ROCm path (#43950)
Fangzhou Ai Cursor Claude
2026-07-01 04:27:42 -04:00
77a9c5ae28 Weight sync refactor + move sparse nccl engine (#44353) Aaron HaoandClaude Sonnet 4.6 2026-07-01 17:25:19 +09:00
f651a8a9a4 [XPU][UT]Enable ut qk_norm_rope_fusion (#42486) Yejing LaiandKunshang Ji 2026-07-01 15:38:03 +08:00
8f82be5705 [CI/Build] Fix LoRA testing (#47242) Jee Jee Li 2026-07-01 15:36:13 +08:00
a461070d1c [Core] Make sleep-mode backend capability flags communicator-agnostic (#47243) Nils Matteson 2026-07-01 01:17:44 -06:00
4470ae84de Remove mantis (#46806) Tiezhen WANGandClaude 2026-07-01 15:13:58 +08:00
697c34b97b [Bugfix] Fix beam search candidate indexing when logprobs count varies (#47126) Chauncey 2026-07-01 15:07:06 +08:00
5b431b905c [Rust Frontend] Coerce completion max_tokens: null to default (#47166) Blas Rodriguez Irizar 2026-07-01 07:41:33 +01:00
89e99202f2 [CPU][Perf]Added tanh AOR for faster gelu activations. (#44639) almayneandLi, Jiang <jiang1.li@intel.com> 2026-07-01 07:24:40 +01:00
b446792306 [ROCm][Bugfix] Fix Triton "out of resource: shared memory" Error In One-Shot LoRA MoE (#47209) Micah Williamson 2026-07-01 01:24:36 -05:00
c3b1f9e827 [ROCm][CI] Enable LoRA TP Distributed Test Group In AMD CI (#47193) Micah Williamson 2026-07-01 01:24:32 -05:00
df802a87b7 [CPU] Remove speculative decoding stream overrides from CPUModelRunner (#47162) Jonathan Mamou 2026-07-01 09:12:49 +03:00
93d8f834dd [Core] Pluggable sleep-mode backend abstraction (RFC #34303) (#44074) Nils Matteson 2026-06-30 23:00:53 -06:00
aeb35b90f0 [Rust Frontend] Add error context in tool parser failures (#46512) Maria Guevara 2026-07-01 06:48:55 +02:00
9a08a5118e fix: skip cooperative top-K on SM120 (#47164) Gabriel Wu 2026-07-01 12:32:54 +08:00
c5200d3565 [Attention][DSA] support dcp for FLASHINFER_MLA_SPARSE (#46076)
Jiangyun Zhu Jingyi Yang Lucas Wilkinson Claude Opus 4.8
2026-07-01 12:32:20 +08:00
3c1396bab6 [Hardware][AMD][CI] Toggle test coredumps on ROCm debug agent (#47222) Matt 2026-06-30 23:30:10 -05:00
9969466a59 [Spec Decode] Support SWA + DFlash for MiMo (#46104) Benjamin Chislett 2026-06-30 23:34:47 -04:00
3406e8f83d [Bugfix][Frontend][gpt-oss] Return raw output when Harmony parser ends non-terminal (#47062) achyuthan.s 2026-07-01 05:46:01 +04:00
a264e41975 [Distributed] Default FlashInfer allreduce to mnnvl on single node (#47219) Woosuk KwonandClaude Opus 4.8 2026-06-30 18:35:56 -07:00
f098ee70c7 [GLM5] Support FlashMLA FP8 KV cache (Hopper & Blackwell) (#47090) Woosuk Kwon 2026-06-30 18:13:21 -07:00
9294dd27eb fix(reasoning): guard rfind in ernie45 streaming </response> branch (#46255) hclandFlora Feng 2026-07-01 09:01:14 +08:00
b1190d03cc [Refactor][GPT-OSS] Harmony Responses API Refactor to use HarmonyParser (#47185) yzong-rh 2026-06-30 19:23:20 -04:00
92c7fac640 [Perf] Restore zero-init of swizzled NVFP4 scale buffer to recover Blackwell decode throughput (#45739) Albert ChengandVadim Gimpelson 2026-06-30 15:56:56 -07:00
ac521f6237 [Bugfix][Structured Outputs] Reject degenerate structured_outputs that crash EngineCore (#45346) Ting SUN 2026-07-01 05:41:33 +07:00
28242824e0 [Bugfix][Frontend] Normalize constrained Harmony recipients (#45657) tarjan1andBen Browning 2026-07-01 05:33:10 +08:00
68294739d1 [Bugfix] Align OpenCV video metadata timeline (#47099) VectorPeak 2026-07-01 04:43:42 +08:00
c8d2f3cb14 [Bugfix] compressed-tensors: allow int8 grouped WNA16 MoE on Marlin (#47154) Joe RowellandRobert Shaw 2026-06-30 21:50:46 +02:00
345b28ff2f [Hardware][AMD][CI] Bump timeouts of various test groups on AMD CI (#47195) Matt 2026-06-30 14:30:53 -05:00
248d1fbb71 [Feat][1/N] CuTeDSL warmup infrastructure, FA4 MLA (#46182) Roberto L. CastroandLucas Wilkinson 2026-06-30 21:17:34 +02:00
11b26c5528 [Bugfix][Tool Parser] PoolsideV1: fix logprobs AttributeError on Responses API (#47138) Joe RowellandRobert Shaw 2026-06-30 21:14:09 +02:00
20434c472e [Feat] Improve Triton JIT diagnostics (#46621) Roberto L. Castro 2026-06-30 20:50:15 +02:00
c8f9c156a5 [ROCm][V1][MLA] Clone prefill backend state per metadata builder (#46993) Andreas Karatzas 2026-06-30 13:43:54 -05:00
953bba488d [PERF] Extend NCCL symmetric memory to AllGather and ReduceScatter (#46703) Woosuk Kwonandsnordmann 2026-06-30 11:38:18 -07:00
3a9784b82c [Feature] DP supervisor using rust frontend (#47076) Wentao Ye 2026-06-30 14:34:05 -04:00
3cecee40f3 [Model Runner V2][Spec Decode] Fix stale values in idx_mapping from CG num reqs padding (#47066) Giancarlo Delfin 2026-06-30 13:25:32 -05:00
a7732537f4 [Bugfix] Restore part of bugfix #42650 after accidental deletion in #43241 (#47039)
Nikita Shapovalov Zhanda Zhu Shang Wang Michael Goin
2026-06-30 20:07:59 +02:00
727971f1c1 Add Medusa speculative decoding e2e test (#41396)
Rishi Puri Anshika Ojha Claudegemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Stefano Castagnetta
2026-06-30 15:02:22 -03:00
25671cb520 [Parser][Bugfix] Ensure tool call or other special tokens don't leak in non-streaming tool parsing (#46875) Ben BrowningandClaude Opus 4.6 2026-06-30 13:46:53 -04:00
27d5f78b63 [CI] Move distributed small LM eval to B200 (#47048)
Lucas Wilkinson OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-30 13:34:25 -04:00
7a341fa109 [XPU] Support ZE_AFFINITY_MASK passthrough in xpu_disagg_acc_test (#47105) liuzhenwei 2026-07-01 01:06:12 +08:00
f41e8ddc97 [ROCm][CI] Move PyTorch Compilation Unit Tests to MI300(gfx942) (#47065) Charlie Fu 2026-06-30 11:32:58 -05:00
245888ff77 [Feature] Detect all2all peer fault with fault tolerance backend and prevent corrupted output (#43637)
fangyuchu Tyler Michael Smithmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-01 00:00:25 +08:00
e840f0d3f5 [Platform] Replace torch.cuda.Event with torch.Event (#47140) Kunshang JiandHarry Mellor 2026-06-30 23:39:59 +08:00

... 49 50 51 52 53 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86