Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

4635cc3e8f Shard the K3 Latent-MoE up-projection on large batches (#50383) Jee Jee Li 2026-08-03 15:49:39 +08:00
0a6446005d [CPU] Migrate unquantized MoE to the modular-kernel experts structure (#50133)
Li, Jiang Claude Opus 5 Fadi Arafeh
2026-08-03 14:55:04 +08:00
b3f97dae24 [Frontend] Require cache_salt to be non-empty via schema (#50816) Cyrus Leung 2026-08-03 14:51:27 +08:00
e481da9508 [XPU] [Linear] add torch as xpu linear backend (#49664) zofiaandKunshang Ji 2026-08-03 14:30:21 +08:00
2755489a26 [INC] fix w4a4 model (#50807) Qiming Zhang 2026-08-02 23:28:58 -07:00
f5bb701fa2 [Bugfix][Frontend] Constrain Anthropic cache_salt to non-empty (#50764)
Omkar Shewale Andreas Karatzas Andreas Karatzas
2026-08-03 00:33:14 -05:00
5e35a6f4f9 cpu_model_runner.py: skip the warm up if CompilationMode.NONE (#50547) YAMAMOTO Takashi 2026-08-03 13:34:29 +09:00
5c4fe4b17e [Bugfix][KV Connector] Propagate EAGLE state across merged Mooncake store groups (#49069) Yifan Qiaoandxijiade.aihemaiti 2026-08-02 19:32:26 -07:00
e42c230e4f [Bugfix] serving_llama70B_tp4 benchmark was silently running at tensor_parallel_size=1 (#50766) Turner JabbourandClaude 2026-08-02 20:26:48 -06:00
9a4fd57cac [Model] Support jina-embeddings-v5-text-nano (EuroBERT encoder backbone) (#50688) Omkar Shewale 2026-08-02 20:37:08 -05:00
0033211c0b [Test][V1] Add sleep/wake correctness regression test for hybrid GDN/… (#44972) chun-wan 2026-08-03 06:02:42 +08:00
0055b8bfa3 [Attention][MiniMax-M3] Add MSA speculative decode verification (#50032) Jason Li 2026-08-02 16:46:55 -04:00
c666810676 [ROCm][Bugfix][Kimi-K3] Preserve MoE correction bias in FP32 (#50761) Fangzhou AiandOpenAI Codex 2026-08-02 16:30:54 -04:00
55c98e370a [Model Runner v2] Enable BGE M3 pooling embed token_classify (#50661) Taneem Ibrahim 2026-08-02 11:46:03 -05:00
96add737ff [Elastic EP] Fix non-contiguous weight transfers (#50641) Itay Alroy 2026-08-02 18:13:42 +03:00
0601850791 [Bugfix][Models] Accept Qwen3_5MoeTextConfig in Qwen3_5MoeProcessingInfo for transformers 5.x compatibility (#50704) Rafli Surya Wijayaandloulanyue 2026-08-02 13:50:40 +08:00
e2fa28594f [1/N] Unify multiple-path encoder cuda graph support (#49934) Isotr0pyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-08-02 10:33:23 +08:00
c67fe497a2 [Bugfix][Doc] Fix references to FusedMoE in doc (#50701) bnellnm 2026-08-01 21:13:06 -04:00
38a466e7b6 [DSV4] Implement Sequence Parallelism (#46789) Woosuk Kwon 2026-08-01 10:22:33 -07:00
dc818c198d [GPT-OSS] Strict tool call and constrained decoding for Harmony (#45560) yzong-rh 2026-08-01 12:10:26 -04:00
127a7cebc9 Add @hongxiayang as code owner for AMD-specific model files and ROCm docs (#50608) Hongxia Yang 2026-08-01 12:02:44 -04:00
03c782eb91 [model registry] some simple typos (#50673) Ning Xie 2026-08-01 23:37:29 +08:00
39f55ffdaa [Core] Offload raw-prompt preprocessing to renderer thread pool in AsyncLLM (#49608) Almog Tavor 2026-08-01 18:10:51 +03:00
63e78ce365 [Benchmark] Add probe requests to vllm bench serve (#49611) Guan-Ming Chiu 2026-08-01 23:10:56 +09:00
ab06486175 [Bugfix][Kernel] Fix dangling temporary in AWQ gemm torch::stable::sum dim arg (#46805) xiaolinchen 2026-08-01 21:56:55 +08:00
652ba59229 [Model Runner V2] Enable encoder token embedding (#50574) Taneem Ibrahim 2026-08-01 06:55:36 -05:00
3986b967b2 (feat): optionally disable lookup on PD decode (#50498) Jeff (Junze) MaandYifan Qiao 2026-08-01 02:41:43 -07:00
81a42d37b1 [Frontend] Add cache_salt support to Anthropic Messages API (#49498) Cao Qian 2026-08-01 00:36:42 -07:00
77469c9057 [ROCm][MLA] Mask the AITER MLA small-head verify flatten causally (#50476) Matt Qinandyudigege86 2026-08-01 02:17:17 -05:00
4ee9702bee Add Understanding the Latency Metrics docs (#50600) Michael Goin 2026-08-01 02:22:40 -04:00
9c110fa522 [Frontend] Cohere chat v2 api support (#47189) andrewbcohereandCyrus Leung 2026-08-01 01:53:54 -04:00
6c91de3689 [Bugfix][Parser] Forward model_config to nested reasoning parsers (#50642) Chauncey 2026-08-01 13:07:34 +08:00
124154a884 Add @shen-shanshan to CODEOWNERS (#50655) Shanshan Shen 2026-08-01 12:34:36 +08:00
f7097a92fd [Bugfix][CI] Prevent common ops imports from initializing CUDA (#50639) Andreas KaratzasandOpenAI Codex 2026-07-31 23:13:29 -05:00
62195e9784 [Rust][Benchmark] Prevent invalid token IDs in random benchmarks (#50058) Reid 2026-08-01 11:20:29 +08:00
eb6453d95b [Build] Update pin to build ABI stable FA2 (#50474) Jane (Yuan) Xu 2026-07-31 23:09:36 -04:00
c4a4a42d80 [Bugfix][Test] Fix monolithic routing replay test buffer capacity (#50640) Amir Samani 2026-07-31 19:37:39 -07:00
fcdc7c2e9c [CI] Organize speculative decoding E2E tests by coverage (#50330)
Michael Goin OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-31 21:05:42 -04:00
b40d859c7b [Kernel][Helion] Add numerics checks to benchmark script (#48968)
Shangdi Yu OpenAI Codex Claude Opus 4.8
2026-07-31 17:16:02 -07:00
e3be89673d Enable ModelOpt FP8 emulation on SM80 (#50019)
Mike G Michael Goinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-31 14:48:27 -07:00
454ea5b526 [MoE Refactor] Combine CompressedTensorsWNA16MarlinMoEMethod with CompressedTensorsWNA16MoEMethod (#44570)
bnellnm Claude Taneem Ibrahim Agata Dobrzyniewicz Tushar Jain Tushar Jain Wentao Ye Jiangyun Zhu
2026-07-31 17:13:03 -04:00
0c4dc7c488 [graceful shutdown] fix http server start firstly before app signal handler register (#49668) Ning Xie 2026-08-01 04:51:33 +08:00
90329913e9 [UX] Reduce startup log noise (#50590) Michael GoinandOpenAI Codex 2026-07-31 16:04:34 -04:00
e8b358b4c0 [Bugfix][Responses] Add tests for Chat Completions Responses API Render Parity (#50334) yzong-rhandBen Browning 2026-07-31 15:45:40 -04:00
4fdd3e0b74 [ROCm][ViT] Detect Triton-AMD kernels at their new aiter location (#40289) AdrianandClaude 2026-07-31 21:30:09 +02:00
9a7ae4b808 [chore] log process manager shutdown with more details (#49437) Ning Xie 2026-08-01 02:55:09 +08:00
726ef437a1 [chore] delete useless code (#49424) Ning Xie 2026-08-01 02:54:08 +08:00
963a658674 Bump Helion to 1.4.0 (#50307) Shangdi Yu 2026-07-31 11:06:49 -07:00
aef85aed5d [Bugfix][TurboQuant] Add KV quant mode for turboquant (#50533) Soila KavulyaandClaude 2026-07-31 10:42:32 -07:00
e67a2e0a56 [ROCm][Quark][7/N] Use MXFP4 linear kernel abstraction for emulation backend (#48949) fxmarty-amd 2026-07-31 19:07:15 +02:00
d87d2ca747 [Compressed-Tensors] Support Kimi-K3 quantized models (#50500) Kyle Sayers 2026-07-31 13:06:30 -04:00
8d8a4e0f2b [ROCm][CI] Restore Mistral tool-parser compatibility after unification (#50515) Andreas Karatzas 2026-07-31 11:32:43 -05:00
7c08664f5c Upgrade tpu-inference to v0.26.0 (#50522) Dennis Yeh 2026-08-01 00:22:42 +08:00
10ad649d6c Update torch version to 2.13.0+cpu (#50412) ylangtsouandYlang Tsou 2026-07-31 23:57:16 +08:00
c036cb257f [Doc] Add BgeM3EmbeddingModel to embedding supported models (#50571) LG 2026-07-31 23:56:01 +08:00
a0cd2b69b3 [Bugfix] Universally align block table width to 128 tokens (#50302) Matthew Bonanni 2026-07-31 11:52:17 -04:00
94e9ef0768 [Bugfix] Don't transpose fused MoE quantization scales in RoutedExperts.load_weights (#50137)
Harry Mellor Claude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-31 16:46:52 +01:00
df71917cf1 [DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. (#49236) Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-07-31 11:45:58 -04:00
5233368dac [ROCm][CI] Fall back to lossless Kimi K3 MXFP4 emulation on gfx942 (#50516) Andreas Karatzas 2026-07-31 10:33:16 -05:00
82ae4164ee [2/N][Attention] Enable masked MHA for sparse MLA prefills (#48770) Matthew BonanniandOpenAI Codex 2026-07-31 09:01:24 -04:00
864a87febb [Kernel][CI] --jit-monitor-mode error e2e tests for kernel warmup infra (#50109) Nicolò Lucchesi 2026-07-31 14:53:16 +02:00
7fdc1ab29e [CI] Remove default_torch_num_threads workaround from llava-onevision-transformers test (#50560) Oğuzhan KIR 2026-07-31 14:29:31 +02:00
0b5b49dde8 [Bugfix][Frontend] Raise VLLMValidationError for user-facing errors in chat_utils.py (#50491) Recoordinateandlatent-9 2026-07-31 23:54:43 +12:00
f5ffc59b6a [Renderer] Warm up the renderer properly. (#50408) wang.yuqi 2026-07-31 19:46:53 +08:00
2c4d348848 [KV Connector] Add per-layer canonical KV page mappings for parallelism-agnostic offload (#48408)
Itay Etelis Itay Etelis Or Ozeri Itay Etelis
2026-07-31 14:42:54 +03:00
b2fb83e7ff [Attention]: Use KVCacheSpec for AttentionMetadataBuilder type hints (#50148) Martin Hickey 2026-07-31 12:20:18 +01:00
92643d68f5 K3 DSpark AR fusion (#50242) Jee Jee Li 2026-07-31 19:12:03 +08:00
17beffd55f [Misc] Clarify mono audio requirement (#50141) Nicolò Lucchesi 2026-07-31 12:46:53 +02:00
6e311c6e20 [MoE Refactor] Rename FusedMoE to FusedMoEFactory (#44941) bnellnmandHarry Mellor 2026-07-31 06:28:18 -04:00
c911120a6f [ROCm][CI] Update Transformers AR+RMS fusion expectation (#50517) Andreas Karatzas 2026-07-31 05:16:52 -05:00
0e9b50074e [chore] clean-up weight prepack for INT8 MoE (#50116) Fadi Arafeh 2026-07-31 11:15:38 +01:00
88bc8fb56f [CPU][s390x] Optimize inference perf and add oneDNN INT8 GEMM for s390x (#50219) Rehan KhanandLi, Jiang <jiang1.li@intel.com> 2026-07-31 15:19:31 +05:30
482cfc20f0 [XPU] Unify XPU RMSNorm kernels with vllm_c and drop redundant XPU-specific implementation (#46981) Chaojun ZhangandKunshang Ji 2026-07-31 17:28:18 +08:00
5d7647a109 [UT] add skipif for rocm aiter sampler UT (#50530) Qiming Zhang 2026-07-31 01:32:49 -07:00
34bb795ff3 [CI] Add M3 MSA tests to CI (#49143) Thien TranandCodex 2026-07-31 16:01:48 +08:00
f727951d3f [Bugfix] Re-land MiniMax M3 default video processor (#50305)
Taneem Ibrahim OpenAI Codex Isotr0py
2026-07-31 03:46:18 -04:00
10e6b40015 [CPU][BugFix] Remove redundant kv cache write (#50437)
Fadi Arafeh jiang1.li Claude
2026-07-31 08:28:44 +01:00
3ee2bd1321 Fix duplicate HunyuanVL image boundary tokens (#49691) Mi JiazhiandJiazhi Mi 2026-07-31 15:20:06 +08:00
0351e9aa1f [XPU][CI]Adjust source_file_dependencies for NixlConnector PD accuracy (4 GPUs) (#50373) xiangdong 2026-07-31 14:06:09 +08:00
1180b604f8 [Multimodal] Expose mm hash algothrim selection to cli args (#49686) Isotr0py 2026-07-31 13:25:19 +08:00
0f17394564 [Model Runner V2] Enable encoder token classification (#50293) Taneem Ibrahim 2026-07-31 00:15:36 -04:00
bebf918044 [Bugfix][Model] Reject encoder-backbone jina-embeddings-v5 checkpoints with a clear error (fixes #50337) (#50352) Sebastian WooandClaude 2026-07-30 20:43:31 -07:00
b49eaf205a [DSv4] Remove sparse-MLA q-head padding for FlashInfer >=0.6.14 (#48047)
Jeff (Junze) Ma Claude Opus 4.8mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Yongye Zhu
2026-07-30 20:30:21 -07:00
2773ec31b3 [ROCm][CI] Use explicit wvSplitKrc skinny-GEMM test tolerance for bf16 (gfx950) (#49309) stefankoncarevicandAndreas Karatzas 2026-07-31 05:25:42 +02:00
541128bdeb [KV Offload] Enable single-copy MLA layout for CPUOffloadingSpec (#50301) Chang GuoandCursor Agent 2026-07-30 20:20:47 -07:00
1d8be5cb44 [XPU] [BugFix] Add deepseek_v4_fp8 to xpu supported_quantization list (#50434) Xiaochang WuandKunshang Ji 2026-07-31 11:12:04 +08:00
ef0d084b4b [XPU] Fix FP8 block scale layout for MLA compatibility (#50349) Ma JianandKunshang Ji 2026-07-31 11:05:35 +08:00
6724051c57 [CI/Build][AMD] Install triton_kernels via CMake (#50328)
Ryan Rock Andreas Karatzasdepthfirst-app[bot] <184448029+depthfirst-app[bot]@users.noreply.github.com>root
2026-07-30 22:05:19 -05:00
4689c7dd61 [ROCm] Add tuned selective_state_update float16 config for AMD Instinct MI325X (#50006) vanshbhatia-amd 2026-07-30 22:43:53 -04:00
5d5f22ed7a [ROCm][CI] Use larger atol value for INT3 in test_quick_all_reduce.py (#50450) music-dinoandAndreas Karatzas 2026-07-31 04:41:44 +02:00
d91f7af7d0 [Hardware][AMD][Kernel][CI][Bugfix] Fix ROCm DeepEP FP8 max (#50467) MattandAndreas Karatzas 2026-07-30 21:37:36 -05:00
60399d48eb [CI] Retry Buildkite API rate limits (#50481) Kevin H. LuuandOpenAI Codex 2026-07-30 19:09:44 -07:00
ab98034d4c [Frontend][Bugfix] Use default tool call IDs for Kimi K3 for conversation-level uniqueness (#50420) Bugen Zhao 2026-07-31 10:02:17 +08:00
f1899b2ffd [Bugfix][ROCm] AITER MLA: size MTP verification decode metadata for real qlen/dtype (#45227)
chaeminlim-mb Edwin Lim limeward QinPR Claude Opus 4.8
2026-07-31 10:50:34 +09:00
4f1da84eb5 Enable gfx1250 ROCm architecture (#46516) +1
Juan Villamizar Douglas Lehr Claude Opus 4.7 Daniel Jaden Mathias jpvillam root Douglas Lehr dllehr Daniel Chan Andreas Karatzas
2026-07-30 20:41:12 -05:00
553fcb82d5 [CI] Retry Hugging Face processor loading (#49908) Andreas Karatzas 2026-07-30 19:48:08 -05:00
d6938b7408 [Bugfix][Rust Frontend] Select earliest-completing stop string (#50200) Samuel LaferriereandClaude 2026-07-30 20:33:19 -04:00
3333d7cb63 [ROCm]: bump AITER to 0.1.19 (#49361) Rohan PotdarandAndreas Karatzas 2026-07-30 19:15:19 -05:00
0bff0ce5d3 [Kimi K3 Bug] Fix deepgemm support for kimi k3 (#50458) Wentao Ye 2026-07-30 20:10:16 -04:00
8700f86a70 [CI] Fix tests/entrypoints/multimodal/openai/chat_completion/test_audio.py::test_chat_streaming_audio (#50451) Nicolò Lucchesi 2026-07-31 01:18:53 +02:00

... 38 39 40 41 42 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86