iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
20,189 Commits 3 Branches 0 Tags
754e1c3de4e35c44da76beddf33ca71081d15d3c
Commit Graph
20189 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Qiming Zhang 754e1c3de4 [CI][XPU] Skip test_fused_shared_expert.py on XPU (#53035) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
 2026-08-20 14:56:09 +08:00
Shanshan Shen d626108b18 [ROCm][Perf] Fuse DeepSeek-V4 mHC post/pre and RMSNorm with AITER (#52737) 
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-08-20 04:51:04 +00:00
akii96 cd5035379c [ROCm] [Bugfix] Preserve CPU query offsets during capture (#51585) 
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
 2026-08-20 12:33:37 +08:00
Amit MoryossefClaude Fable 5Isotr0py
 		9b2aef5124 [Bugfix] Video loading: sample over presentable frames, not header sample count (MP4 edit-list trims) (#48608) 
Signed-off-by: AmitMY <amitmoryossef@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-20 04:23:09 +00:00
Xin Yang 11baa0ebd8 [Attention] Avoid redundant mask compute in GDN metadata build (#52078) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-08-20 11:52:42 +08:00
Andreas Karatzas 4f66bc3e0d [CI][ROCm] Standardize AMD test job labels by device (#52976) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-19 22:33:25 -05:00
Yufeng HeandChauncey e85dd21497 fix: report stop_sequence stop_reason in Anthropic Messages API (#45807) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-08-20 03:23:43 +00:00
pmanczakandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> f85e060ec2 [Doc] Update Gaudi HPU committers (#52726) 
Signed-off-by: pmanczak <pawel.manczak@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-20 10:59:22 +08:00
92hyungjunClaudeNick Hill
 		76fb6d210a [Bugfix][Core] Reserve the KV null block when validating max_model_len (#47272) 
Signed-off-by: hyungjun oh <92hyungjun@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-20 02:49:23 +00:00
Woosuk KwonandOpenAI Codex c0233fcf01 [Model] Remove unused DeepseekV32Indexer forward (#53021) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 19:39:52 -07:00
Micah WilliamsonAndreas KaratzasAndreas Karatzas
 		fbb4c04db4 [ROCm][CI] Speed up test_rocm_aiter_qk_norm_rope_kvcache_fusion (#53004) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-19 21:20:21 -05:00
Woosuk KwonandOpenAI Codex 6a962071bd [Distributed] Enable FlashInfer all-reduce by default (#52998) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 18:17:43 -07:00
Summer YangandWoosuk Kwon 58e5ee0158 [refactor] consolidate cp attn ops (#52839) 
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-19 18:04:11 -07:00
Guanyi Chenandz-zanez bf2866f8bf [KV Connector] Add decode offloading to Mooncake Store consumers (#52466) 
Signed-off-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: Guanyi Chen <939416532@qq.com>
Co-authored-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
 2026-08-19 17:40:35 -07:00
Giancarlo Delfin d4f4d3f40f [Model Runner V2][Spec Decode] Fix draft logits cache column stride in gumbel_sample (#53017) 2026-08-19 17:39:30 -07:00
Ronen Schaffer 0a111cca2c [kv_offload] fix(metrics): rename kv_offload_tiering_block_{queries,hits} → chunk (#52812) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
 2026-08-20 00:24:07 +00:00
Sage Moore 5bf0dbd6fd [Bugfix] vLLM crashes at startup when DeepEP v2 is used with --enforce-eager wiht TRTLLM Bf16 (#51824) 2026-08-19 20:18:47 -04:00
Rohan Potdar 823ec22b78 [ROCm]: Bump triton 3.7 commit (#52819) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-08-19 17:39:08 -05:00
jcotant-inferactClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c205726108 [CI] Fix and extend PR/issue auto-labeling (#51459) 
Signed-off-by: Joe Cotant <joe@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-19 21:45:44 +00:00
vanshbhatia-amdandVansh Bhatia d591d1d511 [Bugfix] Add Kimi K3 MoE support to benchmark_moe.py (#50082) 
Signed-off-by: Vansh Bhatia <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: Vansh Bhatia <210711135+vanshbhatia-amd@users.noreply.github.com>
 2026-08-19 19:42:30 +00:00
Misha GoinandOpenAI Codex 541c6d64c1 [Bugfix][Quantization] Support CT block FP8 with Marlin (#52966) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 12:41:26 -07:00
Andreas KaratzasandOpenAI Codex cb58bb9c1e [CI] Harden RemoteVLLMServer GPU cleanup checks (#52282) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 14:17:17 -05:00
Andreas Karatzas 480d4f0d15 [ROCm][CI] Enable modular OAI Triton MoE tests (#46434) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-19 13:46:12 -05:00
Roger Wang 755492e37d Revert "[Kernel] Gemma-4 FA4 FP8 Kernel" (#52987) 2026-08-19 10:48:46 -07:00
Misha GoinandOpenAI Codex f76d71d7ce [CI/Build] Fix CPU platform pre-commit formatting (#52981) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 10:32:37 -07:00
linitra24andJee Jee Li 9b5f3454f2 [LoRA] Avoid false target matches for unsupported module types (#52313) 
Signed-off-by: linitra24 <Joy25810@foxmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-19 16:48:11 +00:00
lxyand“新颐” 0c8c3f41cf [Bugfix][V1] Sync mamba_block_size via EngineCoreReadyResponse (#50809) 
Signed-off-by: “新颐” <lxy537258@antgroup.com>
Co-authored-by: “新颐” <lxy537258@antgroup.com>
 2026-08-19 16:44:36 +00:00
Andreas KaratzasOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		3a386cfaf5 [ROCm] Give EngineCore cleanup grace after request abort (#52281) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-19 11:39:59 -05:00
EgorandClaude e9e1630e93 [Model] Support bidirectional (encoder-only) attention for DeepSeek e… (#52948) 
Signed-off-by: Egor <erop001@gmail.com>
Signed-off-by: Egor <47443236+Lossfull@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-19 16:36:27 +00:00
Yi LiuOpenAI CodexOpenAI CodexMisha Goinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		54dd98be28 [CT] Support Humming for WNA16 MoE (#48918) 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-19 12:31:32 -04:00
xuebwang-amd c676232313 [Bugfix][Quantization] Fix OCP MX MoE emulation silently skipping mxfp6 activation QDQ (#52704) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
 2026-08-19 11:29:29 -05:00
akii96 583a00257d [ROCm] [Bugfix] Fix Triton fused shared expert alignment (#51632) 
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
 2026-08-20 00:14:52 +08:00
Dinesh Chitlangia 525b7bbb3a [Bugfix][CPU] Enable C++ causal_conv1d GDN path and float32 SSM cache on non-AMX AVX-512BF16 CPUs (#49688) 
Signed-off-by: Dinesh Chitlangia <dineshchitlangia@gmail.com>
 2026-08-19 10:50:59 -05:00
Andreas Karatzas 17dbd42930 [ROCm] Add UE8M0 scale packing for Triton silu_mul_quant (#37835) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-19 10:20:05 -05:00
Andreas Karatzas 160f7f0840 [ROCm][CI] Extended Fused MoE and FP8 MoE test support (#41100) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-19 10:19:42 -05:00
Yohann PrigentandMisha Goin be06873198 [Bugfix] compressed-tensors: restore int8 grouped WNA16 MoE support (#52002) 
Signed-off-by: Yohann Prigent <prigent.yohann@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-08-19 07:55:59 -07:00
Misha GoinandOpenAI Codex 2d7f42b4f3 [Build] Add InstantTensor to CUDA dependencies (#52801) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 10:55:09 -04:00
Xiaoan (Sean) LiuandNick Hill db92053e97 [Core] Skip broadcasting mm tensor data to workers for prefix-cache-covered items (#52041) 
Signed-off-by: Xiaoan (Sean) Liu <shawnliu0327@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-19 07:16:39 -07:00
Gabriel Wu 2b7fcbf527 [Kernel] SM120: stop routing misaligned-M blockwise FP8 GEMMs to the small-M swapAB config (#52775) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
 2026-08-19 21:25:29 +08:00
EilamandJee Jee Li c2e7242ab7 [Bugfix][LoRA] Guard None group members in expand_packed_lora (partial LoRA on Qwen3.5/3.6 GatedDeltaNet) (#47640) 
Signed-off-by: Eilam C <eilamc14@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-19 13:24:26 +00:00
Quentin Gallouédec 92bdee05cb [Bugfix][Frontend] Return all choices from /inference/v1/generate when n > 1 (#52399) 
Signed-off-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co>
 2026-08-19 20:48:45 +08:00
Nave Assaf 58de6cbdc2 Add NemotronH_Omni_Reasoning_V3 as a supported Nemotron architecture (#52929) 
Signed-off-by: Nave Assaf <nassaf@nvidia.com>
 2026-08-19 12:34:50 +00:00
Harry Mellor 2f54100a59 [CI] Fix docs build (#52937) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-19 05:25:46 -07:00
QWERQWERQWE86 58302b4591 [Doc] Fix group numbering in Case 3 of hybrid_kv_cache_manager.md (#52160) 
Signed-off-by: qwerqwerqwe8688-jpg <xuuuuu2021@163.com>
 2026-08-19 11:38:34 +00:00
hy123andOpenAI Codex b160cab156 [Bugfix] Restore model info caching for package backends (#52690) 
Signed-off-by: haoyang.qian <haoyangqian@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 11:18:10 +00:00
wangxiyuan cba06764d7 [Platform] Fill in the missing backend parameter for torch.compile (#51781) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-08-19 10:31:19 +00:00
wang.yuqi eac636a7fa [Frontend] Move api_server.py out openai folder (#52131) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-19 09:43:46 +00:00
Yufeng He 340b7e4909 fix: reject string schemas that mix pattern/format with length bounds (#49996) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
 2026-08-19 17:29:32 +08:00
Qiming Zhang 93eea4f665 [XPU][CI] downgrade sentencepiece (#52904) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
 2026-08-19 09:01:12 +00:00
LuckyStep 9a9aa2b017 [Bugfix] Redact api_key in non-default args log (#52523) 
Signed-off-by: LuckyStep <67696304+Andy365-365@users.noreply.github.com>
 2026-08-19 15:52:53 +08:00