iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,562 Commits 3 Branches 0 Tags
e43db1f5e2eb7ca16fc284a810c548a0d3d44b14
Commit Graph
22562 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
PatrykSafferandMisha Goin e43db1f5e2 Pad to 64 not 128 Q heads in flashMLA sparse (#60029) 
Signed-off-by: Patryk Saffer <patryk.saffer@mistral.ai>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-06 16:03:05 -04:00
Jinzhen LinCodexmgoinClaude Opus 5.5
 		d1f3d8b870 [Quantization] Prefer Humming before Marlin backends on SM90 (#56997) 
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 12:31:35 -07:00
Roger WangandKimi Code bb87d227d4 [Bugfix][Model] Fix mypy safe-super error in EmbeddingGemma2Model (#60280) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-10-06 15:10:00 -04:00
Cameron QuiliciCam QuiliciNick HillYifan Qiao
 		e37e51dd24 [Metrics] Expose cached prompt tokens by cache tier (#56318) 
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
Signed-off-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-10-06 14:52:05 -04:00
Nishant RevurandClaude ad0f67a7ec [Bugfix][Spec Decode] Keep heterogeneous-vocab draft models on Model Runner V1 (#59541) 
Signed-off-by: Nishant Revur <kumrfni@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-06 18:03:44 +00:00
Madhav JivrajaniNickLuccheWei Zhao
 		3403e0f176 [Bugfix] Persist FlashInfer autotune cache per rank to fix the rank-0-only cache-hit deadlock (#57635) 
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-10-06 17:51:01 +00:00
Luciano MartinsandLuciano Martins 02b83919aa [Model] Support EmbeddingGemma2 multimodal pooling architecture (#60254) 
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
 2026-10-06 13:43:38 -04:00
Yin LiYin LiClaude
 		7b665f7a58 [Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) (#58181) 
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-06 17:22:47 +00:00
Matthew BonanniandClaude Opus 5.5 e82b80099a [Perf][HiSparse] Cache per-request residency instead of rescanning every page each step (#60083) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 13:04:55 -04:00
Wentao Ye fba31f31a9 [GLM5.3 Perf] Switch to fp8 kv cache by default for GLM, 2.3%~5.5% E2E Throughput Improvement (#60140) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-06 12:54:20 -04:00
fxmarty-amdvllmellmTan Pin SiangMisha Goin
 		f8d92cae41 [Quantization] Layer re-quantization for linear layers through online quantization API (MXFP8 -> FP8 PTPC showcase) (#55684) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-06 11:38:18 -05:00
eky-amdandClaude Fable 5 29f955cce0 [ROCm] Fuse MLA dual RMSNorm + FP8 group quant for DeepSeek-R1 (#54857) 
Signed-off-by: Ethan Ky <ethan.ky@amd.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-10-06 11:26:38 -05:00
stefankoncarevic 4516896626 [CI/Build] Warm the DP engines before measuring load balance in test_load (#60241) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-10-06 11:23:37 -05:00
Giancarlo Delfin fcf53d240c [Spec Decode] Remove eager metadata rebuild during MTP fused multi-step decode (#58463) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-10-06 12:17:30 -04:00
SeongJun Lee 4ea0c28bc4 [Bugfix] Warn only on per-request do_normalize/do_rescale overrides (#60240) 
Signed-off-by: lesj0610 <lesj0610@gmail.com>
 2026-10-06 15:55:27 +00:00
Jeff-Tseng-dev aad4579bbe Upgrade tpu-inference to v0.31.0 (#60190) 2026-10-06 15:44:56 +00:00
Chris LeonardandShengqi Chen cc73cca9c8 [Build] Migrate vendored DeepGEMM from pybind to TORCH_LIBRARY (abi3) (#48962) 
Signed-off-by: Chris Leonard <chleonar@redhat.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-10-06 08:39:04 -07:00
ovidiusm cbe1f9740d [NIXL] Bump NIXL version to 1.5.0 (#56907) 
Signed-off-by: Ovidiu Mara <ovidium@nvidia.com>
 2026-10-06 16:55:03 +02:00
Bugen ZhaoandClaude Opus 5.5 0719975cee [CI][Rust Frontend] Pass --locked to cargo binstall (#60182) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 22:53:17 +08:00
Sunny RangnaniClaudeJuan Pérez de Algaba
 		8bd7737a03 [Docs] Clarify that vLLM does not isolate tenants sharing a server (#59811) 
Signed-off-by: SunnyR <sunny.rangnani@chainguard.dev>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Juan Pérez de Algaba <124347725+jperezdealgaba@users.noreply.github.com>
 2026-10-06 22:39:59 +08:00
Samuel NordmannandMisha Goin 4a99ca40d6 [Perf][MoE] Eliminate staging from NCCL symmetric reduce-scatter (#49194) 
Signed-off-by: samnordmann <snordmann@nvidia.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-06 07:37:01 -07:00
Jiangyun ZhuandClaude Opus 5.5 bf049ddb8f [Docs] Run pr-checklist before opening a PR (#60235) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 14:12:18 +00:00
Matthias Gehre b267fe4881 [ROCm][Perf] W4A16: pad gfx11 weight and activation strides (#56301) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-10-06 08:22:33 -05:00
Artem PerevedentsevandShashank Varma 049507aa76 [Bugfix][Structured Output] Flag multi-branch allOf as unsupported for xgrammar (#59061) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Shashank Varma <324153016+shashankvarma499@users.noreply.github.com>
 2026-10-06 17:01:04 +04:00
Roy Wang 1a86733c7b [Docs] Add model recipes skill (#57786) 2026-10-06 20:40:10 +08:00
Rodrigo Garcia 3faf214051 [vllm-bench, feature] Added support for Mooncake-style, timed-traces replay. (#55937) 2026-10-06 20:39:35 +08:00
Sherif Waly 25385102fe [Bugfix][KV Offload] Honor speculative cacheability in SimpleCPU (#60071) 
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai>
 2026-10-06 14:25:08 +02:00
Wentao Ye 3e182185aa [Deprecation] Deprecate sonet dataset as scheduled (#60129) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-06 08:15:12 -04:00
Mia Ojeda 3313825983 [Bugfix][DiffusionGemma] Read the causal mask as int32 and compile the sample step (#59992) 
Signed-off-by: ojeda-e <eojeda@league.com>
 2026-10-06 19:25:51 +08:00
Shimi BandielClaudeCyrus Leung
 		09e0ce3d82 [Bugfix][Frontend] Apply model-default reasoning parser in GPU-less render server (#54835) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-06 18:46:24 +08:00
zhangchengzhucufe-dev 68088ed392 Fix grammar in BlockPool.reset_prefix_cache docstring: "to invalid" → "to invalidate" (#59886) 
Signed-off-by: zhangchengzhucufe-dev <zhangchengzhucufe@gmail.com>
 2026-10-06 10:19:54 +00:00
Louie Tsai 888074b42d [CPU][Recipes] Detect model head constraints for automatic tensor parallel selection in vLLM Recipes Tool (#60170) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-10-06 18:15:40 +08:00
Bugen ZhaoandClaude Opus 5.5 6bbad6acdf [Rust Frontend] Pass tool defer_loading through to chat templates (#60203) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-06 18:00:15 +08:00
aoshen02andClaude Opus 5.5 31e2443c90 [Model] LongCat-Flash: scale MLA norms while loading and drop the post-load sweep (#60080) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 08:47:02 +00:00
roikoren755andzjy0516 e11962fc1c [Feat][Mamba2] Enable internal prefill checkpoints (#57329) 
Signed-off-by: Roi Koren <roik@nvidia.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-10-06 07:53:51 +00:00
big_yellow_duckTan Pin SiangvllmellmCodex
 		edfccc9469 [ROCM][Perf] Add ROCM_SEGMENTED_ATTN attention-backend works across RDNA3/3.5/4 (#59132) 
Signed-off-by: root <jeffaw99@hotmail.com>
Signed-off-by: big-yellow-duck <jeffaw99@hotmail.com>
Co-authored-by: Tan Pin Siang <1716735+tanpinsiang@users.noreply.github.com>
Co-authored-by: vllmellm <190700713+vllmellm@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-06 07:53:47 +00:00
LostFox11LostFox11CodexYangShuai52mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		15c1cc12ac [Bugfix][EPLB] Use group-local ranks for torch P2P transfers (#55804) 
Signed-off-by: LostFox11 <wangziyue17@huawei.com>
Co-authored-by: LostFox11 <wangziyue17@huawei.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: YangShuai52 <178856220+YangShuai52@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-06 07:44:57 +00:00
Tarun Kumar 6ad2e06c4a fix: exclude flashinfer-jit-cache from wheel install_requires (#60052) 
Signed-off-by: Tarun Kumar <takumar@redhat.com>
 2026-10-06 06:31:13 +00:00
sungbin1015andClaude 1e5d0ea888 [CI/Build] Fix incorrect CLI arg validation tests in test_cli_args.py (#46779) 
Signed-off-by: sungbin1015 <sbin@solbox.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-06 05:58:47 +00:00
lazymioClaude Fable 5.1Shengqi Chen
 		40c86ae1ce [Kernel] Enable OAI Triton MXFP4 MoE on SM12x (#58877) 
Signed-off-by: lazymio <mio@lazym.io>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-10-06 01:49:13 -04:00
Bugen ZhaoandClaude Opus 5.5 b3082d3044 [Rust Frontend] Remove the PyO3 tool-parser bridge (#59744) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:46:27 +00:00
Bugen ZhaoandClaude Opus 5.5 5145e35e50 [Frontend] Port the MiniMax M3 tool parser to the parser engine (#59743) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:46:27 +00:00
czhu-cohereandaoshen02 6fddc9b21e [Profiler] Add per-session profiling controls to Python and Rust frontends (#57875) 
Signed-off-by: czhu <conway.zhu@cohere.com>
Signed-off-by: Conway Zhu <conway.zhu@cohere.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-06 05:21:19 +00:00
aoshen02andClaude Opus 5.5 9a1f6fd4cb [Model][DeepSeek-V4.1] Permute mega-attention wq_b / wo_a while loading (#60064) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:20:09 +00:00
Chinmay-Kulkarni-AMDandCursor Agent 533d81b4b0 [ZenCPU][Test] Add zentorch/torch version pin consistency test (#52888) 
Signed-off-by: Chinmay Kulkarni <Chinmay.Kulkarni@amd.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-10-06 13:12:19 +08:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		eb2901b735 [Perf][DSv4.1] Run decoder replay layers in CUDA graphs and trim in PIECEWISE graphs (#59532) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-05 22:11:45 -07:00
Luca MotzCodexClaude Opus 5.5Lucas Wilkinson
 		21d93d0d8c [Attention][Model] FP8 KV cache for Triton DiffKV and MiMo-V2.6-Flash (#58128) 
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-05 21:18:29 -07:00
Tianyu Guo 30ab428235 [Bugfix][Attention] Restrict FlashInfer sparse MLA FULL graphs to decode (#59751) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-10-06 04:13:23 +00:00
lzhan011andFlora Feng a3e0243b1c [Bugfix][Parser] Mistral pre-v11: don't fail requests on unexpected tool call JSON (#54844) 
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-06 03:37:30 +00:00
Tianyu GuoandOpenAI Codex 8c417a2dae [Bugfix][Model] Fix Qwen3-Omni interleaved M-RoPE boundary positions (#59842) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-10-06 03:35:53 +00:00