iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,539 Commits 3 Branches 0 Tags
049507aa76200090f8bc6ca4e48ab0e09f2f0d29
Commit Graph
22539 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Artem PerevedentsevandShashank Varma 049507aa76 [Bugfix][Structured Output] Flag multi-branch allOf as unsupported for xgrammar (#59061) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Shashank Varma <324153016+shashankvarma499@users.noreply.github.com>
 2026-10-06 17:01:04 +04:00
Roy Wang 1a86733c7b [Docs] Add model recipes skill (#57786) 2026-10-06 20:40:10 +08:00
Rodrigo Garcia 3faf214051 [vllm-bench, feature] Added support for Mooncake-style, timed-traces replay. (#55937) 2026-10-06 20:39:35 +08:00
Sherif Waly 25385102fe [Bugfix][KV Offload] Honor speculative cacheability in SimpleCPU (#60071) 
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai>
 2026-10-06 14:25:08 +02:00
Wentao Ye 3e182185aa [Deprecation] Deprecate sonet dataset as scheduled (#60129) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-06 08:15:12 -04:00
Mia Ojeda 3313825983 [Bugfix][DiffusionGemma] Read the causal mask as int32 and compile the sample step (#59992) 
Signed-off-by: ojeda-e <eojeda@league.com>
 2026-10-06 19:25:51 +08:00
Shimi BandielClaudeCyrus Leung
 		09e0ce3d82 [Bugfix][Frontend] Apply model-default reasoning parser in GPU-less render server (#54835) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-06 18:46:24 +08:00
zhangchengzhucufe-dev 68088ed392 Fix grammar in BlockPool.reset_prefix_cache docstring: "to invalid" → "to invalidate" (#59886) 
Signed-off-by: zhangchengzhucufe-dev <zhangchengzhucufe@gmail.com>
 2026-10-06 10:19:54 +00:00
Louie Tsai 888074b42d [CPU][Recipes] Detect model head constraints for automatic tensor parallel selection in vLLM Recipes Tool (#60170) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-10-06 18:15:40 +08:00
Bugen ZhaoandClaude Opus 5.5 6bbad6acdf [Rust Frontend] Pass tool defer_loading through to chat templates (#60203) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-06 18:00:15 +08:00
aoshen02andClaude Opus 5.5 31e2443c90 [Model] LongCat-Flash: scale MLA norms while loading and drop the post-load sweep (#60080) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 08:47:02 +00:00
roikoren755andzjy0516 e11962fc1c [Feat][Mamba2] Enable internal prefill checkpoints (#57329) 
Signed-off-by: Roi Koren <roik@nvidia.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-10-06 07:53:51 +00:00
big_yellow_duckTan Pin SiangvllmellmCodex
 		edfccc9469 [ROCM][Perf] Add ROCM_SEGMENTED_ATTN attention-backend works across RDNA3/3.5/4 (#59132) 
Signed-off-by: root <jeffaw99@hotmail.com>
Signed-off-by: big-yellow-duck <jeffaw99@hotmail.com>
Co-authored-by: Tan Pin Siang <1716735+tanpinsiang@users.noreply.github.com>
Co-authored-by: vllmellm <190700713+vllmellm@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-06 07:53:47 +00:00
LostFox11LostFox11CodexYangShuai52mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		15c1cc12ac [Bugfix][EPLB] Use group-local ranks for torch P2P transfers (#55804) 
Signed-off-by: LostFox11 <wangziyue17@huawei.com>
Co-authored-by: LostFox11 <wangziyue17@huawei.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: YangShuai52 <178856220+YangShuai52@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-06 07:44:57 +00:00
Tarun Kumar 6ad2e06c4a fix: exclude flashinfer-jit-cache from wheel install_requires (#60052) 
Signed-off-by: Tarun Kumar <takumar@redhat.com>
 2026-10-06 06:31:13 +00:00
sungbin1015andClaude 1e5d0ea888 [CI/Build] Fix incorrect CLI arg validation tests in test_cli_args.py (#46779) 
Signed-off-by: sungbin1015 <sbin@solbox.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-06 05:58:47 +00:00
lazymioClaude Fable 5.1Shengqi Chen
 		40c86ae1ce [Kernel] Enable OAI Triton MXFP4 MoE on SM12x (#58877) 
Signed-off-by: lazymio <mio@lazym.io>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-10-06 01:49:13 -04:00
Bugen ZhaoandClaude Opus 5.5 b3082d3044 [Rust Frontend] Remove the PyO3 tool-parser bridge (#59744) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:46:27 +00:00
Bugen ZhaoandClaude Opus 5.5 5145e35e50 [Frontend] Port the MiniMax M3 tool parser to the parser engine (#59743) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:46:27 +00:00
czhu-cohereandaoshen02 6fddc9b21e [Profiler] Add per-session profiling controls to Python and Rust frontends (#57875) 
Signed-off-by: czhu <conway.zhu@cohere.com>
Signed-off-by: Conway Zhu <conway.zhu@cohere.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-06 05:21:19 +00:00
aoshen02andClaude Opus 5.5 9a1f6fd4cb [Model][DeepSeek-V4.1] Permute mega-attention wq_b / wo_a while loading (#60064) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:20:09 +00:00
Chinmay-Kulkarni-AMDandCursor Agent 533d81b4b0 [ZenCPU][Test] Add zentorch/torch version pin consistency test (#52888) 
Signed-off-by: Chinmay Kulkarni <Chinmay.Kulkarni@amd.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-10-06 13:12:19 +08:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		eb2901b735 [Perf][DSv4.1] Run decoder replay layers in CUDA graphs and trim in PIECEWISE graphs (#59532) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-05 22:11:45 -07:00
Luca MotzCodexClaude Opus 5.5Lucas Wilkinson
 		21d93d0d8c [Attention][Model] FP8 KV cache for Triton DiffKV and MiMo-V2.6-Flash (#58128) 
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-05 21:18:29 -07:00
Tianyu Guo 30ab428235 [Bugfix][Attention] Restrict FlashInfer sparse MLA FULL graphs to decode (#59751) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-10-06 04:13:23 +00:00
lzhan011andFlora Feng a3e0243b1c [Bugfix][Parser] Mistral pre-v11: don't fail requests on unexpected tool call JSON (#54844) 
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-06 03:37:30 +00:00
Tianyu GuoandOpenAI Codex 8c417a2dae [Bugfix][Model] Fix Qwen3-Omni interleaved M-RoPE boundary positions (#59842) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-10-06 03:35:53 +00:00
Tianyu Guo 48eb44b719 [Bugfix][KVConnector][NIXL] Fix TP handshake test after per-region FA desc flags (#60157) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-10-06 03:19:41 +00:00
simon-veitner-redhatClaude Opus 5.5Robert Shaw
 		6fa67afed9 [Bugfix][Core] Retire stale Mamba spec blocks in prefill checkpoint steps (#59759) 
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-06 11:01:19 +08:00
David CheungBrian WoodsChange72Claude Opus 5
 		2b32d9ac6a [Bugfix][Core] Drop stale block hashes when a session truncates tokens (#59103) 
Signed-off-by: David Cheung <d7cheung@gmail.com>
Co-authored-by: Brian Woods <me@brian.biz>
Co-authored-by: Change72 <changg@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-10-06 02:51:03 +00:00
Vadim GimpelsonClaudeThien Tran
 		b0b23a48d4 [Bugfix] Fix Qwen4Exp FP8 PLE pinned lookup kernel failing to compile below SM89 (#60142) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-06 01:56:44 +00:00
ahmed xijiaatOpenAI CodexNick HillYannik Hintereggermergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		9465e1c783 [Bugfix][NIXL] Recover locally invalidated pull peer metadata (#55471) 
Signed-off-by: xijiade.aihemaiti <3146335281@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-05 18:46:53 -07:00
lzhan011andFlora Feng 9bf4e51030 [Bugfix][Frontend] Malformed $defs in a tool schema should be a 400, not a 500 (#54850) 
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-06 01:37:20 +00:00
amd-xavierwangandTJian efd001756b [ROCm] Disable AITER attention query quantization on RDNA3 (#57750) 
Signed-off-by: xawang <xawang@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-10-06 01:37:18 +00:00
Lucas WilkinsonClaude Opus 5.5Matthew Bonanni
 		2537629eb2 [Bugfix][KVConnector][NIXL] Size per-region replicate flags by each region's block count (#60107) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-06 01:27:50 +00:00
Lucas WilkinsonClaude Opus 5.5Matthew Bonanni
 		0e8f8ecba8 [Bugfix][KVConnector][NIXL] Return NumPy descriptors from mixed-memory local registration (#60108) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-06 01:23:20 +00:00
Miłosz GrunwaldYour NameKunshang Ji
 		4e198cef83 [TEST][XPU][CI] Fix ci uva kernel test instability (#60041) 
Signed-off-by: Your Name <you@example.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-06 09:19:01 +08:00
JohnQinAMDClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		97bd8c74eb [ROCm][DSv4.1][Perf] One-pass dequantize and a gather-sized grid for the compressed K cache (#56720) 
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-05 17:57:46 -07:00
Brandon ZhangClaude Fable 5.1Wei Zhao
 		b2863c0de9 [Kimi-K3] Use the fused K/V pack kernel on the decode-context-parallel prefill path (#56881) 
Signed-off-by: Brandon Zhang <31413216+brandonfzhang@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-10-06 00:13:55 +00:00
Mai Hoàng ĐứcandFlora Feng 5cd8a2626b [Bugfix][Structured Output] Cap JSON schema nesting to prevent 500s and API-server stalls (#60036) 
Signed-off-by: Mazukiri <maihoangduc04@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-06 00:06:59 +00:00
Aarushi JainandAndreas Karatzas fa423b614c [CI][ROCm] Wait for engine teardown between LM Eval models (#60100) 
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-05 23:39:59 +00:00
MasonandYongye Zhu 6dd813e0b7 [Bugfix][MoE] Allow MiniMax2 routing in TRTLLM BF16 monolithic backend (#59571) 
Signed-off-by: Mason Pitt <bradley.b.pitt@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-10-05 16:24:14 -07:00
Vadim GimpelsonCursor Agentmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0eac152707 [Bugfix] Route every speculation-capable row through the speculative path so recurrent state stays consistent (#56531) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-06 03:03:44 +04:00
Ting SUNandFlora Feng 4a705c7b54 [Bugfix][Frontend] Preserve logprobs when top_logprobs is null (#47838) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-05 22:43:31 +00:00
Kevin H. LuuandClaude Opus 5.5 89a00995c9 [CI] Read each profiler round's trace as it stops in test_gpu_profiler (#59769) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 15:25:11 -07:00
RomeJayandFlora Feng 2e06395fd3 [Bugfix][Frontend] Build tool-call grammars from the tools the prompt renders (#59879) 
Signed-off-by: Rongjie Jiang <j1870831863@outlook.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-05 22:01:03 +00:00
Martin HickeyandFlora Feng 3fb1cd796b [Frontend] Extract parse and assembly out of batch chat derender (#60073) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-05 21:57:45 +00:00
Robert ShawClaude Opus 5.5Matthew Bonanni
 		2e3154aa18 [HiSparse] Log steady-state max concurrency and expose host-tier utilization gauges (#58949) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-05 14:20:32 -07:00
Matthew BonanniandClaude Opus 5.5 8542c3eff5 [CI] Fix test_mixed_warmup_gate after #57053 (#60116) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 21:17:00 +00:00
Yida WengandFlora Feng 18c8a65edd [BugFix][Frontend] Pass reasoning_ended through render → generate (#60059) (#60062) 
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-05 20:39:25 +00:00