iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,521 Commits 3 Branches 0 Tags
5145e35e50ca7fa2bf381eaacdb30e234d774d1c
Commit Graph
22521 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Bugen ZhaoandClaude Opus 5.5 5145e35e50 [Frontend] Port the MiniMax M3 tool parser to the parser engine (#59743) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:46:27 +00:00
czhu-cohereandaoshen02 6fddc9b21e [Profiler] Add per-session profiling controls to Python and Rust frontends (#57875) 
Signed-off-by: czhu <conway.zhu@cohere.com>
Signed-off-by: Conway Zhu <conway.zhu@cohere.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-06 05:21:19 +00:00
aoshen02andClaude Opus 5.5 9a1f6fd4cb [Model][DeepSeek-V4.1] Permute mega-attention wq_b / wo_a while loading (#60064) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:20:09 +00:00
Chinmay-Kulkarni-AMDandCursor Agent 533d81b4b0 [ZenCPU][Test] Add zentorch/torch version pin consistency test (#52888) 
Signed-off-by: Chinmay Kulkarni <Chinmay.Kulkarni@amd.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-10-06 13:12:19 +08:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		eb2901b735 [Perf][DSv4.1] Run decoder replay layers in CUDA graphs and trim in PIECEWISE graphs (#59532) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-05 22:11:45 -07:00
Luca MotzCodexClaude Opus 5.5Lucas Wilkinson
 		21d93d0d8c [Attention][Model] FP8 KV cache for Triton DiffKV and MiMo-V2.6-Flash (#58128) 
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-05 21:18:29 -07:00
Tianyu Guo 30ab428235 [Bugfix][Attention] Restrict FlashInfer sparse MLA FULL graphs to decode (#59751) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-10-06 04:13:23 +00:00
lzhan011andFlora Feng a3e0243b1c [Bugfix][Parser] Mistral pre-v11: don't fail requests on unexpected tool call JSON (#54844) 
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-06 03:37:30 +00:00
Tianyu GuoandOpenAI Codex 8c417a2dae [Bugfix][Model] Fix Qwen3-Omni interleaved M-RoPE boundary positions (#59842) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-10-06 03:35:53 +00:00
Tianyu Guo 48eb44b719 [Bugfix][KVConnector][NIXL] Fix TP handshake test after per-region FA desc flags (#60157) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-10-06 03:19:41 +00:00
simon-veitner-redhatClaude Opus 5.5Robert Shaw
 		6fa67afed9 [Bugfix][Core] Retire stale Mamba spec blocks in prefill checkpoint steps (#59759) 
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-06 11:01:19 +08:00
David CheungBrian WoodsChange72Claude Opus 5
 		2b32d9ac6a [Bugfix][Core] Drop stale block hashes when a session truncates tokens (#59103) 
Signed-off-by: David Cheung <d7cheung@gmail.com>
Co-authored-by: Brian Woods <me@brian.biz>
Co-authored-by: Change72 <changg@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-10-06 02:51:03 +00:00
Vadim GimpelsonClaudeThien Tran
 		b0b23a48d4 [Bugfix] Fix Qwen4Exp FP8 PLE pinned lookup kernel failing to compile below SM89 (#60142) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-06 01:56:44 +00:00
ahmed xijiaatOpenAI CodexNick HillYannik Hintereggermergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		9465e1c783 [Bugfix][NIXL] Recover locally invalidated pull peer metadata (#55471) 
Signed-off-by: xijiade.aihemaiti <3146335281@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-05 18:46:53 -07:00
lzhan011andFlora Feng 9bf4e51030 [Bugfix][Frontend] Malformed $defs in a tool schema should be a 400, not a 500 (#54850) 
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-06 01:37:20 +00:00
amd-xavierwangandTJian efd001756b [ROCm] Disable AITER attention query quantization on RDNA3 (#57750) 
Signed-off-by: xawang <xawang@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-10-06 01:37:18 +00:00
Lucas WilkinsonClaude Opus 5.5Matthew Bonanni
 		2537629eb2 [Bugfix][KVConnector][NIXL] Size per-region replicate flags by each region's block count (#60107) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-06 01:27:50 +00:00
Lucas WilkinsonClaude Opus 5.5Matthew Bonanni
 		0e8f8ecba8 [Bugfix][KVConnector][NIXL] Return NumPy descriptors from mixed-memory local registration (#60108) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-06 01:23:20 +00:00
Miłosz GrunwaldYour NameKunshang Ji
 		4e198cef83 [TEST][XPU][CI] Fix ci uva kernel test instability (#60041) 
Signed-off-by: Your Name <you@example.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-06 09:19:01 +08:00
JohnQinAMDClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		97bd8c74eb [ROCm][DSv4.1][Perf] One-pass dequantize and a gather-sized grid for the compressed K cache (#56720) 
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-05 17:57:46 -07:00
Brandon ZhangClaude Fable 5.1Wei Zhao
 		b2863c0de9 [Kimi-K3] Use the fused K/V pack kernel on the decode-context-parallel prefill path (#56881) 
Signed-off-by: Brandon Zhang <31413216+brandonfzhang@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-10-06 00:13:55 +00:00
Mai Hoàng ĐứcandFlora Feng 5cd8a2626b [Bugfix][Structured Output] Cap JSON schema nesting to prevent 500s and API-server stalls (#60036) 
Signed-off-by: Mazukiri <maihoangduc04@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-06 00:06:59 +00:00
Aarushi JainandAndreas Karatzas fa423b614c [CI][ROCm] Wait for engine teardown between LM Eval models (#60100) 
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-05 23:39:59 +00:00
MasonandYongye Zhu 6dd813e0b7 [Bugfix][MoE] Allow MiniMax2 routing in TRTLLM BF16 monolithic backend (#59571) 
Signed-off-by: Mason Pitt <bradley.b.pitt@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-10-05 16:24:14 -07:00
Vadim GimpelsonCursor Agentmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0eac152707 [Bugfix] Route every speculation-capable row through the speculative path so recurrent state stays consistent (#56531) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-06 03:03:44 +04:00
Ting SUNandFlora Feng 4a705c7b54 [Bugfix][Frontend] Preserve logprobs when top_logprobs is null (#47838) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-05 22:43:31 +00:00
Kevin H. LuuandClaude Opus 5.5 89a00995c9 [CI] Read each profiler round's trace as it stops in test_gpu_profiler (#59769) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 15:25:11 -07:00
RomeJayandFlora Feng 2e06395fd3 [Bugfix][Frontend] Build tool-call grammars from the tools the prompt renders (#59879) 
Signed-off-by: Rongjie Jiang <j1870831863@outlook.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-05 22:01:03 +00:00
Martin HickeyandFlora Feng 3fb1cd796b [Frontend] Extract parse and assembly out of batch chat derender (#60073) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-05 21:57:45 +00:00
Robert ShawClaude Opus 5.5Matthew Bonanni
 		2e3154aa18 [HiSparse] Log steady-state max concurrency and expose host-tier utilization gauges (#58949) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-05 14:20:32 -07:00
Matthew BonanniandClaude Opus 5.5 8542c3eff5 [CI] Fix test_mixed_warmup_gate after #57053 (#60116) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 21:17:00 +00:00
Yida WengandFlora Feng 18c8a65edd [BugFix][Frontend] Pass reasoning_ended through render → generate (#60059) (#60062) 
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-05 20:39:25 +00:00
Wentao Ye 74c5cbcd7b [Bugfix][MRV2][Spec Decode] Honor dynamic K in autoregressive speculators, 1.2~1.3x kernel perf improvement (#57053) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-05 14:54:11 -04:00
Raphaël RiallandandSimon Veitner c32495d58f [watermark] compatibility validation (#56801) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
 2026-10-05 14:37:13 -04:00
Giancarlo Delfin 88f3c601b3 [Docs] Add reviewer area of interest for TheEpicDolphin (#60092) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-10-05 18:35:19 +00:00
aoshen02andClaude Opus 5.5 d09ff77637 [Bugfix] Give each data-parallel engine its own global RNG streams (#59788) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 17:38:18 +00:00
Evgeny SavinovCodexTomas Ruiz
 		877ddcc456 [Bugfix][Spec Decode] Reserve the bonus KV slot for fill-in DSpark (#59105) 
Signed-off-by: Evgeny Savinov <notime.sea@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
 2026-10-05 17:35:51 +00:00
akii96 182247257f [ROCm][Perf] Use a stacked hipBLASLt GEMM for the bf16x3 router (#52668) 
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
 2026-10-05 17:28:07 +00:00
Matthew BonanniandClaude Opus 5.5 54d93af9fb [Bugfix][HiSparse] Reject cudagraph_mode=FULL at startup (#59688) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 12:42:27 -04:00
Sohaib Ahmed 64cb683842 [Model] Add Nemotron 3.5 ASR transcription support (#59827) 
Signed-off-by: Sohaib-Ahmed21 <sohaibahmed1919@gmail.com>
 2026-10-05 16:28:09 +00:00
stefankoncarevicandAndreas Karatzas 87954c5b04 [CI/Build] Declare the video modality on the CohereCompass reference model (#60063) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-05 16:25:42 +00:00
Mikko TukiainenandCursor d23af12cb1 [ROCm][Perf] Reach the fused QSA pre-indexer from the AMD path (#57947) 
Signed-off-by: mjkvaak-amd <mjkvaak-amd@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-10-05 16:22:29 +00:00
bzsuniandcjackal 528772a4bf [Bugfix][Core] Retain encoder cache references for repeated multimodal inputs (#59942) 
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-10-05 15:52:46 +00:00
charlesli640 edde9d2ce4 Remove redundant dependency requirement for TPU (#59977) 
Signed-off-by: Charles Li <licharles@google.com>
 2026-10-05 08:45:37 -07:00
stefankoncarevic f2d8fbf459 [CI/Build] Run the prefill token scoring test under batch invariance (#60056) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-10-05 10:32:33 -05:00
Matthew BonanniandClaude Opus 5.5 9e931a0c6c [CI] Add CODEOWNERS for HiSparse (#60060) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 15:18:29 +00:00
Abhay JoshiAntigravityArtem Perevedentsev
 		60932a6401 [Bugfix][Structured Output] Preserve literal values in Guidance disable_additional_properties (#58709) 
Signed-off-by: Abhay Joshi <105213625+abhayjoshi201@users.noreply.github.com>
Co-authored-by: Antigravity <noreply@google.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-10-05 18:12:40 +03:00
Nicolò Lucchesi 30d4032363 [Model][GLM-5.3-Flash] Support DCP for the kpool sparse indexer (#59211) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-10-05 16:44:14 +02:00
Jee Jee Li f9c9e8ac24 [LoRA] Remove tensorizer (#60024) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-10-05 07:01:56 -07:00
Abhay ShuklaandClaude Opus 5.5 d3547f9d03 [Test] Re-enable Voxtral HF reference test on Transformers v5 (#59771) 
Signed-off-by: AbhayShuklaIIT <abhay.shukla@live.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 13:43:49 +00:00