iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,374 Commits 3 Branches 0 Tags
5688a4dd4a1ebb5d2d7b6fa863d20cc903f8b341
Commit Graph
8665 Commits
Author SHA1 Message Date
Yufeng HeKimi CodeIsotr0py
 		5688a4dd4a [Bugfix][GLM-5.3] Size the image encoder cache from the exact token ceiling (#59565) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Kimi Code <kimi-code@moonshot.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-02 10:43:29 +00:00
Farhad Al-AminClaude Opus 5.5Harry Mellor
 		af2a582d5c [Bugfix][Spec Decode] Qualify Transformers backend attention layer names with the model prefix (#59293) 
Signed-off-by: Farhad Al-Amin <99006654+Legend-2727@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-10-02 11:21:20 +01:00
aoshen02andClaude Opus 5.5 8e58ac22af [Feature] Offload KV-init runtime state on sleep (#59158) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 09:49:21 +00:00
Juan Pérez de Algaba c15672cfb4 [Security] Accept zero-sum DeepSeek-OCR pixel tensors (#59417) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
 2026-10-02 09:17:12 +00:00
AlecBugen ZhaoClaude Opus 5.5
 		9d2a6f52b2 [Rust Frontend] Expose gRPC port in Python vllm serve (#59659) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 08:14:37 +00:00
kliuae 0154979656 [ROCm][Perf] Kimi-K3 enable prefill checkpoints on ROCm (#58344) 
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
 2026-10-02 06:58:04 +00:00
Lucas WilkinsonClaude Opus 5.5Matthew Bonanni
 		4056c8ac1f [Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool (#59309) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-01 23:04:21 -07:00
Matthew BonanniClaude Opus 5.5Lucas Wilkinson
 		f7999d2e44 [Bugfix][HiSparse] Size the KV cache from the groups HiSparse allocates (#59450) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-02 04:11:55 +00:00
Qiu ChunshuoandLucas Wilkinson 8486f1ebfe [Perf][PCP] Shard decode requests across PCP ranks (#52162) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-01 23:50:13 -04:00
Guanxin Liaoshen524Claude Opus 5.5aoshen02
 		a810316c08 [Bugfix] Keep batch-invariance NCCL pins out of the weight-transfer group (#59500) 
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-02 11:19:28 +08:00
Martin Hickey fcfacc3499 [Bugfix][Frontend] Seed derender detokenization from the prompt (#59046) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-02 03:17:56 +00:00
Prudhvi VudaandOpenAI Codex ea106db806 [Bugfix][Bench] Clean up synthetic video files and writers (#59529) 
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-02 02:51:34 +00:00
Yoni GozlanCursorBugen ZhaoClaude Opus 5.5
 		9bd993564f [Frontend] Parse tool calls and reasoning from checkpoint response templates (#58604) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 02:33:00 +00:00
Hyunjae Wooandaoshen02 5a661487e2 [Feature] Add native ModelExpress weight transfer backend (#58399) 
Signed-off-by: Hyunjae Woo <hwoo@nvidia.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-02 02:29:35 +00:00
Flora Feng 785b990116 [Distributed] Enable custom all-reduce under VLLM_BATCH_INVARIANT (#58623) 
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-10-02 02:25:48 +00:00
Harry MellorandClaude Opus 5 e3cae8d2ac [Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor (#57387) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-10-02 02:25:18 +00:00
aoshen02andClaude Opus 5.5 866f02db8a [Feature] Release WorkspaceManager scratch on sleep (#59156) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 02:14:37 +00:00
Miłosz GrunwaldmicroslawKunshang Ji
 		4caa060053 [TEST][CI] Fix serve rlhf tests subprocess import errors (#59622) 
Signed-off-by: microslaw <you@example.com>
Co-authored-by: microslaw <you@example.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-02 09:47:47 +08:00
Zijing LiuNick HillClaude Opus 5.5
 		d58da79de7 [Metrics][P/D] Add KV-fetch stage gauges for async KV loads (#58874) 
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-02 01:24:33 +00:00
Anurag SinghAntigravitysfeng33
 		28c57456db [Bugfix][Frontend] Accept Anthropic tool_addition and tool_removal content blocks (#57693) 
Signed-off-by: Anurag Singh <dr.anuragkr@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Antigravity <antigravity@google.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-10-01 20:03:36 -04:00
Divakar VermaandClaude Sonnet 5 e6674262f1 [Bugfix][CI] Widen DBO+DP+EP GSM8K accuracy margin on ROCm (#59700) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-10-01 23:37:36 +00:00
hclandHarry Mellor e35082e7b6 [Bugfix][CLI] Include inherited field docstrings in get_attr_docs (#49821) 
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com>
Signed-off-by: hclsys <chenglunhu@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-10-01 23:35:58 +00:00
Aarushi Jain d2125ea681 [ROCm][CI] Raise the MI355 DeepSeek-R1 GSM8K startup wait to 1800s (#59666) 
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com>
 2026-10-01 22:30:13 +00:00
Flora Feng a5105dba05 [Frontend] Port Step-3.5 parsers to the streaming parser engine (#59321) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-10-01 22:18:37 +00:00
Matej SirovatkaLucas WilkinsonClaude Fable 5.1Matthew Bonanni
 		ca65eb67d9 [Perf][HiSparse] Avoid repeated prefix scans and residency updates (#57930) 
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-01 21:48:18 +00:00
Yongye ZhuandClaude Opus 5.5 7566d83bd3 [Attention][MiniMax-M3] NVFP4 KV cache on the MSA sparse attention path (#59300) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 14:39:55 -07:00
Zijing Liu ea84051819 [KVConnector][NIXL] Count completion notifications that arrive after KV expiry (#58875) 
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
 2026-10-01 21:30:58 +00:00
ai-jzai-jzFlora Feng
 		2a52715ef7 [Bugfix][Responses] Use standard reasoning content-part events (#59652) 
Signed-off-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-01 19:20:57 +00:00
Chanbin LimandChendi.Xue 47de9d4d04 [KV Connector][NIXL] Coalesce host-buffer KV copies across cache groups (#54483) 
Signed-off-by: beenpow <57864041+beenpow@users.noreply.github.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
 2026-10-01 13:10:54 -05:00
roiponyroi.ponyTaneem Ibrahim
 		e9be5323f0 [Perf] Integrate flash-maxsim Triton kernels for late-interaction scoring (#40337) 
Signed-off-by: roi.pony <roi.pony@ibm.com>
Co-authored-by: roi.pony <roi.pony@ibm.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-10-01 14:06:43 -04:00
Matthew BonanniandClaude Opus 5.5 3a69636645 [Bugfix][HiSparse] Fix a chunked-prefill preemption livelock (#59494) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 13:02:31 -04:00
fululi12Claude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		26bfdfb6bc [ROCm][Perf] Parallelise AITER MLA page-index expansion over token chunks (#57978) 
Signed-off-by: lifulu <fululi12@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 12:00:15 -05:00
Shijie Lyu 9aaef4296f [Bugfix][Frontend] Check reused prompt token ids against the vocab before streaming (#59555) 
Signed-off-by: Shijie Lyu <lshjhf@gmail.com>
 2026-10-01 09:26:55 -07:00
aniskumar-nvandCyrus Leung 83cadd65d9 [Bugfix] Tie lm_head.weight for Nemotron Parse when checkpoint omits it (#53020) 
Signed-off-by: Anish Kumar <aniskumar@nvidia.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-01 09:26:33 -07:00
ai-jzJingqiao Zhangzjy0516
 		b538d807ff [Bugfix][MRV2] Keep GDN prefill checkpoint metadata local to each cache group (#59536) 
Signed-off-by: Jingqiao Zhang <ai-jz@users.noreply.github.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jingqiao Zhang <ai-jz@users.noreply.github.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-10-01 22:59:15 +08:00
Maroon AyoubClaudeRobert ShawRobert Shaw
 		5d9214d6b2 [Bugfix][Frontend] Strip x-anthropic-billing-header billing header from /v1/chatcompletions (#59419) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-01 13:44:40 +00:00
Jiangyun ZhuClaude Opus 5.5Thomas Parnell
 		cb6458c5ce [Core] Remove deprecated mamba_cache_mode "all" (#58997) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
 2026-10-01 02:59:37 -07:00
Mark McLoughlinandOpenAI Codex 91fcec1235 [Core][Logging] Add built-in JSON formatter (#58739) 
Add `--logging-config.formatter=json` to enable the python-json-logger
formatter without having to use a complete Python logging dictConfig.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-01 09:11:46 +01:00
Ganesh RCursorDouglas Lehr
 		5addde2a0a [CPU][Zen] Pass f32 weight scales to the zentorch INT8 MoE (#59434) 
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-10-01 15:51:53 +08:00
jiangyunfan1andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> e12b735109 [Test] Make Anthropic messages test compatible with SDK 1.x via extra_body (#57780) 
Signed-off-by: jiangyunfan1 <jiangyunfan1@h-partners.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 07:38:29 +00:00
Ganesh RCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e12291d733 [CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers (#54024) 
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 15:18:25 +08:00
Wentao Ye 4d6874d137 [GLM 5.3 Perf] Skip qlnorm calculation for MHA, 4.4~7.7% E2E TTFT Improvement (#58845) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-01 15:13:57 +08:00
Luca MotzClaude Opus 5.5Vadim Gimpelson
 		87a4bf664f [Perf][GDN] Slice pure spec-decode rows instead of a host-mask gather (#58763) 
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-10-01 06:53:06 +00:00
aoshen02Codexfh3Claude Fable 5.1
 		4a9d6f9572 [Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs (#55935) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 14:43:38 +08:00
Shuolei WangandThien c9578f10ab [Qwen3.8-Flash-Next] Fuse main QK-norm/RoPE/gate and KV-cache write into the QSA pre-indexer launch (#57097) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Thien <thien@inferact.ai>
Co-authored-by: Thien <thien@inferact.ai>
 2026-10-01 14:21:06 +08:00
vx120Cuiping Chengaoshen02Claude Opus 5.5
 		70be6aae19 [Feature][Spec Decode] Support sampling mask replay for MRV2 MTP (#59359) 
Signed-off-by: Cuiping Cheng <chengcuipingswu@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Cuiping Cheng <chengcuipingswu@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 13:51:39 +08:00
Shijie Lyu 37cecc8141 [Bugfix][Tokenizer] Fix off-by-one max_token_id from vocab_size (#59491) 
Signed-off-by: Shijie Lyu <lshjhf@gmail.com>
 2026-10-01 05:43:30 +00:00
Juntian LiuandClaude Opus 5.5 4c2d277643 [Perf][DSv4.1] Faster Engram host lookups: sorted rows, inline big lookups (#59327) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-01 11:54:44 +08:00
namgyu-younandThien Tran 4ed438b79c [Perf][Qwen4Exp] Add SM100 low-latency decode GEMM plans (#59214) 
Signed-off-by: namgyu-youn <namgyu.dev@gmail.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-01 03:40:55 +00:00
Yoni Gozlan 2eaa3bc5ac [Frontend] Enrich chat_parsing streaming events (#58603) 
Give serving adapters the delimiter spans and opener captures they need,
and keep streaming after a region that fails to parse.

Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Fix mypy index types in the event-stream test.
 2026-10-01 11:13:13 +08:00