iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,504 Commits 3 Branches 0 Tags
1eacb41e0faceb95b9e89021a6bc2d968dde250a
Commit Graph
21504 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
lesj0610 1eacb41e0f Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes 
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
 2026-09-17 15:01:29 +09:00
aoshen02andClaude Fable 5.1 8538017f4b [Frontend] Omit absent fields from /inference/v1/generate stream chunks (#57264) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-17 05:58:56 +00:00
Qiming Zhang e6b1a5e5e3 [XPU] Enable XPU eplb (#44987) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com>
 2026-09-17 13:19:13 +08:00
Zhou zihengTianyu Guojiangkuaixue123Codex
 		2743dc4f92 [Feature][Multimodal] Add cross-encoder caching to Mooncake P2P (#56242) 
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-17 13:16:41 +08:00
Matt 2d8a5c5741 [ROCm][Bugfix] Reduce CUDA graph divergences (#57229) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
 2026-09-17 05:12:13 +00:00
Kevin H. Luu 0bfc7a15d0 [CI] Fix ruff docstring violations in EC connector files (#57283) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-09-16 21:13:15 -07:00
Robert ShawandRobert Shaw c38f92aae7 [A2A] Allow EPLB + SharedExpert Overlap + DeepEPv2 (#57236) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-09-17 04:02:39 +00:00
MattandClaude Code 68f2c7e74f [ROCm][CI] Fix Nixl+Offloading PD edge cases on TheRock image (#57249) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
 2026-09-16 22:56:13 -05:00
Alec d2a2a82954 [Rust Frontend] Bump vllm-proto to 0.3.0 (#57233) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
 2026-09-17 03:41:55 +00:00
lesj0610 b8adc7710e Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes 
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
 2026-09-17 12:32:30 +09:00
Itay EtelisandOr Ozeri 9854b580df [Bugfix][KV Offload] Compact canonical MLA rows (#56799) 
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-09-17 06:24:06 +03:00
Nick Hill 5efbf0886a [CI] Support FORCE_COLOR env var; enable colors in CPU CI output (#56913) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-16 22:40:21 -04:00
omerpaz95 fa3622a4a1 [EC Connector] Add Metrics Collection (#54960) 
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
 2026-09-17 10:36:23 +08:00
Oxana KorzhandAndreas Karatzas 2307a1d16c [Core] Don't issue a blocking collective RPC during engine handshake (#57226) 
Signed-off-by: okorzh <okorzh@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-16 21:30:59 -05:00
lesj0610 77b8783b25 Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes 
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
 2026-09-17 11:29:18 +09:00
Chuan (Richard) LiandCursor f730a93d2b [Scheduler] Add --max-num-active-seqs to cap RUNNING admission (#56758) 
Signed-off-by: Li <chuali@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-17 10:22:12 +08:00
Jiangyun ZhuandCodex b1f9ebc0c3 [Test][Core] Add hybrid model prefix cache hit-rate coverage (#57127) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-16 19:17:45 -07:00
Andrey TalmankhluuCodex
 		062f81d8ef Use skip-redaction option for crcr-report (#56532) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-16 19:01:54 -07:00
AlecandCodex 28ec03bc6e [Bugfix][Rust Frontend] Accept appended EngineCoreOutput fields (#56533) 
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
 2026-09-17 01:54:43 +00:00
dongluwandAndreas Karatzas 19b6ff62f2 [ROCm][Bugfix] Ignore descales for unquantized AITER caches (#56726) 
Signed-off-by: donglu <donglu@cohere.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-17 01:29:50 +00:00
Robert ShawandRobert Shaw ed769b4a45 [A2A] Add DeepEPv2 to SP Supported List (#57210) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-09-16 21:29:10 -04:00
Bugen Zhao e52da038d1 [Rust Frontend] Normalize native renderer reasoning controls (#56998) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-16 18:15:05 -07:00
Rohan PotdarClaude Opus 4.8Matt
 		3e267bae70 [ROCm] triton+triton_kernels 3.8 mxfp4 MoE support (gpt-oss + DeepSeek-V4) (#55934) 
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Matt <156021403+mawong-amd@users.noreply.github.com>
 2026-09-16 20:03:34 -05:00
Divakar VermaandAndreas Karatzas 95f4925c3a [ROCm][CI] Enable AITER FP8/unquantized cases in modular-kernel sweep + fix MoRI per-tensor FP8 dispatch (#57074) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-16 19:53:06 -05:00
Thillai Chithambaram a615f53364 [Bugfix][Metrics][MFU] Size activation traffic from the model dtype (#57070) 
Signed-off-by: Thillai Chithambaram <thillaichithambaram.a@gmail.com>
 2026-09-17 00:23:28 +00:00
kyleliang-nvKyle LiangOpenAI Codex
 		c58ff535f4 [Bugfix][Spec Decode] Fix EAGLE and dense draft startup with EP (#56930) 
Signed-off-by: Kyle Liang <kyleliang-nv@users.noreply.github.com>
Co-authored-by: Kyle Liang <kyleliang-nv@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-16 17:03:36 -07:00
Aarushi JainandAndreas Karatzas 91b96a533c [ROCm][CI] Shard MI300 Multimodal Processor (#57056) 
Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-16 18:55:14 -05:00
lesj0610 6c5552776b Merge remote-tracking branch 'origin/main' into lesj/qwen4exp-bugfixes 
Both sides edited the same import block in the Qwen4Exp NVIDIA and AMD
models and nothing else collided. This branch adds
`from ..common.mtp import make_mtp_hidden_buffer`, the helper the MTP
buffer extraction calls; upstream ec4a3a5370 widened the `..config`
import to `ATTENTION_LAYER_TYPES, QSA_LAYER_TYPE, Qwen4ExpConfig` for the
layer-type constants it introduced. The two changes are independent, so
the resolution keeps both lines. Every symbol still has a use outside the
import in both files.

Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
 2026-09-17 08:16:59 +09:00
andrewor14 62f9482583 [Tests] Delete deprecate torchao tests for v1 configs (#52956) 
Signed-off-by: andrewor14 <andrewor14@gmail.com>
 2026-09-16 23:11:27 +00:00
xinnywinneandBenjamin Chislett fbf2c5e8be [Frontend] Add per-request metrics to Responses API (#55084) 
Signed-off-by: wxin <wxin@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-09-16 22:52:05 +00:00
aoshen02 0983aef8da [Frontend] Return prompt metadata from /inference/v1/generate (#53187) 2026-09-16 15:29:07 -07:00
Robert ShawandClaude 6c3d9816af [BugFix][KV Connector] Fix Deadlock with KVConnector + MTP under KV Pressure (#57104) 
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-16 22:22:07 +00:00
yzong-rhandFlora Feng 5a6ccc5892 [Bugfix] Add Responses cache_write_tokens for API compat and CC parity (#57222) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-16 21:52:34 +00:00
Yongye Zhu 929aa2a7ed [Build] Bump DeepGEMM pin to a6bbb80 (#57218) 2026-09-16 14:49:35 -07:00
vllm-agentkimi-no-na-waRobert Shaw
 		24a63b3340 [CI][Bugfix] Initialize _transfer_layer_group_ids in region_pull_worker fixture (#57211) 
Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com>
Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-09-16 21:20:28 +00:00
Flora Feng e0705731be [Refactor] Remove unused interface methods (#57194) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-16 21:00:48 +00:00
danisereb 2bdbbc8080 [BugFix] Fix is_supported of cutlass FP8 linear (selected and fails on A100) (#55884) 
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
 2026-09-16 12:31:43 -07:00
Woosuk Kwon 30b847c1fa [Perf][DSV4.1] Remove MegaMoE padding and shared padding workaround (#57204) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-09-16 19:09:30 +00:00
Shanshan Shen bc0f47cd03 [ROCm][Bugfix] Revert #56433 + #51692 to fix accuracy breakdown for DeepSeek-V4 (#57132) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-09-16 18:58:23 +00:00
cjackal 403f182c02 [MM] Keep raw pixels through dp-sharded ViT path (#56872) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-09-16 18:47:27 +00:00
jxjandHarry Mellor 6b6bf6b3c3 [Bugfix] Remove unsupported comma-separated detailed trace values (#55702) 
Signed-off-by: git-jxj <65210887+git-jxj@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-16 18:44:48 +00:00
AlecandCodex 9639cbde04 [Rust Frontend] Expose local DP size in gRPC Control metadata (#57116) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-16 11:28:08 -07:00
Nicolò Lucchesi fc8132a5e5 [Bugfix] Fix np.float64 leaking into the KV transfer metrics log (#57068) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-09-16 18:09:05 +00:00
Juntian LiuCodexIsotr0py
 		9f9e1dac26 [Bugfix][Model] Restore causal image SWA for DeepSeek V4.1 (#57152) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-16 18:09:00 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Misha Goin 1085b64425 [Benchmark] Retire stale benchmarks and consolidate RMSNorm (#57083) 
Signed-off-by: Misha Goin <mgoin64@gmail.com>
 2026-09-16 13:21:43 -04:00
Cyrus Leung 0b9e018fb9 [MM] Further cleanup _apply_hf_processor_main (#53610) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-16 10:21:00 -07:00
Rebecca LeeandCursor 6ca2b23e22 [ROCm] Expose kFp8DynamicTokenSym on AITER PTPC linears (#54248) 
Signed-off-by: Rebecca Lee <rebecca.lee@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-16 11:48:30 -05:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Nicolò Lucchesi 804e5377aa [Bugfix][HiSparse][PD] Align region-mapped pulls across logical block sizes (#57077) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-09-16 18:39:07 +02:00