iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
15,858 Commits 3 Branches 0 Tags
bf45e6d0a558da2b8d7b60efb07b4aa394f3b60b
Commit Graph
1169 Commits
Author SHA1 Message Date
Nikita Shapovalov 692db29cd4 [Bugfix] Fix Ray compiled-DAG SHM channel stalls by detaching zero-copy np.ndarray logprobs buffers (#35736) 
Signed-off-by: Nikita Shapovalov <nikita@poolside.ai>
 2026-04-16 23:49:29 +08:00
Nicolò Lucchesi 3daca38e22 [Misc] toy_proxy_server handle min_tokens (#39706) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-04-16 15:08:22 +00:00
Harry MellorkhluuCyrus Leungjiang1.li
 		03f8d3a548 Update to transformers v5 (#30566) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: khluu <khluu000@gmail.com>
Signed-off-by: Kevin H. Luu <khluu000@gmail.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
 2026-04-15 16:29:15 -07:00
Wentao Ye 7c636432c6 [CI Bug] fix flaky test (#39938) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-15 17:20:06 -04:00
Wentao Ye 39ac640490 [Bug] Fix batch invariant test issue, bs=1 with max_seq_num = 1 (#39320) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-15 16:28:43 -04:00
zhanqiuhu 0b790a2501 [Speculative Decoding] Add DFlash speculators config parsing (#38300) 
Signed-off-by: Zhanqiu Hu <zhu@redhat.com>
 2026-04-15 16:22:15 -04:00
zhanqiuhu 41488f2acd [Bugfix][NIXL] Fix _logical_to_kernel_block_ids conversion for non-mamba models (#39724) 
Signed-off-by: Zhanqiu Hu <zhu@redhat.com>
 2026-04-15 20:08:58 +00:00
Zhewen LiZhewen LiOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		55e1a8e103 [Mooncake] Fix mixed MLA+Eagle block-size validation (#39596) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-15 11:36:47 -07:00
Csrayz 68be0f853e [Metrics] Add request_id to FinishedRequestStats to enable correlation between metrics and requests (#39710) 
Enables external `StatLogger` plugins to correlate per-request metrics
with request-level context. Also, this is a pre-requisite for Prometheus
exemplars in #30972.

Signed-off-by: Csrayz <33659823+Csrayz@users.noreply.github.com>
 2026-04-15 11:24:17 +00:00
Or Ozeri 235e1f930a [kv_offload+HMA][3/N]: Remove block_size from KVEvents (#36644) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-04-15 11:53:19 +03:00
wliao2 3abf858443 [Test] Refactor hard coded device string in test files under compile/quantization/models/model_executor folders (#38901) 
Signed-off-by: Liao, Wei <wei.liao@intel.com>
 2026-04-15 11:02:35 +08:00
zhanqiuhu 30679319e8 [CI][KVConnector][Metrics] Update multi KV connector edge case according to prefill stats changes (#39808) 
Signed-off-by: Zhanqiu Hu <zhu@redhat.com>
 2026-04-14 18:59:15 +00:00
omerpaz95andOr Ozeri be0c855ebd [KV Offload] Unified memory layout for offloading workers (#37206) 
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-04-14 21:33:33 +03:00
Shanshan ShenandRoger Wang 80118853f4 [MM][Perf][CG] Support ViT full CUDA graph for Qwen3-VL video inference (#38061) 
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-04-14 16:49:32 +08:00
Mark McLoughlinandOr Ozeri d3af8c1831 [Core][Metrics][BugFix] Replace num_cached_tokens/num_external_computed_tokens with PrefillStats (#37460) 
Related to `Counters can only be incremented by non-negative amounts`
error with the `vllm:prompt_tokens_by_source_total` metric.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Or Ozeri <or@ozery.com>
 2026-04-14 09:00:45 +01:00
Monishvermergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nicolò Lucchesi
 		8213e8f880 Bug/test eagle dp v0 (#38938) 
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-04-13 20:50:08 +00:00
mukesh-haiClaude Sonnet 4.6Mark McLoughlin
 		5c18b961d6 [Core][Metrics] expose waiting request breakdown via labeled metric (capacity/deferred) (#38435) 
Signed-off-by: Mukesh Baphna <mukesh@hippocraticai.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
 2026-04-13 15:30:55 -04:00
zhanqiuhu 10d9872d3a [CI][Metrics] Fix local_cache_hit assertion after prompt tokens metrics updates (#39709) 
Signed-off-by: ZhanqiuHu <zhu@redhat.com>
 2026-04-13 15:16:56 +00:00
Nicolò Lucchesi 3244a2ebf2 [KVConnector][NIXL] Organize NIXL connector into its own directory (#39354) 
The number of features supported by the connector has grown substantially
and the `nixl_connector.py` file has accumulated a lot of code. Creates a separate
directory and isolates connector/scheduler code in the hope of improving clarity
and maintainability.

Further refactor of components aimed at improving clarity and simplifying code
will follow soon.

Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-04-12 13:10:50 +00:00
Mark McLoughlin 72ff142c37 [Core][Metrics] Remove vllm:prompt_tokens_recomputed metric (#38709) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-04-12 12:22:01 +03:00
Martin HickeyandOr Ozeri cc07dad789 [HMA] [KVEvent] Enable GPU-side KV events for HMA (#37688) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Or Ozeri <or@ozery.com>
 2026-04-12 10:01:02 +03:00
Yan MaandKunshang Ji 0f3ce4c74b [XPU] Fix spec-decode UTs under tests/v1/spec_decode (#38491) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-04-11 01:31:00 +00:00
zhanqiuhu e281cb721c [CI] Add MultiConnector (Nixl+Offloading) e2e edge case tests (#39343) 
Signed-off-by: ZhanqiuHu <zhu@redhat.com>
 2026-04-10 17:35:03 +00:00
Nicolò Lucchesi c9dddc144b [CI] Add Nixl+OffloadingConnector e2e integration tests (#39200) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-04-10 21:40:40 +08:00
Yan Ma ec68d53b2b Add platform manual_seed_all API (#38468) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-04-10 13:43:50 +08:00
Chauncey ecbfbb8d61 [Feature] Add auto-detection for reasoning_config when only reasoning_parser is set (#38214) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-10 01:36:26 +00:00
Lucas KabelaandLuka Govedič a8c6ee9b78 [Performance Improvement] Update batched_count_greater_than to handle batch size 1 without recompile (#38933) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-04-09 23:51:31 +08:00
wang.yuqi 66c079ae83 [Frontend][4/n] Improve pooling entrypoints | pooling. (#39153) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-09 10:09:45 +00:00
sihao_liandKunshang Ji e80e633927 [XPU] Skip VLLM_BATCH_INVARIANT for XPU in EAGLE DP test (#39164) 
Signed-off-by: sihao.li <sihao.li@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-04-09 12:45:16 +08:00
Chendi.Xue ef5a226819 [PD][HeteroArch]Fix accuracy issue with CPU_ATTN as Decoder and Flash_ATTN as prefiller (#38935) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-04-09 11:19:07 +08:00
Wentao Ye 3352bf8b03 [CI Bug] Fix pre-commit issue in main (#39347) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-08 14:10:05 -07:00
triangleXIVandmgoin 7c94ae16c6 [BugFix] --max-model-len=-1 causes over-limit requests to hang and starve the entire service (#39102) 
Signed-off-by: triangle14 <y1019026570@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-04-08 14:03:17 -07:00
Rishi PuriBenjamin ChislettFlora Feng
 		ad05edfbca tests/v1/e2e/spec_decode: assert async scheduling is used (#39206) 
Signed-off-by: Rishi Puri <riship@nvidia.com>
Signed-off-by: Rishi Puri <puririshi98@berkeley.edu>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-04-08 20:30:03 +00:00
Wentao Ye 2018137242 [Feature] Batch invariant nvfp4 linear support (#39322) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-08 16:29:13 -04:00
Or Ozeri 512c5eb455 [kv_offload+HMA][5/N]: Track group block hashes and block IDs (#37109) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-04-08 19:50:28 +03:00
haosdent 8904fc4d19 [Bugfix] Fix V1 logprobs empty strings for multi-byte UTF-8 tokens when logprobs > 0 (#34875) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-04-08 15:30:00 +00:00
Giancarlo Delfin 5daf62271d [Model Runner V2] Fuse probabilistic rejection sample kernels (#38496) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-04-07 17:37:37 -07:00
ibifrostandSimon Mo 96b5004b71 [KVConnector] Support 3FS KVConnector (#37636) 
Signed-off-by: wuchenxin <wuchenxin.wcx@alibaba-inc.com>
Signed-off-by: ibifrost <47308427+ibifrost@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2026-04-07 15:46:00 +00:00
Ronen Schaffer 7c139ab23f [KV Offload] Clean up ARC/LRU refactoring leftovers: group ARC tests and fix stale comment (#38217) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
 2026-04-07 15:14:45 +03:00
Andreas Karatzas a435e3108d [ROCm][CI] Fix test repo-root assumptions (#39053) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-04-07 13:36:21 +08:00
zhanqiuhu bfdc0a3a99 [NIXL][Mamba][3/N] Heterogeneous TP: 3-read conv state transfer (#37635) 2026-04-06 19:07:02 +02:00
Walter Beller-Morales e69a265135 [Feat][Core] safely abort requests when FSM fails to advance (#38663) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
 2026-04-06 08:00:16 -07:00
Julien Denize fef56c1855 [Mistral Grammar] Support Grammar Factory (#38150) 
Signed-off-by: juliendenize <julien.denize@mistral.ai>
 2026-04-06 10:28:51 -04:00
Greg PereiraandRobert Shaw 4dd49b06f8 [Bug] Fix Import paths for encoder_cudagraph modules (#38997) 
Signed-off-by: greg pereira <grpereir@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-04-05 19:11:58 +00:00
Aaron Batilo 9a528260ef [Bugfix][Spec Decode] Fix extract_hidden_states for VLM models (#38987) 
Signed-off-by: Aaron Batilo <abatilo@coreweave.com>
 2026-04-05 02:41:54 -07:00
Yusuf Mohammadandyusuf 46f02e00f2 [Bugfix] Fix AWQ models batch invariance issues (#38670) 
Signed-off-by: yusuf <yusuf@deeplearningmachine.mynet>
Signed-off-by: <>
Co-authored-by: yusuf <yusuf@deeplearningmachine.mynet>
 2026-04-03 14:54:15 +00:00
wliao2 32e0c0bfa2 refactor hard coded device string in test files under tests/v1 and tests/lora (#37566) 
Signed-off-by: Liao, Wei <wei.liao@intel.com>
 2026-04-03 11:21:47 +08:00
zhanqiuhu 7b743ba953 [CI] Fix: pass string cache_dtype in test_register_kv_caches (#38836) 2026-04-02 19:42:09 +00:00
wang.yuqi a9b4f07ba2 [Frontend] Re-enable running MaxSim on GPU (#38620) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-03 00:03:13 +08:00
Chauncey cbe7d18096 [Misc] Rename think_start_str/think_end_str to reasoning_start_str/reasoning_end_str (#38242) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-01 09:56:45 -07:00