iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
23,331 Commits 3 Branches 0 Tags
6a95f62e2fbfebca86ec1e476e3b2e6fd8a574ab
Commit Graph
139 Commits
Author SHA1 Message Date
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
Kevin H. LuuClaude Opus 5.5Andreas Karatzas
 		03ee2e134d [CI][ROCm] Fix order-dependent test_rocm_aiter_sampler_defers_import (CPU tensors hit Triton) (#62213) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-18 01:10:08 +00:00
Bowen WangClaude Opus 5.5Nick Hillvllm-agent
 		7b6310c109 [Perf][Sampler] Speed up the small-batch split top-p pipeline (#61996) 
Signed-off-by: Bowen Wang <bowen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-16 21:22:11 -07:00
Akshat AnandcipheraxatBowen Wang
 		ce49c47cbb [Bugfix][Sampler] Carry the boundary tie count across blocks in the split top-p mask kernel (#62046) 
Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: Bowen Wang <bowen@inferact.ai>
 2026-10-17 02:26:57 +00:00
Rohan PotdarandClaude Opus 5.5 7900d2eabc [Test] Accept tied tokens at the k-th value in the Triton top-k test (#60726) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 08:18:44 -05:00
stefankoncarevic 90f4936d9e [CI/Build] Widen the ragged prefill scoring tolerance, drop batch invariance (#60192) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-10-07 16:21:47 +08:00
Mia Ojeda 3313825983 [Bugfix][DiffusionGemma] Read the causal mask as int32 and compile the sample step (#59992) 
Signed-off-by: ojeda-e <eojeda@league.com>
 2026-10-06 19:25:51 +08:00
Raphaël RiallandandSimon Veitner c32495d58f [watermark] compatibility validation (#56801) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
 2026-10-05 14:37:13 -04:00
stefankoncarevic f2d8fbf459 [CI/Build] Run the prefill token scoring test under batch invariance (#60056) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-10-05 10:32:33 -05:00
aoshen02aoshen02Claude Opus 5.5
 		e319f86f15 [Feature] Per-row candidate IDs for prefill token scoring (M2 of #56860) (#56984) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-03 14:17:09 +00:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
TaimysCodexLuyi XiaoTaimysHarry MellorClaude Opus 5.5
 		5225f1bacb [MyPy][3/N] Fix MyPy errors in test groups (part 3) (#55939) 
Signed-off-by: Taimys <ffy2281693949@gmail.com>
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Co-authored-by: Taimys <ffy2281693949@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 11:06:07 +00:00
sashko-zakharchuk a96ee5935d [MRV2] Add DRY as a custom logits processor example (#50584) 
Signed-off-by: Oleksandr Zakharchuk <oleksandr.zakharchuk@gmail.com>
 2026-09-29 19:46:00 +00:00
aoshen02aoshen02Claude Opus 5
 		b721a4c709 [Feature] Add fixed-token prefill scoring (#54335) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-28 07:50:09 -07:00
Matt MastracciandClaude Fable 5.1 353e491efd [Perf] DiffusionGemma: one-pass sampler statistics kernel (#58226) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-25 14:14:55 -04:00
Matt MastracciClaude Fable 5.1Lucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Misha Goin
 		ab3de6edf2 [Perf] DiffusionGemma: constrained reads over the request's logprob_token_ids (#58216) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-24 13:15:08 -07:00
Wentao Ye 3c42385d7d [Refactor] Remove dead or duplicate tests (#58446) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-24 10:29:56 -04:00
Wei ZhaoCodexYongye ZhuKevin H. Luu
 		44bf011f44 [Dependency] Upgrade FlashInfer version to 0.7.0 (#58069) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-09-23 22:45:23 -07:00
Matt MastracciClaude Fable 5.1Lucas WilkinsonCodexRazorback16
 		1b3b88ec2b [Core] structured generation mode for DiffusionGemma model (Jev-like) (#57250) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Razorback16 <razorback16@protonmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Razorback16 <razorback16@protonmail.com>
 2026-09-22 20:51:17 +00:00
Martín el CheikhandClaude 2e50824766 [Bugfix] Fall back to native sampling when FlashInfer cannot target the GPU (#48956) 
Signed-off-by: martin el cheikh <martinelcheikh@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-17 12:27:54 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Roberto L. CastroLucas WilkinsonOpenAI Codex
 		aed894c190 [6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-12 17:24:24 -07:00
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> bf95f58d10 [Core] Triton kernel for small-batch top-p only masking (#54651) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-03 07:52:47 -07:00
Nick HillAnh Trananhtra3889
 		a4d70bef37 [Model Runner V2] Reserve CUDA graph memory (#53306) 
Signed-off-by: Anh Tran <anh.tran.3889@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Anh Tran <anh.tran.3889@gmail.com>
Co-authored-by: anhtra3889 <anhtra@nvidia.com>
 2026-08-23 23:11:19 -07:00
Zhixia LiuianzxliuClaudeaoshen02CodexNick Hill
 		a1c5b1fd9f [Core][V1] Support trace_decode_token_ids for deterministic decode replay (#46701) 
Signed-off-by: ianzxliu <ianzxliu@tencent.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Zhixia Liu <liuzhixia.nju@gmail.com>
Co-authored-by: ianzxliu <ianzxliu@tencent.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-20 15:12:43 +08:00
92hyungjunClaudeNick Hill
 		76fb6d210a [Bugfix][Core] Reserve the KV null block when validating max_model_len (#47272) 
Signed-off-by: hyungjun oh <92hyungjun@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-20 02:49:23 +00:00
Henry Su c94cdd0ae0 [Bugfix][Sampling] Clear empty side on thinking-budget asymmetric SWAP (#49613) 
Signed-off-by: Henry Su <henrysu4707@gmail.com>
 2026-08-15 21:13:02 +00:00
Andreas KaratzasandOpenAI Codex 1ab2801dde [ROCm] Remove stale SDPA and skinny GEMM workarounds (#50907) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 14:19:16 -04:00
Qiming Zhang 5d7647a109 [UT] add skipif for rocm aiter sampler UT (#50530) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
 2026-07-31 16:32:49 +08:00
Michael GoinandOpenAI Codex d6247d7173 [Spec Decode][Perf] Replicate DSpark Markov head across TP ranks (#49731) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-29 11:43:51 -04:00
Zach Zhu 58f9659397 [Frontend][Core] Standardize request error handling with VLLMError hierarchy (#49665) 
Signed-off-by: Zach Zhu <zzqshu@126.com>
 2026-07-29 04:20:40 +00:00
vllm-agentandvllm-agent 1134545b6f Revert "[Sampler] Stop upcasting logits to fp32 in apply_sampling_params" (#48641) (#49033) 
Co-authored-by: vllm-agent <vllm-agent@users.noreply.github.com>
 2026-07-21 09:36:45 +01:00
Michael Goinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick Hill
 		425c4eafb0 [Sampler] Stop upcasting logits to fp32 in apply_sampling_params (#48641) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-17 18:15:20 -07:00
aoshen02Wojciech WaisFederico Kamelhar
 		41ea2dd44a [Bugfix][V1/V2] Fix prompt_logprobs to respect logprobs_mode (#47680) 
Signed-off-by: Wojciech Wais <wojciech.wais@gmail.com>
Signed-off-by: Federico Kamelhar <209537060+fede-kamel@users.noreply.github.com>
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: Wojciech Wais <wojciech.wais@gmail.com>
Co-authored-by: Federico Kamelhar <209537060+fede-kamel@users.noreply.github.com>
 2026-07-17 21:58:59 +01:00
Turner JabbourandClaude 7aab6e2684 [ROCm][Bugfix] Enable the fp32 head_dtype torch.mm fast path on ROCm (#48688) 
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-15 08:18:22 +00:00
Karthik KothuriandClaude 4f7fffb92f [Core][LoRA] Support fp32 lm_head (head_dtype) on the LoRA path (#48525) 
Signed-off-by: Karthik Kothuri <karthikkothuri2009@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-15 09:51:09 +08:00
Micah Williamson 0f0f28b537 [Bugfix][CI] Fix test_head_dtype quant_method test on ROCm (#48654) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-07-14 18:31:36 -05:00
Karthik KothuriClaudewang.yuqi
 		107a03ba63 [Core] Support fp32 lm_head for generation models via head_dtype (RFC #48305 §3.6) (#48390) 
Signed-off-by: Karthik Kothuri <karthikkothuri2009@gmail.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-07-13 16:43:34 +08:00
Guan-Ming Chiu ed051fab54 [Bugfix] Reject sampling params unsupported by diffusion models (#45418) 
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
 2026-07-07 11:25:36 +00:00
Kunshang Ji fb42e5219e [Platform] Replace torch.cuda.mem_get_info with torch.accelerator.get_memory_info (#44825) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-06-30 14:39:52 +08:00
JessieWeiClaude Opus 4.8Robert Shaw
 		3c43237233 [Bugfix][Model Runner V2][Spec Decode] Fix int32 offset overflow in sampler kernels (#46560) 
Signed-off-by: xiaojun.wei <jessiewei747@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-06-24 11:00:57 -07:00
Nick Hill 7ee4d22009 [Spec Decode] Reject placeholder (-1) draft tokens in rejection sampler (#46533) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-06-24 03:32:32 +00:00
Ting SUN a46f3eb232 [Bugfix][Model Runner V2] Preserve all allowed_token_ids in the logit bias kernel (#46245) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-06-23 07:01:13 +00:00
Micah Williamson fa36f86d77 [CI] Torch 2.11 flaky test_spec_decode_logprobs and gritlm tests (#45772) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-06-23 01:26:54 +00:00
Chaojun ZhangandKunshang Ji 556b063e45 [XPU] Fix test_spec_decode_logprobs: use FLASH_ATTN for XPU in GPU_DETERMINISM_KWARGS (#44468) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-06-17 11:07:04 +08:00
Juan Pérez de Algaba 8a5cf1ccd6 [Security] Fix remote DoS via invalid recovered token reinjection (#44744) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-06-10 02:31:43 -07:00
Andreas Karatzas 82a42234be [ROCm][CI] Defer AITER sampler import and isolate server test PYTHONPATH (#44823) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-10 08:56:11 +00:00
Tianyu ZhangNick HillOpenAI Codex
 		7fe7800fa4 [BUG] Fix FP64 Gumbel precision coverage (#43150) 
Signed-off-by: tianyu-z <zhangtianyupro@gmail.com>
Signed-off-by: Tianyu Zhang <53099276+tianyu-z@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-06-05 19:04:14 +08:00
Andreas Karatzas 53b88d1dfc [CI] Reject out-of-vocabulary before they reach the GPU logprob path (#44042) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-02 22:27:52 -05:00