Commit Graph - vLLM - iHore Software Repository

iHore/vLLM

Fork 0

6ab6ffb428 [Feat][DSV4] Fuse q pad into deepseek v4 fused kernel (#43162) Yongye Zhu 2026-05-26 11:12:54 -04:00
445ded18c1 [ROCm][CI] Extend ROCm quick reduce coverage (#40990) Andreas Karatzas 2026-05-26 08:57:13 -05:00
d565357a90 [Docs][ROCm] MoRI-IO Connector Usage Guide (#43603) Simon Danielssonandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-26 15:52:30 +02:00
a970fb5a1a Fix CuPy runtime deps and restore humming (#43530) Mohammad Miadh Angkad 2026-05-26 20:59:40 +08:00
861b97765d [XPU] Fix fused MoE LoRA kernel crash on XPU by using platform-agnos num_compute_units (#43646) Chaojun Zhang 2026-05-26 18:40:32 +08:00
ebd0692f80 [Model] Use AutoWeightsLoader for InternLM2 (#38278) Javier De JesusandCyrus Leung 2026-05-26 12:39:26 +02:00
739af5c7e1 [Reasoning] [Bugfix] Reject invalid thinking_token_budget values (#43402) linzm1007andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-26 18:37:30 +08:00
5d09f471f4 [Misc] Support interleaved custom image benchmark datasets (#43636) Thibault Castells 2026-05-26 19:37:25 +09:00
681d7dd38b [Misc][Refactor][ROCm] Convert MoRI-related envvars to extra config args (#43303) Simon DanielssonandTJian 2026-05-26 12:33:35 +02:00
755043cf3c [KV Transfer] Enable HMA by default for connectors that support it (#41847) Ethan Feng 2026-05-26 18:28:51 +08:00
97e4022c6c [Bugfix] Apply fc_norm in Eagle3DeepseekV2 combine_hidden_states (#43482) Yubo WangandClaude 2026-05-26 00:46:10 -07:00
b3269454b1 [chores][log] change registry log from warning to debug (#43045) Hank_ 2026-05-26 15:13:46 +08:00
a37e47100c Add CuTe DSL sparse compressor support (#43584)
Jie Fang OpenAI Codex Yongye Zhu
2026-05-26 15:11:12 +08:00
e6adbd7834 Upgrade tpu-inference to v0.20.0 (#43394) Sting Lin 2026-05-26 14:26:25 +08:00
771e1e48b1 [CPU] Enable non-divisible GQA for decode workitems in mixed batches (#43032) zhao, zhenhui 2026-05-26 14:15:47 +08:00
d56612c621 [GDN] GDN Prefill kernel for SM100 (#43273) Thien Tran 2026-05-26 13:02:11 +07:00
6f955986e1 [Bugfix][Model] Fix GPT2ForSequenceClassification sub-module prefix (#43579) Huanyu YangandCyrus Leung 2026-05-26 13:43:19 +08:00
d5cf7b4a2c [Frontend] Split the offline inference APIs and utils. (#43553) wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-26 13:20:24 +08:00
f815c99954 [Bugfix] fix device mismatch in MiniCPM-o-4_5 resampler (#43194) Yan Ma 2026-05-26 13:12:50 +08:00
c2a4005c70 [KV Connector] Propagate MooncakeStore load failures (#42788) Dao007forever 2026-05-25 22:12:15 -07:00
7966fc7233 [KV Connector][Bugfix] MooncakeStore: don't double-apply Eagle prune in load_mask (#43516) Dao007foreverandClaude 2026-05-25 22:11:57 -07:00
aa2b56ffb0 [DeepSeek V4] Move MegaMoE input prep kernel to nvidia/ops (#43632) Woosuk Kwon 2026-05-25 21:08:29 -07:00
ec5de7fa7d [LoRA] Add one shot triton kernel For MoE LoRA (#42290) Jee Jee Li 2026-05-26 10:47:04 +08:00
71d810bbf4 [XPU] Ensure RNG offset alignment with PyTorch requirements in XPU sampler (#43028)
Chaojun Zhanggemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-26 10:01:30 +08:00
d4004455d2 [Kernel] Remove NormGateLinear (#43554) Jee Jee Li 2026-05-25 17:49:19 +08:00
716d5294e6 [Misc] Print accuracy value for PD tests even on success (#43583) Nicolò Lucchesi 2026-05-25 11:10:01 +02:00
873758c13a [KV Connector] Handle Mooncake finish after preemption (#43281) Zhewen LiandZhewen Li 2026-05-25 01:58:38 -07:00
5c1aec3dc0 Reduce memory usage for granite_speech. (#42933) Yihukiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-25 14:12:57 +08:00
0c942c69d6 [Doc] Add section on escalating stalled contributions (#43568) Roy Wang 2026-05-25 14:11:01 +08:00
81252d4e24 [Feat][KVConnector] Support DSV4 in SimpleCPUOffloadBackend (#42296) Yifan Qiao 2026-05-24 23:04:30 -07:00
3df1c7c43e [Docker] Non-root support for vllm-openai; add opt-in vllm-openai-nonroot target (#40275) Nguyễn Thế DuyandClaude 2026-05-25 12:45:31 +07:00
1b26fa361e [Docs] Reorganize offline inference docs. (#43552) wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-25 13:44:39 +08:00
6cbe448eed fix: MoE model using shared routed experts crashes on AMD GPUs (#42373) weizhoublue 2026-05-25 12:03:05 +08:00
b06813e872 [Kernel] Add mhc_pre_big_fuse_with_norm_tilelang (#43474) Jee Jee Li 2026-05-25 09:19:45 +08:00
d0a100c87a File system secondary tier implemented in python (#41735) Rotem ShavittandOr Ozeri 2026-05-24 21:14:44 +03:00
d56285c747 Tuning script and configs for Triton Mamba SSU kernel (#43083) daniserebandBanani Ghosh 2026-05-24 20:12:44 +03:00
1806d1adfc [ROCm] [DSv4] [Perf] Support DeepSeek v4 MTP (#43385) TJian 2026-05-24 18:43:08 +08:00
5940590855 [ROCm][CI] Stabilize 400 error return code for invalid schema inputs (#43016) Andreas Karatzas 2026-05-24 05:06:49 -05:00
357fddf614 [kv_offload]: Add DSv4 support (#43142) Or Ozeri 2026-05-24 11:10:12 +03:00
0902d8e62f [KV Connector] Keep MooncakeStore full hits block-aligned (#43494) Dao007foreverandClaude 2026-05-23 23:15:03 -07:00
33d7cbe02c [Model Runner v2] Force v1 runner for tests (#43233) Wentao Ye 2026-05-23 19:37:24 -04:00
b32fe416ea [Bugfix] Fix reasoning dropped on streaming boundary deltas (#42691) Flora Feng 2026-05-23 19:18:30 -04:00
10d264a2b9 Revert "[Misc] add humming to dependencies" (#43492) Michael Goin 2026-05-23 17:21:13 -04:00
46f95b2ec2 [ROCm][Critical] Fix the GDN import bug (#43486) TJian 2026-05-24 05:12:58 +08:00
819c610f9b [Mooncake] Add metrics for MooncakeStoreConnector operations (#43392) Dao007forever 2026-05-23 13:34:40 -07:00
4438b6e7dc [MoE] Migrate W4A8 CT to oracle kernel setup (#42680) Siddharth BedekarandOpenAI Codex 2026-05-23 10:56:01 -07:00
8737e4a857 [Docs] Fix stale version number in token_classify.md (#43489) Holegots 2026-05-24 01:42:20 +08:00
7c2ff1f819 [Docs] Fix stale version number in token_embed.md (#43488) Holegots 2026-05-24 01:06:56 +08:00
a0be71ee47 [MM] Enable FlashInfer metadata support for Qwen2.5-VL vision attention (#42787) Hua HuangandIsotr0py 2026-05-23 09:08:40 -07:00
d8b385b7ea [Bugfix][Frontend] Fix input_audio parsing when uuid is present (#43414) ffggsandCyrus Leung 2026-05-24 00:03:19 +08:00
2a7d5b7324 [ROCm][CI] Remove benchmarks test group and shard long test groups (#41669) Andreas Karatzas 2026-05-23 10:31:46 -05:00
5bb8d2767a [Kernel] Batch invariant NVFP4 linear using cutlass (#39912)
Jakub Zakrzewski Wentao Ye Yongye Zhu
2026-05-23 15:41:12 +02:00
3f3e862681 fix(eagle3): read norm_before_fc from eagle_config for NVIDIA checkpoint (#42143) GuangYaoZheng 2026-05-23 16:21:34 +08:00
82536acc54 Keep scheduler alive for delayed KV connector frees (#43433) Gabriel Wu 2026-05-23 14:23:32 +08:00
09a219c075 [ModelOpt] Support Qwen3.5/3.6 VLM quantized prefix mapping (#42546) Wei-Ming Chen 2026-05-22 23:23:31 -07:00
d19db10974 [Bugfix] Fix native Triton top-k/top-p kernel assumes contiguous logi… (#42739) Xiaogang Zhouandxiaogang.zhou 2026-05-23 13:56:16 +08:00
3a1c062151 [Misc] Added missing return type annotations to improve mypy and IDE tooling (#43383) Taneem Ibrahim 2026-05-23 00:28:22 -05:00
a7be0f342d [7/n] Migrate pos_encoding and norm kernels to libtorch stable ABI (continued) (#43209)
Chris Leonard Mikayla Gawarecki Shengqi Chen
2026-05-23 01:20:00 -04:00
54d153637b [XPU] reudce host overhead of XPU MOE (#42915) Qiming ZhangandKunshang Ji 2026-05-22 22:09:34 -07:00
a5bbd81e2e [XPU]feat: enable FP8 block-scaled quantization on XPU (#42952) Ma JianandKunshang Ji 2026-05-23 12:33:18 +08:00
d28bdf9344 [ROCm][CI] Fix ROCm LoRA Transformers fallback with full CUDA graphs (#41577) Andreas Karatzas 2026-05-22 23:31:32 -05:00
84e351555a [Bugfix] Auto-raise max_num_batched_tokens for prefix-LM multimodal models (#43051) Ashwin Giridharanandabinggo 2026-05-22 21:23:50 -07:00
76ea1d5d2f [ROCm][CI] Stabilize Granite tool-use and test URL construction (#43017) Andreas Karatzas 2026-05-22 23:21:11 -05:00
6a4723a2e0 [ROCm][CI] Stabilize runner teardown between sampler tests (#43023) Andreas Karatzas 2026-05-22 23:19:54 -05:00
367cb81966 [DSV4] More multi-stream enablement for c4a (#42925) Yongye Zhu 2026-05-22 21:22:27 -04:00
3cb83c9592 Add model to WeightTransferEngine.__init__ (#42922) Sumanth R Hegdeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-22 17:52:15 -07:00
552bbe6f4e [Attention] Add head_dim=512 support for FlashInfer trtllm attention backend (#38822) Duncan Moss 2026-05-22 17:27:35 -07:00
6d30655b13 elastic_ep: stage/commit MoE quant method on reconfigure (#40881) Itay Alroy 2026-05-23 01:57:26 +03:00
8de5cabeb7 [XPU]fix: add XPU platform guards to DeepSeek-V4 ops (#42950) Ma JianandKunshang Ji 2026-05-23 06:29:45 +08:00
4e2eba28be [Perf] Optimize hidden state extraction logic (#37374) Benjamin Chislettandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-22 18:23:08 -04:00
f743254143 DSv4 fused Q-norm kernel grid refactor (#42353) gnovack 2026-05-22 15:21:33 -07:00
47d4407d7c [Model Runner V2] Support sharing kv cache layers (#35045) Nick Hill 2026-05-22 15:18:23 -07:00
e203006a8b [Quantization][ModelOpt] W4A16 NVFP4 fused MoE + mixed-precision dispatch (#42566) Juhi Mittal 2026-05-22 13:51:49 -07:00
08cb46789d mhc_post - remove sts & add vectorized copies (#43437) gnovackandgeorge 2026-05-22 13:44:29 -07:00
4e597b7491 [Bugfix] Clear error message for FP8 torchao quantization on unsupported GPUs (#36854) haosdentandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-23 04:09:17 +08:00
23f7b11bf4 [Bugfix] Detect wrong libcute_dsl_runtime.so variant in FlashInfer GDN (#43427) Artem Perevedentsev 2026-05-22 22:33:33 +03:00
977703aa94 [RFC][EPLB][#32028] Remove dead torch.accelerator.synchronize() from sync path (#40733) SandishKumarHNandSandishKumarHN 2026-05-22 19:19:24 +00:00
2b94d1c0ca [Frontend] Simplify AuthenticationMiddleware path extraction (#43426) Russell BryantandClaude Opus 4.6 2026-05-22 14:59:14 -04:00
843715739b [Refactor] Extract DeepSeek V4 sparse MLA impl into model folder (#43149) Yongye Zhu 2026-05-22 13:06:31 -04:00
b21f3d56d4 [KV Connector] MooncakeStore: don't co-queue save with load to avoid double delayed-free (#43371) Dao007foreverandClaude Opus 4.7 2026-05-22 09:14:11 -07:00
c7624bea5e [Bugfix] Source num_qo_heads from Attention layers in Flashinfer/Triton metadata builders (#42650) Zhanda ZhuandShang Wang 2026-05-22 12:10:03 -04:00
91f5b92438 [Rust Frontend] [Refactor] Extract a newtype for utility call ID (#43405) Bugen Zhao 2026-05-22 23:22:11 +08:00
f0feb15e7f [Multimodal] Simplify ViT CUDA graph interfaces (#41234) Isotr0py 2026-05-22 22:31:00 +08:00
fb21d8b4f9 Add NVFP4 MOE support for Deepseek V4. (#42209) sychen52 2026-05-22 07:21:51 -07:00
a377631d21 [CI] Fix AMD docker build tests (#43329) haosdent 2026-05-22 22:06:24 +08:00
d3a563501b [EPLB] Change default EPLB communicator (#43110) Ilya MarkovandMarkov Ilya 2026-05-22 15:43:27 +02:00
15f7cd33dc [LoRA] Reduce memory of 2D weights when EP is set (#42737) Jee Jee Li 2026-05-22 21:41:56 +08:00
79ff0ffa98 [BugFix] wire make_empty_intermediate_tensors on AyaVision and Voxtral (#43118) Keyi LiandKeyi Li 2026-05-22 05:26:41 -07:00
4658bf882b [Bugfix] Clear P0 mm sender cache on sleep/pause to fix mm_hash desync (#43001) Tobias Wasner 2026-05-22 12:54:29 +02:00
b3c7ffcab8 [Misc] Replace assert with proper exceptions for security and validation in pooling (#43286)
Taneem Ibrahim Claude Opus 4.6 Nick Hill
2026-05-22 05:43:33 -05:00
d3d1cf6972 [XPU]feat: add XPU fallback for MoE topk routing and MXFP4 backend (#42951) Ma JianandKunshang Ji 2026-05-22 18:22:45 +08:00
7e1b45a092 [Attention] Mamba attention module refactor (#41126) wangxiyuan 2026-05-22 17:13:12 +08:00
65b7a812a2 [CPU] Experimentally enable Triton and MRV2 (#43225) Li, Jiang 2026-05-22 16:48:17 +08:00
2380bfc210 [Docs] Note image preprocessing difference between qwen_vl_utils and vllm. (#43393) wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2026-05-22 16:43:14 +08:00
a761697717 Fix the docker build failure in tpu-inference (#43360) mrjunwan-lang 2026-05-22 01:36:17 -07:00
694d9a81bb [BugFix] Fix setuptools-rust dep in requirements files (#43377) Nick Hill 2026-05-22 00:25:10 -07:00
6bb8753db1 Correcting the mock classes for MM GC tests (#43321) Weida Hong 2026-05-22 15:21:35 +08:00
025d4f5cd2 [CI] Fix "test_awq_load[gemma4-moe-*]" failure (#43296) haosdent 2026-05-22 15:13:59 +08:00
5ea76fa89a [CI] Fix test_lora_with_spec_decode on V2 model runner (#43314) haosdentandWentao Ye 2026-05-22 14:24:18 +08:00
fa1ff88b31 [Model] Fix MiniCPM-V 4.6 vit_merger qkv weight loading (#43213) tc-mb 2026-05-22 13:44:06 +08:00

... 63 64 65 66 67 ...

Commit Graph Select branches Hide Pull Requests main patches sm86 Mono Color

Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86