Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 1dc3fe08ea gemma3 multi-gpu bug-fix (#42630) pmaybankandTJian 2026-05-15 05:32:05 -04:00
• d26a28ab03 fix: propagate revision/code_revision pins to all artifact boundaries (#42616) Juan Pérez de AlgabaandCyrus Leung 2026-05-15 11:31:54 +02:00
• d735968f6d [ROCm][CI] Stage B gating (#42025) Andreas Karatzas 2026-05-15 03:49:27 -05:00
• ccde9540be DeepSeekV4-Pro enable cuda graph full and piecewise mode (#42604) bobofang_amdandTJian 2026-05-15 16:45:30 +08:00
• 75fd68c7a5 [Entrypoints] Split the pooling offline API into PoolingOfflineMixin. (#42267) wang.yuqi 2026-05-15 16:05:57 +08:00
• 4b364f810e [Core][DSV4] Skip caching SWA blocks that can never serve a prefix-cache hit (#42258) Yifan Qiao 2026-05-15 00:59:18 -07:00
• 31fa757cf9 [Misc] Make it simpler to replace out-of-tree layer classes with related LoRA layers. (#42306) yupengandJee Jee Li 2026-05-15 15:20:42 +08:00
• 2676ab1e0b [Deprecation] Remove old locations of get_tokenizer and resolve_hf_chat_template (#35024) Cyrus Leung 2026-05-15 15:13:32 +08:00
• 27b85d2084 [Bugfix] Clarify CPU backend memory error messages reference shared flag (#42479) Daniel WandLi, Jiang <jiang1.li@intel.com> 2026-05-15 02:35:05 -04:00
• e30f39c4f1 Update Intel Xeon model list and vLLM Benchmark Suite BKMs (#42607) Louie Tsai 2026-05-14 22:14:03 -07:00
• bf610c2f56 [Bugfix] Fix inverted condition causing thinking_token_budget to be silently ignored (#41674) 3 people 2026-05-14 21:48:49 -07:00
• faa4b76afa [Model] Support InternS2 Preview (#42705) Isotr0pyandzxy 2026-05-15 12:30:26 +08:00
• f351455f0f [CPU][RISC-V] Add RVV-optimized attention kernels for RISC-V Vector Extension (#40119) lyd1992andClaude 2026-05-15 12:08:23 +08:00
• 56434e8651 [Bugfix] Fix incorrect chat template format for Qwen3.5 (#42660) Cyrus Leung 2026-05-15 11:52:52 +08:00
• 0d4d334eaa Bump llguidance to 1.7 (#42150) Chao-Ju Chenandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-15 08:35:27 +08:00
• fa2a33b893 [Quant] Consolidate GPTQ: rename gptq_marlin.py to auto_gptq.py (#38288) 3 people 2026-05-14 17:25:52 -07:00
• 3b6a204789 [Model Runner V2][Bug Fix][DSV4] Ensure lazy attention state initializations happen during cudagraph capture (#42444) Giancarlo Delfin 2026-05-14 18:16:17 -05:00
• f8848b2f2d [Bugfix] Add swiglu limits to deepgemm fp8 methods (#41986) Yongye ZhuandClaude 2026-05-14 18:43:13 -04:00
• 4cfcc0866f [CI][ROCm] Remove unsupported cases in test_fusion.py (#38680) Charlie Fu 2026-05-14 16:37:18 -05:00
• f887aa1a53 [Aiter][ROCm] RMSNormGated+GroupedQuantFP8 fusion (#40710) TresandCursor 2026-05-14 21:37:09 +02:00
• 9898f94abe [Attention] Remove deprecated MLA prefill arguments (#42555) Matthew Bonanni 2026-05-14 13:34:06 -04:00
• ae4f59f0ec [Model Runner v2] Oracle for model runner v2 - qwen3 dense model by default [1/N] (#39337) Wentao YeandNick Hill 2026-05-14 13:02:33 -04:00
• f3d5360591 [Bugfix][Multimodal] PyAV video backend returns keyframes labeled as targets (#42586) Ranran 2026-05-14 11:56:59 -04:00
• a7737cb4f3 [Fix] Misc Fixes in ViT CUDA Graph (#38040) Baorun (Lauren) Mu 2026-05-14 11:49:06 -04:00
• b8a25d0e12 [Bugfix] Fix LM detection for Nemotron Parse (#42641) Cyrus Leung 2026-05-14 23:42:10 +08:00
• f07b1da797 [ROCm] Enable gluon paged MQA logits on gfx950 (MI355X) (#42062) frida-andersson 2026-05-14 17:39:26 +02:00
• f60c6b33a5 [V1][DP][LB] Publish request counts at the start of each engine step (#41626) Vadim GimpelsonandNick Hill 2026-05-14 19:39:24 +04:00
• 24337fb860 PD disagg with NIXL Connector: GDN support (Qwen3.5) (#41869) zhanqiuhuandNicolò Lucchesi 2026-05-14 10:33:01 -04:00
• c7560af424 [RFC] Replace shared-memory routed experts with ModelRunnerOutput transfer and HTTP support (#39568) 4 people 2026-05-14 22:12:30 +08:00
• 2317682f95 [Bugfix] Fix TRTLLM ragged MLA prefill workspace warmup (#42112) Mohammad Miadh Angkad 2026-05-14 21:48:56 +08:00
• 5bd8c71e79 [kv_offload] Implement reset_cache() for the offloading connector (#41956) 3 people 2026-05-14 14:00:10 +01:00
• 6548560496 [Compile] Fix compile warning with topk softplus sqrt (#41261) Wentao Ye 2026-05-14 08:12:50 -04:00
• 0a65d46628 [DSV4] Fuse norm and router for low latency scenario (#41263) 3 people 2026-05-14 20:11:02 +08:00
• 1ea9401364 [Quantization][Autoround][Toolkit] Add W4A16 Support (#39778) Zhenzhong Xu 2026-05-14 19:18:49 +08:00
• 9946c38b7f [XPU] Fix double-transpose in XPUFP8ScaledMMLinearKernel for W8A8 quant method (#41689) Libin Tangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-14 02:17:39 -07:00
• 23c85343fb [Bug] Fix DeepSeek V4 AttributeError: module 'cutlass.cute.nvgpu' has no attribute 'LoadCacheMode' (#42342) Wentao YeandRoger Wang 2026-05-14 05:00:20 -04:00
• 768f4a6f26 [CI][AMD][BugFix] Prevent triton compiler error when running test_moe_layer with use_ep = True on ROCm (#40857) rasmith 2026-05-14 03:44:22 -05:00
• addef3299c [CI][AMD] Skip tests where models have problems or fails on both HW types (#42126) rasmith 2026-05-14 03:21:06 -05:00
• ce29c26b31 Update Dockerfile.rocm for AINIC & Thor NIC (#40453) 4 people 2026-05-14 15:24:27 +08:00
• 8c79ad6580 Revert "[Core] Replace routing replay with device cache and async D2H pipeline" (#39917) (#42434) aoshen02 2026-05-14 14:49:01 +08:00
• 0d2732dd91 [MLA Attention Backend] Add TOKENSPEED_MLA backend for DSR1/Kimi K25 prefill + decode on Blackwell (#41778) Yongye ZhuandRoger Wang 2026-05-14 02:48:02 -04:00
• fd7d858c8a Use hidden_pad and intermediate_pad from vLLM #34301 (#42098) Rebecca Lee 2026-05-13 23:21:04 -07:00
• b26558d4a3 [CI][XPU] skip ut of offload connector (#42598) liuzhenwei 2026-05-14 13:13:53 +08:00
• bf0d2dc6d7 [Misc] Fix mypy error in parser_manager type narrowing (#42441) Sarah Salah 2026-05-14 05:48:59 +03:00
• ca60a4e84f [Fix] Weight loading for qwen3_5 using runai_streamer (#42521) Harsh ShahandHarsh Shah 2026-05-14 08:06:20 +05:30
• 77e1421a68 [Bugfix] Fix EPLB initialization for VLM wrapper models (#39805) Roy Wang 2026-05-14 10:26:15 +08:00
• 751b9f14bd [XPU][CT] Support mxfp8 moe model (#41918) Kunshang Ji 2026-05-14 09:47:10 +08:00
• 70c00163ff [Feature] Add instruction support for score/rerank chat templates (#42412) Krish Gupta 2026-05-14 07:11:22 +05:30
• f51f6844f9 [Bugfix][Spec Decode] Wire draft_probs into probabilistic draft_model rejection (#40269) Siddharth Bedekar 2026-05-13 18:04:03 -07:00
• 665f9c4253 [Bugfix] Fix Gemma4ToolParser streaming float corruption (#42128) longguo 2026-05-14 09:03:30 +08:00
• 1087676a90 [Refactor] Use shared utils in hermes tool parser (#42570) Flora Feng 2026-05-13 20:35:45 -04:00
• 63cc8a55a9 fix(tool-parser): preserve "none"/"nil" strings as valid enum values in minimax_m2 (#39599) 4 people 2026-05-13 17:35:34 -07:00
• ca7e4546da [CI] set max transformers version for skywork model (#42104) Divakar Verma 2026-05-13 19:53:49 -04:00
• b2198670b1 [Bugfix] V1: support tuple model outputs in ubatch wrapper (dbo + spec decode) (#40789) Yufeng HeandMatthew Bonanni 2026-05-14 06:47:51 +08:00
• f1cc7aad3c [Bugfix] Fix DeepSeek V4 MTP HC state handling (#42320) Mohammad Miadh Angkad 2026-05-14 06:44:52 +08:00
• 597ed13803 [Core][MM] Do not use urllib3 to parse data URLs (#42535) Lukas Geiger 2026-05-13 23:21:01 +01:00
• 6b5c389ee3 expose flex block size for batch invariant mode (#41252) liangel-02 2026-05-13 17:11:57 -04:00
• 8efd508204 [Quantization] Rework quantization_config to use QuantKey and allow for activation override (#41566) Michael Goin 2026-05-13 16:58:32 -04:00
• cca32d55a2 [PD] Fix broken NIXL EP installation (#42542) ovidiusm 2026-05-13 22:55:51 +02:00
• 873910d608 [Frontend] add support for thinking_token_budget in completions (#42116) Walter Beller-Morales 2026-05-13 16:01:52 -04:00
• 3f611f6106 [CI] Fix pre-commit issue (#42563) Wentao Ye 2026-05-13 15:37:26 -04:00
• a505cf807e [ModelRunner V2] Share identical MTP weights (#42538) Nick Hill 2026-05-13 11:57:04 -07:00
• 40330967ab [Quark] Support loading Quark NVFP4 checkpoints in vLLM (#35859) fxmarty-amdandKyle Sayers 2026-05-13 20:17:36 +02:00
• ab1ad0d7a9 Remove verifier model type check in speculative config (#42536) Fynn Schmitt-Ulms 2026-05-13 14:14:39 -04:00
• 0f69128a37 [Bugfix] Handle real-world gpt-oss tool call output in Harmony parsing (#42454) Ben Browning 2026-05-13 13:54:46 -04:00
• b3c69595a6 [MM][CG] Support ViT CG for Qwen2-VL (#41736) John CalderonandRoger Wang 2026-05-13 13:52:35 -04:00
• 2f821faeae [Spec Decode] Support hybrid attention models in extract_hidden_states (#39949) Michael GoinandClaude Opus 4.6 2026-05-13 13:45:53 -04:00
• 5794c65f8c [Bugfix][Model] Gemma4 MoE routing closure captures per_expert_scale, breaking functional_call substitution (#42250) 4 people 2026-05-13 14:43:12 -03:00
• 256dbcaabf [Feature] Support custom callable proposer backend for speculative decoding (#39487) CynicDora 2026-05-14 00:53:01 +08:00
• e35c0d4c63 [Feature] Support compile mode for batch invariance on SM80 (#42456) Wentao Ye 2026-05-13 11:02:39 -04:00
• 11f6b545d4 [kv_offload] Add multi-tier KV cache offloading framework (#40020) Ronen Schaffer 2026-05-13 17:21:43 +03:00
• a8887c208f [Bugfix] [ROCm] [DSV4] [Perf] Add aiter mhc support (#41946) TJianandYongye Zhu 2026-05-13 21:43:15 +08:00
• 0ddaf6dffa [XPU] [CT] Enable CT W4A4MxFp4 path and add xpu kernel (#38896) zofiaandKunshang Ji 2026-05-13 21:43:00 +08:00
• 67671692ac [CI] Re-enable Nemotron Parse parity test and switch testing to nemotron-parse v1.2 (#42498) Marek Wawrzos 2026-05-13 15:05:27 +02:00
• 0a62f5eec9 [AMD] skip machete tests for rocm (#42326) hissu-hyvarinen 2026-05-13 15:11:03 +03:00
• 3b1ef03be4 [Bugfix][Quark] Fix W8A8 INT8 garbage outputs on Step-3.5-Flash (and other 3-key fused-MoE Quark exports) (#41892) PikaPikachu 2026-05-13 19:59:49 +08:00
• 3c413a5481 Triton attention: add USE_TD constexpr for tensor descriptor Q/K/V load/store (#40327) Artur Fierkaandquinnlp 2026-05-13 13:57:41 +02:00
• 79fd1bc7ed [kv_offload] Add req_id to ReqContext for per-request tracking (#42507) Ronen Schaffer 2026-05-13 14:11:10 +03:00
• cee6751e54 [Bugfix][Qwen3-VL] Fix pipeline-parallel deepstack initialization (#42394) SILONG ZENG 2026-05-13 18:58:42 +08:00
• 16863072ca [Bugfix] Fix scipy audio resampling ratio (#42233) JooHo LeeandJooHo Lee 2026-05-13 19:52:41 +09:00
• d628a3c5cb [ROCm][CI] Skip ROCm batch invalid-input test pending torch fix (#41572) Andreas Karatzas 2026-05-13 05:50:47 -05:00
• 74dffae666 [ROCm] Run AITER RMSNorm pad fusion before AR RMS fusion (#42411) akii96 2026-05-13 13:35:12 +03:00
• 97c4317bf5 [Bugfix][Frontend] Default max_tokens server-side on /inference/v1/generate (#42329) halleriteandClaude Opus 4.7 2026-05-13 11:16:46 +02:00
• f6e868fbdf [CI] Use uv with Python 3.12 for PyPI wheel upload (#42470) Kevin H. LuuandClaude Opus 4.6 2026-05-13 02:12:06 -07:00
• 13bf242100 [Feat][KVConnector] Add bind_gpu_block_pool() to KVConnectorBase_V1 (#39654) Yifan Qiaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-13 02:10:29 -07:00
• 140dc2ec30 [Bugfix] Install nvidia-cutlass-dsl[cu13] extra on CUDA 13 platforms (#42438) Jiangyun Zhu 2026-05-13 16:57:21 +08:00
• 9ce74042d3 [Bugfix][SimpleCPUOffloadBackend] Dedup in-flight CPU offload stores across scheduler steps (#41289) Yifan Qiaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2026-05-13 01:53:32 -07:00
• a8c13d2837 Patch SlidingWindowSpec.real_page_size_bytes for nvfp4 kv (#42464) sychen52 2026-05-13 01:46:30 -07:00
• 92def124bc [MM][Perf][CG] Support ViT full CUDA graph for Qwen3.5 (#42151) Shanshan Shen 2026-05-13 16:00:32 +08:00
• 85b2fecab7 [5/n] Migrate CUTLASS MLA, hadamard, awq, allspark and DSV3 fused a gemm to torch stable ABI (continued) (#42339) Chris LeonardandMikayla Gawarecki 2026-05-13 03:24:39 -04:00
• 503697c9ce [chore] Refactor pooling metadata token ID accessors (#42368) 3 people 2026-05-13 02:08:01 -04:00
• 71bcd02ef3 [Bugfix][PD] Fix multi-node TP (TP>8) (#39907) Nicolò Lucchesi 2026-05-13 07:20:57 +02:00
• dcacdf9a88 [Attention] Sync FA with upstream (#41052) Matthew Bonanni 2026-05-12 23:34:18 -04:00
• 18f6bf5a21 [MoE Refactor] Add sequence parallel tests to test_moe_layer.py (#41299) bnellnm 2026-05-12 21:52:19 -04:00
• 07534b8782 [PD] Bump NIXL connector dependency to 1.x (#42364) Alec 2026-05-12 18:05:01 -07:00
• 3d635c58c0 [Perf] Optimize MLA compute_prefill_context memory allocation (#42460) Wentao Ye 2026-05-12 19:23:46 -04:00
• ebeb09d822 [KV Transfer] Add MooncakeStoreConnector for KV cache offloading via Mooncake distributed store (#40900) +313 people 2026-05-13 07:09:10 +08:00
• 184577ae46 [Build] DeepGEMM: trim comments, add integration notes + TODOs (#42429) Michael Goin 2026-05-12 18:57:58 -04:00
• 8c4fc4202a [CI] Inline build artifact annotations in release pipeline (#42357) Kevin H. Luu 2026-05-12 15:57:43 -07:00
• fe8b42e80c [CI] Fix test_async_scheduling.py flakiness (#42455) Nick Hill 2026-05-12 14:38:32 -07:00