Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 63fe3a700f [PD] let p2p nccl toy proxy handle /chat/completions (#21734) Chauncey 2025-07-28 19:45:50 +08:00
• 0ae970ed15 [Bugfix] Fix glm4.1v video_grid_thw tensor shape scheme (#21744) Isotr0py 2025-07-28 19:26:49 +08:00
• 65e8466c37 [Bugfix] Fix environment variable setting in CPU Dockerfile (#21730) Li, Jiang 2025-07-28 19:02:39 +08:00
• 1b769dccf3 [Bugfix] Fix Ernie4_5_MoeForCausalLM shared experts (#21717) Jee Jee Li 2025-07-28 19:02:25 +08:00
• 2cc571199b [feature] add log non default args in LLM (#21680) rongfu.leng 2025-07-28 17:21:22 +08:00
• a4ed731546 [Model] Prioritize Transformers fallback over suffix matching (#21719) Cyrus Leung 2025-07-28 17:15:31 +08:00
• d128d0d554 Migrate KeyeImageInputs and KeyeVideoInputs to TensorSchema (#21686) Benji Beck 2025-07-28 01:16:35 -07:00
• a6c050286a [v1][mamba] Added mamba_type into MambaSpec (#21715) Asaf Joseph Gardinandasafg 2025-07-28 11:15:55 +03:00
• 139a7f07bd [BugFix] Fix ChunkedLocalAttention when the hybrid kv-cache is disabled (#21707) Lucas Wilkinson 2025-07-28 03:18:47 -04:00
• 150d9e6337 [Bugfix] fix max-file-size type from str to int (#21675) Ning Xie 2025-07-28 15:06:52 +08:00
• 139a97ec56 [Bugfix] Fix shape checking for Fuyu (#21709) Cyrus Leung 2025-07-28 15:05:56 +08:00
• 18cc33dd60 [bugfix] fix profile impact benchmark results (#21507) rongfu.leng 2025-07-28 13:44:24 +08:00
• 7656cf4cf3 [Bugfix] [issue-21565] Fix the incompatibility issue with stream and named function calling when Thinking is disabled (#21573) Hongsheng Liuandwangzi 2025-07-28 13:43:50 +08:00
• 3ea57a56d9 Migrate Idefics3ImagePixelInputs and Idefics3ImageEmbeddingInputs to … (#21683) Benji Beck 2025-07-27 22:37:23 -07:00
• 75856bc2cb Migrate GraniteSpeechAudioInputs to TensorSchema (#21682) Benji BeckandCyrus Leung 2025-07-27 22:37:20 -07:00
• 304dcdf575 Migrate GLMVImagePixelInputs to TensorSchema (#21679) Benji Beck 2025-07-27 22:36:11 -07:00
• 88e46c7c8d Migrate Glm4vImageInputs, Glm4vVideoInputs to TensorSchema (#21678) Benji BeckandCyrus Leung 2025-07-27 22:36:08 -07:00
• d8937de4c8 Migrate Gemma3ImagePixelInputs to TensorSchema (#21676) Benji Beck 2025-07-27 22:36:05 -07:00
• e626d286f5 [FEAT] [ROCm] [AITER]: Add AITER HIP block quant kernel (#21242) TJian 2025-07-27 22:07:06 -07:00
• c7ffe93d9c [Model] Support TP/PP/mamba2 kernel for PLaMo2 (#19674) 3 people 2025-07-28 14:00:47 +09:00
• 15a72ac478 [V1] Exception Handling when Loading KV Cache from Remote Store (#21534) Adelineandliuyumoye 2025-07-28 11:34:17 +08:00
• 04ff4be310 [Misc] Add fused_moe configs for Qwen3-Coder-480B-A35B-Instruct-FP8 (#21700) Jee Jee Li 2025-07-28 11:12:18 +08:00
• 93269bb43e Fix GLM tool parser (#21668) Yuxuan ZhangandChenhui Zhang 2025-07-28 10:46:38 +08:00
• 82acf2184d Fix typo for limit-mm-per-prompt in docs (#21697) Joachim Studnia 2025-07-27 19:45:37 -07:00
• 86ae693f20 [Deprecation][2/N] Replace --task with --runner and --convert (#21470) Cyrus LeungandHarry Mellor 2025-07-28 10:42:40 +08:00
• 8f605ee309 [Attention] Make CutlassMLA the default backend for SM100 (blackwell) (#21626) Alexander Matveevandmgoin 2025-07-27 16:13:00 -04:00
• a9b2a1d704 [Misc] Refactor vllm config str (#21666) Ning Xie 2025-07-28 00:51:44 +08:00
• 57c22e57f9 Fix CUDA permute/unpermute for use with DeepGemm Moe (#17934) Caleb_Du 2025-07-27 22:08:00 +08:00
• bda9d0535f [Refactor] Refactor MOE NVFP4 Code Base: ModelOpt + Compressed Tensor (#21631) Wentao Ye 2025-07-27 08:25:21 -04:00
• 3d847a3125 [VLM] Add video support for Intern-S1 (#21671) Isotr0py 2025-07-27 19:49:43 +08:00
• 5f8c9a425e Migrate Florence2ImagePixelInputs to TensorSchema (#21663) Benji Beckandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-07-27 02:43:02 -07:00
• 1cbf951ba2 [Misc] add default value for file pattern arg (#21659) Ning Xie 2025-07-27 13:14:51 +08:00
• a8936e5193 Refactor: Remove numpy dependency from LoggingStatLogger (#20529) ZiTian.Zhao 2025-07-27 12:06:21 +08:00
• 01a395e9e7 [CI/Build][Doc] Clean up more docs that point to old bench scripts (#21667) Ye (Charlotte) Qi 2025-07-26 21:02:12 -07:00
• 971948b846 Handle non-serializable objects in vllm bench (#21665) Huy Do 2025-07-26 20:35:22 -07:00
• eed2f463b2 [VLM] Support HF format Phi-4-MM model (#17121) Isotr0py 2025-07-27 11:07:57 +08:00
• 20950b29fb Migrate ChameleonImagePixelInputs to TensorSchema (#21657) Benji Beck 2025-07-26 19:34:25 -07:00
• 3339cba3ff Migrate FuyuImagePatchInputs to TensorSchema (#21662) Benji Beck 2025-07-26 19:34:14 -07:00
• 0b8caf9095 Migrate DeepseekVL2ImageInputs to TensorSchema (#21658) Benji Beck 2025-07-26 19:34:11 -07:00
• ccf27cc4d4 Migrate Blip2ImagePixelInputs and Blip2ImageEmbeddingInputs to TensorSchema (#21656) Benji Beck 2025-07-26 19:33:52 -07:00
• c657369841 support torch.compile for bailing moe (#21664) Jinzhen Lin 2025-07-27 07:54:32 +08:00
• 6c66f28fa5 Remove xformers requirement for Mistral-format Pixtral and Mistral3 (#21154) Wenchen Lo 2025-07-26 16:20:29 -07:00
• de509ae8eb [NVIDIA] Explicitly disable shuffled weights for flashinfer blockscale moe fp8 kernels (#21411) Kaixi Hou 2025-07-26 07:10:36 -07:00
• e7c4f9ee86 [CI/Build][Doc] Move existing benchmark scripts in CI/document/example to vllm bench CLI (#21355) Ye (Charlotte) Qi 2025-07-26 07:10:14 -07:00
• 9094d11c5d [Bugfix][Apple Silicon] fix missing symbols when build from source on Mac with Apple Silicon (#21380) Yeju Zhou 2025-07-26 22:09:57 +08:00
• 56e544f24b [Refactor] Remove moe_align_block_size_triton (#21335) Wentao Ye 2025-07-26 10:08:29 -04:00
• 97d6c30cc9 [BugFix] Fix shared storage connector load kv only load attention layer (#21428) WeiQing Chen 2025-07-26 22:07:40 +08:00
• a40a8506df [Misc] Improve memory profiling debug message (#21429) Ye (Charlotte) Qi 2025-07-26 07:07:21 -07:00
• c215f5c877 [Bug] Fix has_flashinfer_moe Import Error when it is not installed (#21634) Wentao Ye 2025-07-26 10:06:14 -04:00
• 1cd6eaba54 Support encoder-only models without KV-Cache (#21270) Maximilien de BayserandRussell Bryant 2025-07-26 10:09:52 -03:00
• f27fdfc3ed [Bugfix] Investigate Qwen2-VL failing test (#21527) Isotr0py 2025-07-26 21:09:29 +08:00
• de10ff0b7c Migrate AyaVisionImagePixelInputs to TensorSchema for shape validation (#21622) Benji Beck 2025-07-26 06:08:18 -07:00
• 9d197280fa Migrate AriaImagePixelInputs to TensorSchema for shape validation (#21620) Benji Beck 2025-07-26 06:08:15 -07:00
• e98def439c [Take 2] Correctly kill vLLM processes after benchmarks (#21646) Huy Do 2025-07-26 06:06:05 -07:00
• 05c1126f29 [Misc] remove unused try-except in pooling config check (#21618) Reid 2025-07-26 20:20:03 +08:00
• 875af38e01 Support Intern-S1 (#21628) 5 people 2025-07-26 19:14:04 +08:00
• 7728dd77bb [TPU][Test] Divide TPU v1 Test into 2 parts. (#21431) QiliangCui 2025-07-25 23:20:30 -07:00
• 2f6e6b33fb [Bugfix] Fix isinstance check for tensor types in _load_prompt_embeds to use dtype comparison (#21612) Alexandre JUAN 2025-07-26 05:11:10 +02:00
• a55c95096b Correctly kill vLLM processes after finishing serving benchmarks (#21641) Huy Do 2025-07-25 19:06:21 -07:00
• 97349fe2bc [Docs] add offline serving multi-modal video input expamle Qwen2.5-VL (#21530) WeiQing Chen 2025-07-26 09:37:32 +08:00
• 62965de5fe [Model] Ultravox: Support Llama 4 and Gemma 3 backends (#17818) Farzad AbdolhosseiniandPatrick Li 2025-07-26 04:12:31 +03:00
• 7ae75fa6d0 [Feature] Add support for MoE models in the calibration-free RTN-based quantization (#20766) Alex Kogan 2025-07-25 21:09:34 -04:00
• f1b286b2fb [TPU] Update ptxla nightly version to 20250724 (#21555) Chengji Yao 2025-07-25 17:09:00 -07:00
• c7742d6113 [Bugfix] Always set RAY_ADDRESS for Ray actor before spawn (#21540) Rui Qiao 2025-07-25 17:08:30 -07:00
• cea96a0156 [Bugfix] Fix sync_and_slice_intermediate_tensors (#21537) Rui Qiao 2025-07-25 17:07:58 -07:00
• 2eddd437ba Add interleaved RoPE test for Llama4 (Maverick) (#21478) Yong Hoon Shin 2025-07-25 17:07:26 -07:00
• 75d29cf4e1 [Perf] Cuda Kernel for Int8 Per Token Group Quant (#21476) Wentao Ye 2025-07-25 20:07:07 -04:00
• 41d3082c41 Add Unsloth to RLHF.md (#21636) Daniel Han 2025-07-25 17:06:48 -07:00
• 7cfea0df39 [TPU][Test] Rollback PR-21550. (#21619) QiliangCui 2025-07-25 13:22:01 -07:00
• 5ac3168ee3 [Docs] add auto-round quantization readme (#21600) 3 people 2025-07-25 23:52:42 +08:00
• 396ee94180 [CI] Unifying Dockerfiles for ARM and X86 Builds (#21343) Kebe 2025-07-25 22:33:56 +08:00
• e189b50f53 Add support for Prithvi in Online serving mode (#21518) mgazzandCyrus Leung 2025-07-25 15:01:27 +01:00
• 136d750f5f [Kernel] Improve machete memory bound perf (#21556) czhu-cohere 2025-07-25 06:53:21 -07:00
• b3caeb82e7 [ROCm][AITER] Enable fp8 kv cache on rocm aiter backend. (#20295) who who whoandamd-ruitang3 2025-07-25 21:50:21 +08:00
• eab2f3980c [Model] Replace Mamba2 RMSNorm Gated with Fused Triton Kernel (#20839) Chih-Chieh YangandYu Chin Fabian Lim 2025-07-25 09:49:36 -04:00
• 9fe98d4250 [Frontend] Add request_id to the Request object so they can be controlled better via external load balancers (#21009) kourosh hakhamaneshi 2025-07-25 06:49:11 -07:00
• 29c6fbe58c [MODEL] New model support for naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B (#20931) bigshanedogg 2025-07-25 22:05:42 +09:00
• c72f049cb4 [Model] Fix Ernie4.5MoE e_score_correction_bias parameter (#21586) xyxinyangandzhouchong 2025-07-25 21:02:53 +08:00
• f3a683b7c9 [Bugfix][Logprobs] Fix logprobs op to support more backend (#21591) Mengqing Cao 2025-07-25 20:53:07 +08:00
• 46d81d6951 [V1] Get supported tasks from model runner instead of model config (#21585) Cyrus Leung 2025-07-25 20:36:45 +08:00
• 5c3f2628d5 [Quantization] Enable BNB support for more MoE models (#21370) Jee Jee Li 2025-07-25 18:57:34 +08:00
• 7311f74468 [Bugfix] GGUF: fix AttributeError: 'PosixPath' object has no attribute 'startswith' (#21579) Kebe 2025-07-25 18:42:23 +08:00
• 8ed01e32f7 Add H20-3e fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct (#21598) Xu Wenqing 2025-07-25 17:36:55 +08:00
• e38e96a3c0 [Tests] Harden DP tests (#21508) Nick Hill 2025-07-25 10:27:24 +01:00
• 40d86ee412 [TPU][Bugfix] fix OOM issue in CI test (#21550) Chengji Yao 2025-07-24 23:01:53 -07:00
• 85d051f026 [Misc] Removed undefined cmake variables MOE_PERMUTE_ARCHS (#21262) Yang Chen 2025-07-24 22:54:23 -07:00
• 5140f54b89 [CI/Build] fix cpu_extension for apple silicon (#21195) Ignacio Sica 2025-07-25 02:53:59 -03:00
• 947edd099e [Misc][Tools] make max-model-len a parameter in auto_tune script (#21321) Chengji Yaoandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-07-24 22:46:43 -07:00
• fde60ee775 [Model] Fix a check for None but the return value was empty list in Gemma3 MM vision_embeddings (#21479) hfan 2025-07-25 01:46:06 -04:00
• b38bc652ac [Model] Support tensor parallel for timm ViT in Deepseek_vl2 (#21494) Jason Gu 2025-07-25 13:45:16 +08:00
• adaf2c6d4f [Bugfix] fix modelscope snapshot_download serialization (#21536) Ning Xie 2025-07-25 13:44:38 +08:00
• 42343f1f89 [CI] Update CODEOWNERS for CPU and Intel GPU (#21582) Li, Jiang 2025-07-25 12:58:03 +08:00
• 965bc71b04 Integrate TensorSchema with shape validation for Phi3VImagePixelInputs (#21232) Benji Beck 2025-07-24 21:43:52 -07:00
• 807a328bb6 [Docs] Add requirements/common.txt to run unit tests (#21572) Zhou Fang 2025-07-24 20:51:15 -07:00
• e0be2c4d09 [TPU][Test] Temporarily suspend this MoE model in test_basic.py. (#21560) QiliangCui 2025-07-24 20:44:50 -07:00
• 9c8b2c2a8a [DP] Support api-server-count > 0 in hybrid DP LB mode (#21510) Nick Hill 2025-07-25 04:18:16 +01:00
• 2212cd6cfb [Bugfix] DeepGemm utils : Fix hardcoded type-cast (#21517) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-07-25 08:47:29 +05:30
• ce3a9b1378 [Kernel] adding fused_moe configs for upcoming granite4 (#21332) Burkhard RingleinandThomas Parnell 2025-07-25 05:16:59 +02:00
• 2ce90e5b01 Fix GLM-4 PP Missing Layer When using with PP. (#21531) Yuxuan Zhang 2025-07-25 11:07:38 +08:00
• 633f6e804b [Bug] Fix DeepGemm Init Error (#21554) Wentao Ye 2025-07-24 23:07:22 -04:00