Commit Graph

Select branches

Hide Pull Requests

main

patches

sm86

Mono Color

• 03ee48111d Feature: Support Relu2 in FusedMoE fp8 cutlass path (#27261) amirkl94 2025-11-16 20:39:44 +02:00
• 5a87076d6e [Model][QwenVL] Optimize Qwen2_5_VisionAttention q,k preparation (#28769) Lukas GeigerandIsotr0py 2025-11-16 17:37:15 +00:00
• ac1daf3233 fix comment typo (#28802) Ning Xie 2025-11-17 01:03:21 +08:00
• 63fed55506 [Doc]: fix typos in various files (#28811) Didier Durand 2025-11-16 15:30:06 +01:00
• 8d259fad6c Fix gpt oss weight loading with EP + bf16 (#28765) Anna Shors 2025-11-16 05:12:45 -08:00
• 3bc1175798 [Bugfix] Fix host and port join for ipv6 in bench serve (#28679) scottzh8andScott Zhang 2025-11-16 02:20:57 -08:00
• af02c40970 Fixed gpt-oss _load_weights_other() parameter position bug (#28715) DezhanandDezhan Tu 2025-11-16 01:46:29 -08:00
• b316ac6589 [V1] Support MP Executor for multi node distributed inference (#23691) Lucia FangandNick Hill 2025-11-16 01:01:21 -08:00
• a55b64635c [Model] Allow users to control skip reading cache per request. (#28194) wang.yuqi 2025-11-16 16:04:50 +08:00
• d231876ce3 [Benchmark] Fix client seed synchronization in multi-turn benchmark (#28512) ai-jz 2025-11-15 23:04:32 -08:00
• f849ee739c Adding a benchmark for batch invariance (#28161) Bram Wastiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-11-16 00:22:17 -05:00
• be263f7645 [BugFix] Fix AssertionError: DCP not support reorder_batch_threshold > 1 now. (#28751) Lucas Wilkinson 2025-11-15 17:35:06 -05:00
• 2bb4435cb7 [Doc]: fix typos in various files (#28567) Didier Durand 2025-11-15 20:27:50 +01:00
• 07cadab27a [Model][Qwen3VL] Cache positional embedding indices (#28475) Lukas GeigerandRoger Wang 2025-11-15 19:03:09 +00:00
• 637f292196 [CI] Fix broken pipeline (#28781) Nick Hill 2025-11-15 08:44:14 -08:00
• e439c784fa Add support for Eagle with separate lm-head and embed_tokens layers (#28549) Eldar Kurtić 2025-11-15 15:12:02 +01:00
• 085a525332 [Model] Fix lmhead init bug of bailing_moe (#28777) 3 people 2025-11-15 21:44:12 +08:00
• 89d3679221 [Doc] Fix failing doc build (#28772) Cyrus LeungandHarry Mellor 2025-11-15 21:33:27 +08:00
• cb15ee28db Allow Gemma3 to take image embeddings (#28483) tingtinggithub 2025-11-15 04:18:08 -08:00
• f36292dbee [compile] Enable sequence parallelism matching w/o custom ops enabled (#27126) 4 people 2025-11-15 03:46:12 -08:00
• 173b356abf [PERF] Remove TRTLLM Gen attn kernel limitation max_seq_len <=131072 (#28755) Vadim Gimpelson 2025-11-15 14:13:41 +04:00
• 638e4196d1 [Misc] Make SchedulerConfig.max_model_len init-only (#28733) Cyrus Leung 2025-11-15 17:59:31 +08:00
• 1ec978c209 [Kernel][Moe Configs] llama4 maverick fp8 moe config tp8 on mi325 (#28709) Zhewen Li 2025-11-15 01:10:48 -08:00
• 74b5267d3a Use narrow over indexing in hadacore_transform to prep for ABI stable (#28756) Jane (Yuan) Xu 2025-11-15 04:10:15 -05:00
• dd6ac1c2bb [RL] [V1] Remove unused device argument from reset_kv_cache (#28766) Zhuohan Li 2025-11-14 23:59:42 -08:00
• 98b4d389ed [Redo] #26368 (#28771) Cyrus LeungandJialin Ouyang 2025-11-15 14:47:41 +08:00
• 6965ef436f [Performance][DeepGEMM] Estimate expected_m (#28694) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-11-15 00:52:14 -05:00
• c9e665852a [NIXL] heterogeneous block_size support (#26759) Chendi.XueandNicolò Lucchesi 2025-11-14 23:51:32 -06:00
• 363aaeef0f Fix IntermediateTensors initialization and add type hints (#28743) Mohammad OthmanandMohammad Othman 2025-11-15 06:31:36 +02:00
• ac86bff8cb Revert "[Core] Performance: Use list[np.ndarray] instead of list[list… (#28773) Nick Hill 2025-11-14 20:24:00 -08:00
• edfe498189 [Bugfix] Build hadacore kernels on >SM90 (#28748) Michael Goin 2025-11-14 22:51:05 -05:00
• f05d474c8a [Model][Qwen3VL] Use mm_position to compute mrope positions (#28730) Lukas GeigerandCyrus Leung 2025-11-15 03:45:11 +00:00
• 9fc81ec765 [TPU] Fix import error in tpu launch (#28758) QiliangCui 2025-11-14 16:58:32 -08:00
• 186352b270 [Core] Performance: Use list[np.ndarray] instead of list[list[int]] for output tokens for GC optimization (#26368) Jialin Ouyang 2025-11-14 16:04:04 -08:00
• 58e61e56b7 [Test] Rework e2e async scheduling tests (#28744) Nick Hill 2025-11-14 16:01:09 -08:00
• 75f01b9d3c [ROCm][CI/Build] Upgrade to ROCm 7.1 and AITER main (#28753) Gregory Shtrasberg 2025-11-14 18:53:21 -05:00
• ba041d980b [Log] Save profiler results to file instead of stdout (#28144) rasmithandRandall Smith 2025-11-14 17:26:39 -06:00
• e0c910bb89 [Hybrid] [Kernel] Fix chunk scan kernel when BLOCK_SIZE_DSTATE > 128 (#28295) Thomas Parnell 2025-11-14 23:55:42 +01:00
• bf3ffb61e6 [Bugfix] Fix ChunkedLocalAttention CUDA Graph setting (#28739) Benjamin Chislett 2025-11-14 17:14:46 -05:00
• e5c78956c0 [Bugfix] Fix incorrect use of hidden_states for shared_experts due to do_naive_dispatch_combine (#28740) Alexander Matveev 2025-11-14 17:13:46 -05:00
• 2e0ad629b0 Avoid bytecode hook and simplify TorchCompileWrapperWithCustomDipatch (#25110) Laith Sakka 2025-11-14 14:11:10 -08:00
• 5a84b76b86 [ROCm][CI/Build] Change install location of uv (#28741) Gregory Shtrasberg 2025-11-14 16:34:18 -05:00
• 0de4f217ab [Bugfix] TypeError: 'NoneType' object is not callable (#27410) Marcin Ostrowski 2025-11-14 22:13:53 +01:00
• f08eab2acc [CI] Fix macos smoke test uv cache issue (#28736) Michael Goin 2025-11-14 15:29:55 -05:00
• 8977ffb5e6 [ROCm][Bugfix] Fix compilation errors with fused_qknorm_rope_kernel.cu (#28682) Sage Moore 2025-11-14 11:06:01 -08:00
• fd4555089a [BugFix] Fix misprint introduced by modular_kernel refactoring. (#28728) Andrey Khalyavin 2025-11-14 21:58:18 +03:00
• cec275efce [Bugfix] resolve Qwen3-VL GPTQModel quantized model loading failure (#28663) GuanHandIsotr0py 2025-11-15 02:44:27 +08:00
• e2741f6cbc [Chore] Rename SchedulerConfig.chunked_prefill_enabled (#28735) Cyrus Leung 2025-11-15 02:39:57 +08:00
• 67187554dd [Docs] Enable some more markdown lint rules for the docs (#28731) Harry Mellor 2025-11-14 18:39:19 +00:00
• a425dc256e [Bugfix] [ROCm] [AITER]: Fix aiter block quant not compatible with torch compile dynamo (#28716) TJian 2025-11-14 10:30:50 -08:00
• 964d65deed LLaMA4 LoRA Adapter Enablement (#28602) Fardin HoqueandWei Wei 2025-11-14 10:27:56 -08:00
• 9261eb3dc1 docs(lora_resolvers): clarify multi-resolver order and storage path requirement (#28153) Chen WangandHarry Mellor 2025-11-14 13:08:30 -05:00
• cdd7025961 [kernel] Improve FP8 PTPC on Hopper for larger shapes (#28692) czhu-cohere 2025-11-14 12:59:11 -05:00
• 085424808e Remove audio optional dependency for mistral-common (#28722) 3 people 2025-11-14 18:54:38 +01:00
• a17e36f223 Fix typo in comment: existance -> existence (#28737) Mohammad Othman 2025-11-14 19:35:45 +02:00
• 8cc40f8992 [Attention] Bump FA for removed method (#28429) Matthew BonanniandCyrus Leung 2025-11-14 12:13:37 -05:00
• 6f1e7f7226 [DisaggEverything] Tokens in<>out /generate endpoint (#24261) Nicolò LucchesiandHarry Mellor 2025-11-14 17:58:01 +01:00
• d54a18a47e [CI][CPU] Smoke test for Apple Silicon using GHA MacOS runner (#28688) Michael Goin 2025-11-14 11:37:18 -05:00
• 5f3cd7f7f2 [Docs] Update the name of Transformers backend -> Transformers modeling backend (#28725) Harry Mellor 2025-11-14 16:34:14 +00:00
• c934caee88 [Fix] improve aspect ratio in dummy image generation and add common VLM tests for PaddleOCR-VL (#28711) dongbo910220 2025-11-15 00:07:20 +08:00
• 3f8a874065 [Kernels] Enable FlashInfer FP8 Blockscale on SM90 (for TEP DSR1) (#27134) 3 people 2025-11-14 08:02:44 -08:00
• 511a6b611d [Config] Clean up SchedulerConfig initialization (#28665) Cyrus Leung 2025-11-14 22:41:02 +08:00
• 96b23b8e3b [Bugfix][Nixl] Fix kernel physical<>logical block_size issue (#28677) Nicolò Lucchesi 2025-11-14 15:40:05 +01:00
• 433c0f8675 [Model] Fix bailing_moe accuracy problem (#28277) zhaozx-cn 2025-11-14 21:33:02 +08:00
• 8d3748d3c7 [Doc] Fix macOS installation dependency resolution issue (#26721) Fasal Shah 2025-11-14 18:13:56 +05:30
• db56a59970 [BugFix] Fix FA3 IMA with FULL_AND_PIECEWISE and cascade attention (default) (#28702) Lucas Wilkinson 2025-11-14 07:19:22 -05:00
• 9324e10275 Fix KV sharing fast prefill with cudagraph enabled (#28537) Yong Hoon ShinandCyrus Leung 2025-11-14 01:53:42 -10:00
• 4516d44b7f [DCP] Support Decode Context Parallel (DCP) for GQA with Flashinfer (#25438) 4 people 2025-11-14 19:24:10 +08:00
• 41b92f7d38 [Model][MM] Extract conv layer as CustomOp (#28455) Shanshan ShenandIsotr0py 2025-11-14 19:16:13 +08:00
• 360bd8762f [Frontend] Added chat-style multimodal support to /classify. (#27516) 6 people 2025-11-14 03:03:55 -08:00
• ecf8230d4d [Metrics] Log number of preempted requests (#28522) lyn610 2025-11-14 17:47:45 +08:00
• 8cfbe89b93 [Misc] fix comment in test_envs (#28529) Xing Liu 2025-11-14 01:32:46 -08:00
• fd75d3e8c0 [Minor] avoid register new custom and just import silly_attn (#28578) Boyuan Feng 2025-11-14 01:32:31 -08:00
• c9a3a02149 Add output token counting to gsm8k eval (#28594) Michael Goin 2025-11-14 04:32:03 -05:00
• bc3e43069a [BugFix] Fix multi-modal async scheduling race condition (#28706) Nick Hill 2025-11-14 01:11:13 -08:00
• c36bcfe6b3 [Bugfix] fix dots.ocr pp support (#28705) Jiangyun Zhu 2025-11-14 17:01:26 +08:00
• 529cea343d use default CCL_ZE_IPC_EXCHANGE (#28700) Yan Ma 2025-11-14 16:55:29 +08:00
• 93103575ce [BugFix][CI/Build][ROCM] Fix import error and apply assert in appropriate case in test_struct_output_generate (#28311) rasmithandRandall Smith 2025-11-14 00:41:29 -06:00
• 15ae8e0784 [Bugfix][CI/Test][Spec Decode] Fix illegal memory access in offline_inference/spec_decode.py (Issue 27619) (#28432) 3 people 2025-11-14 00:34:01 -06:00
• 0b25498990 [Misc] add ignore mapper for quark quantization (#28275) haoyangli-amd 2025-11-14 13:56:35 +08:00
• 0aecd9138f [Misc] Update xformers to 0.33.0.post1 (#28678) Roger Wang 2025-11-13 21:52:53 -08:00
• da14ae0fad [XPU][CI]disable lm cache uts (#28696) Kunshang Ji 2025-11-14 11:15:50 +08:00
• 01bea115c4 [Misc] Remove warn_for_unimplemented_methods (#28613) Cyrus Leung 2025-11-14 11:10:10 +08:00
• b39a5026eb [ci][amd] fix basic models extra init test (#28676) Bradley D 2025-11-13 18:44:36 -08:00
• 622e6106a9 [CPU][Bugfix] Fix Apple Silicon M1 compilation failure (#28681) Michael Goin 2025-11-13 20:49:55 -05:00
• 2aa75c752b [ROCm] Bump up the version of amd-smi to 6.4.3 (#28680) Sage Moore 2025-11-13 17:24:28 -08:00
• 4d5943bda6 [quantization][config] enable override existing quant_config (#28510) Hank_andMichael Goin 2025-11-14 09:24:10 +08:00
• f2b8e1c551 Mirrored test group definitions for AMD (2025-11-11) (#28573) Alexei-V-Ivanov-AMD 2025-11-13 18:16:34 -06:00
• 6e25b1cddf [KV Connector] Test async mode in scheduler tests (#28550) Mark McLoughlin 2025-11-13 23:30:59 +00:00
• e64011f29a [CI] Bug: Fix ci entrypoint pooling (#28684) Wentao Ye 2025-11-13 17:19:35 -05:00
• 1b622deba7 [Misc] Update CODEOWNERS for simon-mo and comaniac (#28675) Simon Mo 2025-11-13 13:01:43 -08:00
• faed7bf07e [Bugfix] [CPU] bump torch to 2.9.0 for Darwin to fix segmentation fault (#27791) KebeandMichael Goin 2025-11-14 05:48:08 +09:00
• 262d263f6c [Bugfix] Eliminate tuple inputs to submodules in graph partitioning (#28533) Yanan Cao 2025-11-13 12:09:05 -08:00
• 968060c15a [bugfix] correct local_chunk_len for DCP in reorg_kvcache with long context (#28526) Qiuandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 2025-11-14 03:29:22 +08:00
• 5d6ce2b960 [Perf] Support stream interval for reducing host overhead (#27869) elvischenvandNick Hill 2025-11-14 02:21:25 +08:00
• f9f3b596f3 [Attention][Bugfix] Fix FA sink support (#28660) Matthew Bonanni 2025-11-13 12:20:01 -06:00
• 119c4927b3 [Bugfix] Fix validate model input for decoder models (#27099) 4 people 2025-11-13 19:18:47 +01:00
• fe1cd7704d [Performance][B200] silu_mul_quant: pack scales in int32 (#28358) Varun Sundar RabindranathandVarun Sundar Rabindranath 2025-11-13 13:16:55 -05:00
• fdfd5075aa [TPU] patch TPU wheel build script to resolve metadata issue (#27279) Johnny Yang 2025-11-13 09:36:54 -08:00
• 327c0a9a23 [BugFix] Ensure EngineArgs.create_engine_config is idempotent (#28515) Nick Hill 2025-11-13 09:14:08 -08:00