|
9833aa53d5
|
[Bugfix] Fail fast with a clear error when CPU offload region exceeds available space (#50358)
Signed-off-by: Alex <jihui.huang@daocloud.io>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-05 20:59:55 +03:00 |
|
|
Ronen SchafferandOr Ozeri
|
8543522ca7
|
[KV Offload] Support out-of-tree secondary tier managers via module_path (#51007)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-05 20:30:53 +03:00 |
|
|
zexplorerhj
|
d4da0c55af
|
[Model][Frontend] Add Ling 3.0 Flash BF16, MTP, and parser support (#51045)
Signed-off-by: zexplorerhj <zhjoneson@163.com>
|
2026-08-06 00:38:28 +08:00 |
|
|
|
66b3c0e61f
|
[Kernel][Model] Optimize FA4 mm_prefix range lookup (#50294)
Signed-off-by: Juqi Li <2223621784@qq.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-05 07:33:49 -07:00 |
|
|
ChaunceyandOr Ozeri
|
62c5e21621
|
[KV Offloading] Support partial-tail prefix reuse with fine-grained prefix matching (#50507)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-05 22:31:50 +08:00 |
|
|
Gabriel WuandPavani Majety
|
08b8613b7b
|
[Bugfix][Model] Fix MiniMax-M3 NVFP4 inference correctness (#48929)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Pavani Majety <pmajety@nvidia.com>
|
2026-08-05 07:25:46 -07:00 |
|
|
music-dino
|
a9b39d6dcb
|
[Bugfix] Enable chunked prefill for qwen3.5-0.8B ppl test (#51153)
Signed-off-by: Dino Music <Dino.Music@amd.com>
|
2026-08-05 13:21:43 +00:00 |
|
|
|
fd859d2e06
|
[KV Connector][Mooncake] Add store group semantics (#44956)
Signed-off-by: Schatten <czhengt@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-08-05 08:25:38 +00:00 |
|
|
MINJUN GIL
|
b92352ca2c
|
[Perf][KV Offload] Avoid quadratic ARC batch eviction (#50992)
Signed-off-by: MINJUN GIL <alswnsrlf12@naver.com>
|
2026-08-05 10:58:36 +03:00 |
|
|
Tianyu Guo
|
50c51682a1
|
[Bugfix][EC Connector] Don't stop an encoder-instance request before its images are encoded (#50275)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
|
2026-08-05 05:53:58 +00:00 |
|
|
TrainToGPBandClaude
|
eb3dce9757
|
[Bugfix][Model] Gemma3n/Gemma4: pad variable-length audio batches (#50958)
Signed-off-by: TrainToGPB <kaiba0514@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-05 12:46:00 +08:00 |
|
|
|
c25093a305
|
[Kernel] Batch invariant NVFP4 MoE using cutlass (#40372)
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-08-05 04:31:32 +00:00 |
|
|
|
41a7e7da0c
|
[KV Offload] Support partial secondary-tier load results (#50321)
Signed-off-by: Moein Khazraee <moein@nvidia.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-04 20:48:58 -07:00 |
|
|
aoshen02andyihengz
|
6153dbe036
|
[R3] Unify routed expert shape configuration (#50940)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: yihengz <yihengz@nvidia.com>
|
2026-08-04 20:20:41 -07:00 |
|
|
Divakar Verma
|
bb543efdfd
|
[ROCm][CI] Add aiter per-token FP8 quant roundtrip and RMSNorm determinism tests (#50905)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
|
2026-08-04 21:32:08 -05:00 |
|
|
Michael Goin
|
9cd9f8c20b
|
Prune redundant tests points in correctness_e2e/[test_sequence_parallel,test_async_tp] (#51068)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-08-04 19:18:50 -07:00 |
|
|
Michael Goin
|
d16e7f9da4
|
[CI] Prune PyTorch Compilation Unit Tests (#51069)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-08-05 09:53:50 +08:00 |
|
|
Nick Hill
|
77b519912b
|
[CI][Bugfix] Fix test_shutdown_on_engine_failure startup deadlock (#51050)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-04 20:29:35 -05:00 |
|
|
Yifan Qiaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
ffee32460d
|
[BugFix][Mooncake] Use global data_parallel_index for the DP engine index (#48061)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-04 17:10:51 -07:00 |
|
|
Tyler Michael SmithandOpenAI Codex
|
12292d94b2
|
[CI] Detect and fail evals on when NaNs appear in logits (#50323)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-04 23:02:58 +00:00 |
|
|
Kevin H. LuuandOpenAI Codex
|
a5149b2fee
|
[CI] Stabilize GLM-5.2 PCP evaluation (#51015)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-04 14:13:59 -07:00 |
|
|
NVShreyasandOpenAI Codex
|
8ae8337ffa
|
[Spec Decode] Enable fused non-causal TokenSpeed MLA for DSpark (#50911)
Signed-off-by: Shreyas Misra <shreyasm@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-04 18:05:40 +00:00 |
|
|
Wentao Ye
|
d31de3c421
|
[Kimi K3 Perf] option to shard the shared expert for non mega case, 16.98 GiB memory/GPU saved (#50912)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-08-04 14:00:44 -04:00 |
|
|
Nick HillandClaude
|
7635a9002b
|
[Core] Explicitly manage torch CPU threads in workers (#49919)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-04 10:06:38 -07:00 |
|
|
amitz-nvandOpenAI Codex
|
52c0e3cb08
|
[Kernel] Add support for Flashinfer Mamba SSU algorithm selection (#50157)
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-04 09:40:23 -07:00 |
|
|
|
7ac2ec7582
|
[Kimi-K3][AMD] Fuse AttnRes state updates and norms (#50593)
Signed-off-by: Liuyinfeng01 <199041580+LiuYinfeng01@users.noreply.github.com>
Co-authored-by: Liuyinfeng01 <199041580+LiuYinfeng01@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-04 12:32:13 -04:00 |
|
|
aoshen02andaoshen02
|
7153fd70f6
|
[Bugfix][MoE] Filter packed expert weights during EP loading (#49558)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
|
2026-08-04 16:32:03 +00:00 |
|
|
|
166f4e2dc3
|
fix: fuse weightless RMSNorms at their declared width (#50867)
Signed-off-by: Anuj Bolewar <anujbolewar@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Anuj Bolewar <anujbolewar@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-08-04 16:03:09 +00:00 |
|
|
Bugen Zhao
|
7743486190
|
[Frontend] DeepSeek V4 0731 reasoning effort prompts & mappings (#50580)
|
2026-08-04 08:51:32 -07:00 |
|
|
Jiangyun Zhuandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
f9c74b4b9c
|
[Mamba] enable prefix cache by default (#50991)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-04 15:20:51 +00:00 |
|
|
|
199644d410
|
[Bugfix][Attention] Guard sparse MLA masked MHA workspace (#50906)
Signed-off-by: yimdev <5779256+yimdev@users.noreply.github.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-08-04 09:59:33 -04:00 |
|
|
Andrii SkliarandAndrii Skliar
|
5789897aa4
|
[Spec Decode] Add top-k DSpark Markov projection (#49969)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
|
2026-08-04 12:52:53 +00:00 |
|
|
Harry Mellor
|
59b2fdfc4e
|
Support MLA properly in the Transformers modeling backend (#48250)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-08-04 13:24:11 +01:00 |
|
|
Lazurite
|
6a9fdf0dc7
|
[Bugfix] Resolve seq-cls num_labels from the top-level config for multimodal checkpoints (#50950)
Signed-off-by: Lazurite <43494437+Rapisurazurite@users.noreply.github.com>
|
2026-08-04 11:39:39 +00:00 |
|
|
Bowen WangandClaude Opus 5
|
adbf08d977
|
[Bugfix][Reasoning] kimi_k3: O(delta) reasoning-end check on the decode path (#50886)
Signed-off-by: abmfy <abmfy@icloud.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-08-04 10:37:03 +08:00 |
|
|
fxmarty-amdandAndreas Karatzas
|
385d4c084e
|
[ROCm][AITER] Hotfix for memory access fault errors in AITER triton MOE routing (#50859)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-08-03 21:07:46 -05:00 |
|
|
Nick HillandClaude
|
74295e3bd4
|
[CI][Bugfix] Fix flaky test_store_orders_after_compute_write (#50926)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-04 00:54:49 +00:00 |
|
|
Andreas Karatzas
|
8adc840c45
|
[ROCm][Test] Use BF16 for Jina v5 nano MTEB test (#50917)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
|
2026-08-04 08:33:05 +08:00 |
|
|
|
f42761204f
|
llmd+vllm+mori-ep(inter node wide-ep)+mori-io(write) for 2p2d with dp=ep=16 tp=1 (#45043)
Signed-off-by: Shiksha Patel <shikpate@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-04 00:13:15 +00:00 |
|
|
|
c810937573
|
[Kernel] Extend CuTe DSL skinny GEMM to GLM-5.2 (#49791)
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-08-03 16:49:41 -07:00 |
|
|
Ryan HambyandIsotr0py
|
6a9109d865
|
[Bugfix] Fix Qwen3-Omni crash on video with no audio track when use_audio_in_video=True (#48420)
Signed-off-by: RyanJHamby <ryanhamby22@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-08-03 16:22:00 -07:00 |
|
|
Karen Chung
|
f57123aa2d
|
feat(frontend): session id plumbing into requests (#48048)
Signed-off-by: Karen Chung <karenc@nvidia.com>
|
2026-08-03 22:18:58 +00:00 |
|
|
Varun Vinayak ShenoyandNick Hill
|
e578de311c
|
[ModelRunnerV2] Fix scalar Mamba state update with int32 mappings (#50327)
Signed-off-by: Varun Shenoy <varun.vinayak.shenoy@oracle.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-03 22:05:12 +00:00 |
|
|
Raphaël Rialland
|
4a3447d200
|
[Bugfix][Model Runner V2] Restore multimodal draft capability detection (#50417)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
|
2026-08-03 14:18:14 -07:00 |
|
|
aoshen02andNick Hill
|
42ab184ea7
|
[MRV2] Enable routed-experts capture (#50721)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-03 21:00:13 +00:00 |
|
|
Andreas Karatzas
|
e279f71583
|
[ROCm][Test] Fix AITER MXFP4 oracle contract (#50728)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-08-03 20:53:18 +00:00 |
|
|
Francesco FuscoandClaude
|
755513d9a2
|
[Hybrid] Stage the postprocess inputs with a single loop over the request list (#48120)
Signed-off-by: Francesco Fusco <ffu@zurich.ibm.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-03 20:24:22 +00:00 |
|
|
Vineeth Sai Varikuntla
|
7f2e78ba4d
|
[Bugfix] Emit a valid media type from encode_{audio,image,video}_url (#49056)
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com>
|
2026-08-03 19:02:23 +00:00 |
|
|
|
952694e384
|
[Bugfix] Validate NIXL speculative config compatibility (#49230)
Signed-off-by: Tzu-Ling <tzulingk@nvidia.com>
Co-authored-by: GPT-5.6 Sol <noreply@openai.com>
Co-authored-by: Cursor Grok 4.5 <noreply@cursor.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-04 02:51:04 +08:00 |
|
|
|
f0de1a604c
|
[Bugfix] Shard UniformTypeKVCacheSpecs block table width under DCP (#50823)
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-08-03 17:34:29 +00:00 |
|