MOVZX and Hinata
2066d3143f
[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
...
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
cut deleted while keeping the model tree
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 17:10:56 +07:00
MOVZX and Hinata
54a396586e
[Mini-VLLM] Repair test suite after SM86 cut
...
- Restore example connector, test helpers, and dropped patch tests
- Guard MLA metadata import in the example connector
- Point the PLE test at common.ops.ple after upstream #60021 move
- Cut tests for removed features: tools, DRY, epd proxy, STT batch,
diffusion scheduler, INC/Quark PLE cases, mamba-2 mixer
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 01:06:30 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
Andreas Karatzas and Claude
706791d516
[CI][AMD] Move 17 test groups from MI300 to MI355 and test large DeepSeek/Nemotron evals on AITER ( #61955 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-16 10:58:59 -05:00
Kevin H. Luu and Claude Opus 5.5
32feecd91f
[CI] Pin DeepSeek-V4.1-Flash-NVFP4 eval to its FP8-Engram revision ( #61763 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 00:05:18 -07:00
2cc2e68d1c
[Core][Mamba] Remove leftovers of mamba_cache_mode="all" ( #60043 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 00:58:59 -07:00
8faac3dd2a
[Bugfix] [Spec Decode] DeepSeek V4/V4.1 NVFP4: keep ignored MTP/DSpark experts on MXFP4 ( #59655 )
...
Signed-off-by: Shobhit Verma <shobhitv@nvidia.com >
Signed-off-by: v-shobhit <161510941+v-shobhit@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
2026-10-08 00:10:32 +00:00
f38c6778ce
[quantization] Add per-token NVFP4 quantization MoE support for ReLU2 ( #56740 )
...
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-07 22:01:09 +00:00
Benjamin Chislett and Claude Opus 5.5
dac35c5b66
[Misc] Add chat-completion options to gsm8k_eval.py CLI ( #60114 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 20:13:44 +00:00
Aarushi Jain
d2125ea681
[ROCm][CI] Raise the MI355 DeepSeek-R1 GSM8K startup wait to 1800s ( #59666 )
...
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com >
2026-10-01 22:30:13 +00:00
djramic and Andreas Karatzas
b56b54bc54
[ROCm][CI] Relax simple-nemotron-h-8b GSM8K threshold on ROCm ( #59477 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-30 19:27:21 +00:00
Andreas Karatzas
6ebb5bd64f
[ROCm][CI] Expand MI355 mirrors and route MIG-sized jobs to DPX ( #59137 )
...
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
2026-09-29 03:40:07 -05:00
Andreas Karatzas and Codex
0ae480ff1a
[CI][ROCm] Mirror large-model GSM8K evaluations on MI355 ( #58683 )
...
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-29 02:06:44 -05:00
beef814e1d
[ROCm][CI] Add missing test coverage for upstream parity ( #50519 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-28 13:12:07 -07:00
73859fec58
[Frontend] Switch Python Harmony dependency to oss-harmony ( #55128 )
...
Signed-off-by: Anton Peganov <apeganov@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-28 00:43:56 +04:00
4013f3ad6c
[ROCm] Fix CI runtime and tests for MI355 DPX ( #58244 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Mahesh Kunreddi <mahesh.kunreddi@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-24 23:15:11 -05:00
stefankoncarevic and Andreas Karatzas
e33de821c0
[ROCm][CI] skip the ROCm MRV1 default where MRV1 cannot serve the config ( #58535 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-24 15:36:36 -05:00
Andreas Karatzas and OpenAI Codex
c3f5270270
[ROCm][CI] Add the MI355 TurboQuant t3nc mirror ( #58432 )
...
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-24 02:16:09 -05:00
f69cc7c178
[Quantization][Bugfix] Bump humming-kernels to 0.1.16 ( #58054 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-09-23 08:22:45 -07:00
HDCharles
9f07d023d0
[Quantization] Enable humming wNaM asymmetric quant (zero_point) with compressed-tensors ( #46528 )
2026-09-23 10:23:11 -04:00
Andreas Karatzas and OpenAI Codex
a9f07d0bc5
[ROCm][CI] Mirror the three TurboQuant evaluation groups on MI355 ( #58282 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-23 02:37:23 -05:00
Thang Nguyen and Kimi
4ec0b95c44
[CI] Split LM Eval TurboQuant KV Cache into per-config jobs ( #57113 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Kimi <noreply@moonshot.ai >
2026-09-22 14:49:06 -07:00
78d7836525
[CI] Split (H200) LM Eval Large Models into per-model jobs ( #57965 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-21 17:19:17 -07:00
Andreas Karatzas and OpenAI Codex
82daf9f575
[CI][ROCm] Add ten AMD parity groups ( #57876 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-21 11:08:06 +00:00
7d99c2c4fd
[Feature][Humming] Humming feature integration ( #56685 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-20 08:11:01 -07:00
Kevin Li and kevinli
911548247a
[Distributed][MoonEP] BF16 integration of MoonEP balanced EP backend ( #52101 )
...
Signed-off-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com >
Co-authored-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com >
2026-09-18 10:34:01 -04:00
Kevin H. Luu
04a3a00eb7
[CI] Raise GSM8K startup max wait to 2400s for flaky 30B+ MoE eval configs ( #57361 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-09-18 03:40:54 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Lucas Wilkinson and OpenAI Codex
38ca7a899c
[Bugfix][HiSparse] Preserve per-layer offsets in KV cache bindings ( #57027 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-15 20:27:11 +00:00
241e9391ed
[Bugfix][Attention] Stabilize sparse-MLA DCP for GLM PCP evals ( #55879 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Jason Yao <wsyjh8@gmail.com >
2026-09-15 09:58:30 -07:00
Kevin H. Luu
ec6b14c044
[CI] Select the supported DCP backend for PCP eval ( #56677 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
2026-09-14 23:05:59 -07:00
b7e8dd8f37
[watermarking hardening] Add e2e test and basic GSM8K quality tests ( #56309 )
...
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-14 11:29:13 -04:00
ebe1dec2da
[PCP][DCP] Enable PCP+DCP on sparse-MLA models ( #56157 )
...
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai >
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai >
Signed-off-by: PatrykSaffer <patryk.saffer@mistral.ai >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai >
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Patryk Saffer <PatrykSaffer@users.noreply.github.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-12 23:24:44 +00:00
7ee8a6dd01
[watermarking] Dual-key gumbel-max watermarking for speculative decoding support ( #56122 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
2026-09-12 18:42:01 +00:00
d43bb2f37f
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering ( #53781 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com >
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-09-12 00:36:47 -04:00
Andreas Karatzas and OpenAI Codex
ea5f41edd7
[CI][ROCm] Restore Wikitext coverage for Qwen OCP-MX ( #55410 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-05 10:50:54 +08:00
bf7a14d307
[CI][ROCm] Add DSpark evals ( #54852 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-02 13:14:32 -05:00
Misha Goin and OpenAI Codex
872084fb77
[CI] Add Kimi-K3-pruned75-DSpark-TP4 gsm8k eval ( #54817 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-02 10:18:38 -04:00
Kevin H. Luu
1b4b2a18b8
[CI] Shard H100 MoE refactor integration tests ( #52352 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
2026-09-02 01:10:14 -07:00
Kevin H. Luu
4bf06be985
[CI] Disable CUDA graphs for GLM PCP evals ( #54745 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
2026-09-01 20:24:46 +00:00
fxmarty-amd and Andreas Karatzas
d1c15e589d
[CI] Speed up quantization test group ( #53291 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-01 13:15:24 -05:00
Joe Rowell and Misha Goin
65ce85fcdc
Add Laguna-XS-2.1-INT4 to nightly CI ( #52961 )
...
Signed-off-by: Joe Rowell <joe@poolside.ai >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-08-31 14:22:00 -04:00
+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00
7ab2923489
[Flashinfer] Upgrade Flashinfer version to 0.6.18 ( #54313 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-08-30 16:01:05 -07:00
Elvir Crnčević
f3b91f1af2
Remove wrongly added e2e test ( #54099 )
2026-08-27 20:04:17 -06:00
Qiu Chunshuo
b3af042abd
Bugfix: use PCP slot mappings for PIECEWISE capture ( #53869 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
2026-08-27 05:39:36 +00:00
ca29cd48ba
[Quantization][Humming] Support MXFP4 weight + block-FP8 activation for MoE ( #51332 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Signed-off-by: Elvir Crnčević <elvircrn@gmail.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-25 10:21:00 -04:00
Misha Goin and OpenAI Codex
bca7bea240
Remove VLLM_TEST_FORCE_FP8_MARLIN to replace with linear_backend/moe_backend ( #52182 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-18 12:58:04 -04:00
Kevin H. Luu
bb233626ca
[CI] Shard Humming A100 eval ( #52325 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
2026-08-17 02:03:33 -07:00
Yongye Zhu and Claude Opus 5
967e104fad
[Config] Unify indexer cache dtype under attention_config.indexer_kv_dtype ( #52550 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-17 10:20:04 +08:00