iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,341 Commits 3 Branches 0 Tags
49d708e158f7b7cfbb00c26f088cd9139c0cf180
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Misha GoinClaude Opus 5.5Kevin H. Luu
 		cffc0cfa31 [CI] Fix B200 FusedMoE Layer Kernels hang on NVSHMEM IB setup (#61695) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-16 12:00:12 -04:00
Misha GoinandClaude Opus 5.5 4d9a8d6a41 [Bugfix] Make Humming DeviceInfo lookups traceable under torch.compile (#61858) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 11:12:21 -07:00
Misha GoinandClaude Opus 5.5 1c4c319792 [Frontend] Advertise --enable-x/--disable-x for boolean CLI flags (#61479) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 01:08:02 -07:00
Misha GoinClaudeKevin H. Luu
 		984c242ede [Quantization] Support compressed-tensors MXFP4 MoE with FP8 activations via Mxfp4MoEMethod (#59815) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-14 13:33:35 -07:00
Misha GoinandClaude Opus 5.5 45bb710eca [CI] Add nightly Python 3.14 smoke test (#60456) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 10:11:26 -07:00
Misha GoinSimon VeitnerClaude
 		fa39582756 [Frontend] Serve LiquidAI/d1-3B on /v1/systemone (#60676) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-12 13:00:47 -04:00
Misha GoinandClaude d79b52b182 [Misc] Consolidate csrc shared headers and prune dead code (#60721) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-12 09:46:22 -07:00
Misha GoinandClaude Opus 5.5 29e4d2d42e [Frontend][Build] Add vllm download-kernels for FlashInfer precompiled kernels (#58765) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 21:22:35 -07:00
Misha GoinandClaude 3766a67a32 [WideEP] Explain how to fix a too-old NCCL for DeepEP v2 (#60449) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-09 12:34:58 -07:00
Misha GoinandClaude 1dc60dd367 [CI] Pin pydantic in the mypy pre-commit hook (#60894) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-09 11:42:41 -07:00
Misha GoinClaude Opus 5.5Kevin H. Luu
 		ab905a885d [CI] Run GH200 test on the shared arm64 CI image (#60253) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-08 15:22:20 -07:00
Misha GoinandClaude e3a29aa9b3 [Bugfix][MoE] Size the permute scratch from its inputs (#60447) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-08 15:32:07 -04:00
Misha GoinandClaude 30c7405890 [MoE] MXFP4 oracle: add CUTLASS W4A4 backend and BF16 activation fallback (#60079) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-08 11:39:09 -07:00
Misha GoinandClaude Opus 5.5 a982c81e3e [Bugfix] Limit GlmMoeDsaForCausalLM fp8 KV cache default to SM100 (#60281) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 13:37:13 -07:00
Misha GoinandClaude 1ab18599ca [Agents] Add pre-commit check for agent files and skills (#59657) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-02 09:38:22 -07:00
Misha GoinandClaude 083060d046 [CI/Build] Add agents auto-label rule (#59641) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-01 13:28:27 -04:00
Misha GoinandClaude 362cfb72de [Agents] Expose PR checklist skill to Claude (#59638) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-01 16:54:54 +00:00
Misha Goinandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 1ee7f78e80 [Perf] Reduce redundant Triton sampler warmup specializations (#58605) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-29 21:23:27 +00:00
Misha GoinandOpenAI Codex d7f5722d7b [Bugfix][Logging] Preserve application log record factories (#58747) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-28 13:20:16 +00:00
Misha Goin 79ac28c5bd [CI] Reduce CUDA graph mode test overhead (#58749) 2026-09-26 11:22:33 -04:00
Misha GoinandCodex 71891bdb94 [Perf] Parallelize registered CUDA Triton kernel warmup at startup (#58582) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-24 19:52:00 -04:00
Misha Goin 4f72bd0152 [Bugfix][Quantization] Give LM heads standard linear metadata (#58444) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-24 16:39:49 -07:00
Misha Goin 58aa2f1ef1 [Refactor] Move auxiliary files out of the repository root (#58572) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-24 12:38:56 -07:00
Misha Goin cd06e81c41 [Bugfix][Quantization] Add Humming to the W4A8 (INT4xFP8) MoE oracle (#58427) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-24 12:37:33 -07:00
Misha Goin 89d9fd83e2 [CI] Disable JIT warmup by default in VllmRunner (#58452) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-23 18:53:58 -05:00
Misha Goin 5c3b61e9c4 [Core] Disable JIT warmup in eager mode (#58197) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-22 17:35:23 -04:00
Misha Goin 01e15bc94c [Bugfix] Backport Inductor custom-op pattern matching fix (#58189) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-22 14:26:49 -07:00
Misha Goin 25965ee583 [Kernel] Remove AllSpark INT8 W8A16 GEMM backend (#58001) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-22 14:07:34 -07:00
Misha GoinandErinYin e5fce7b56b [Core][Kernel] Share persistent workspaces for Marlin and Humming (#57421) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: ErinYin <liang.yin@daocloud.io>
 2026-09-20 17:28:33 +08:00
Misha Goin ac78445bc9 [MoE] Encapsulate TRT-LLM BF16 weight layout handling (#57405) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-17 17:47:30 -04:00
Misha Goin db7a24c230 [Perf] Add fused DFlash2 grouped convolution (#55960) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-17 21:38:59 +00:00
Misha Goin 9a5bd373cf [CI] Retire Weight Loading smoke tests (#57398) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-17 16:04:03 -04:00
Misha Goin 1085b64425 [Benchmark] Retire stale benchmarks and consolidate RMSNorm (#57083) 
Signed-off-by: Misha Goin <mgoin64@gmail.com>
 2026-09-16 13:21:43 -04:00
Misha Goin 0384e72693 [UX] Add thinking support to vllm chat (#57045) 2026-09-16 09:54:10 +01:00
Misha GoinandOpenAI Codex c6fa1f05d1 [Perf][Kernel] Add sampled filtering for persistent top-k (#56346) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-15 14:15:59 -07:00
Misha Goin 00e5cda926 [Agents] Link Triton skill to JIT kernel warmup guide (#56499) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-11 12:05:41 -07:00
Misha Goin 8a7f98c8c3 [Kernel][MoE] Optimize batched_moe_align_block_size with cooperative writes (#53280) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-11 12:05:22 -07:00
Misha Goin 8f269a93bb Revert "[CI] Remove deleted nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 and its arch aliases" (#55392) 2026-09-04 14:45:24 -07:00
Misha GoinClaude Opus 4.8ElizaWszola
 		b7624069ff [Fusion] Manual ActivationQuantFusionPass initial application (#51415) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
 2026-09-03 13:37:31 -04:00
Misha GoinandOpenAI Codex 872084fb77 [CI] Add Kimi-K3-pruned75-DSpark-TP4 gsm8k eval (#54817) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-02 10:18:38 -04:00
Misha Goin 2f01039666 [Misc] Share Buildkite CI failure skill across agents (#54806) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-01 10:22:09 -07:00
Misha GoinandOpenAI Codex 82b7d49a6e [MoE] Generalize masked activation for padded layouts (#51217) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-01 11:34:13 -04:00
Misha GoinandOpenAI Codex 5bbc58c0ad [Bugfix] Remove race in fused groupwise RMSNorm quantization (#54111) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-27 19:01:22 -07:00
Misha GoinandOpenAI Codex c9331d8064 [CI] Preserve Rust Docker cache across commits (#53290) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-25 11:13:36 -07:00
Misha Goin 7de96050c5 [Spec Decode] Enable adaptive DSpark on SM100 sparse MLA (#52783) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-25 13:31:15 -04:00
Misha Goin f620499ee3 [Bugfix][Spec Decode] Reapply group geometry for FlashAttention metadata (#53336) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-24 08:08:43 -07:00
Misha Goin 9ff7041b55 [Bugfix][Spec Decode] Use group geometry for FlashAttention metadata (#53002) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-21 17:07:19 -07:00
Misha Goin a34f2ab70a [Test] Add focused hybrid MTP prefix-cache regressions (#53189) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-21 17:05:39 -07:00
Misha GoinCodexOpenAI Codex
 		91a893de64 [Bugfix][Spec Decode] Scope DSpark backend inheritance to DeepSeek V4 (#52809) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-20 18:42:08 -07:00
Misha GoinandOpenAI Codex bfb6c13499 [Bugfix][MoE] Tune FlashInfer experts to scheduler token limit (#52989) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-20 16:26:24 -04:00
Misha GoinandOpenAI Codex 541c6d64c1 [Bugfix][Quantization] Support CT block FP8 with Marlin (#52966) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 12:41:26 -07:00
Misha GoinandOpenAI Codex f76d71d7ce [CI/Build] Fix CPU platform pre-commit formatting (#52981) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 10:32:37 -07:00
Misha GoinandOpenAI Codex 2d7f42b4f3 [Build] Add InstantTensor to CUDA dependencies (#52801) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 10:55:09 -04:00
Misha GoinandOpenAI Codex bca7bea240 Remove VLLM_TEST_FORCE_FP8_MARLIN to replace with linear_backend/moe_backend (#52182) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-18 12:58:04 -04:00
Misha Goin 7075ddac28 Support DSpark configs with architectures=DSparkDraftModel + model_type=qwen3 (#52197) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-17 12:48:30 -04:00
Misha Goin 4ab5e5012a [Refactor] Simplify B12X linear kernels and warmup (#52368) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-17 09:45:45 -07:00
Misha Goin 2e2ffd104b [CI Failure] Fix CUDA wheel build for the Kimi K3 fused MLA kernel (#52210) 2026-08-13 13:56:36 -07:00
Michael Goin 2d24355eb8 [Bugfix] Fix packed GDN decode launch for large batch-head grids (#52030) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-13 07:19:08 -07:00
Michael Goin 8e958902ee [Bugfix] Disable fine-grained prefix-cache hits for incompatible hybrid KV layouts (#51843) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-12 04:08:40 -07:00
Michael Goin ded6c452c3 [Bugfix] Support HF-config compat for Inkling (#51850) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-11 15:32:28 -04:00
Andreas KaratzasandOpenAI Codex 1ab2801dde [ROCm] Remove stale SDPA and skinny GEMM workarounds (#50907) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 14:19:16 -04:00
Michael Goinandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> e3fe212eaf [Bugfix] Generalize KV block zeroing to AttentionSpec (#51749) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 12:13:09 -04:00
Michael GoinOpenAI CodexOpenAI Codex
 		21c667aa64 [Docker] Cache test dependencies before vLLM install (#51184) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-10 10:34:59 -07:00
Michael Goin 3a749ce816 [Build] Skip precompiled wheel fetch during metadata hooks (#51424) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-10 09:38:53 -07:00
Michael GoinandOpenAI Codex c8fe1d5715 [Spec Decode] Register Qwen3.6 dSpark acceptance coverage (#51310) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-07 17:07:55 -04:00
Michael GoinandOpenAI Codex 566c80edf9 [CI] Run basic fullgraph correctness on one GPU (#51271) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-06 12:09:30 -04:00
Michael Goin e6d67fddb4 [CI] Prune PyTorch Fullgraph Test (#51074) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-05 12:27:27 -07:00
Michael GoinandOpenAI Codex 8f158d0ee2 [MoE] Share apply_moe_activation support metadata (#44359) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-05 12:05:47 -07:00
Michael GoinandShengqi Chen e8586b2878 [Build] Remove Ubuntu build-stage option from the CUDA dockerfile (#51060) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-08-05 10:50:58 -07:00
Michael Goin 9cd9f8c20b Prune redundant tests points in correctness_e2e/[test_sequence_parallel,test_async_tp] (#51068) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-04 19:18:50 -07:00
Michael Goin d16e7f9da4 [CI] Prune PyTorch Compilation Unit Tests (#51069) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-05 09:53:50 +08:00
Michael Goin 4ee9702bee Add Understanding the Latency Metrics docs (#50600) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-31 23:22:40 -07:00
Michael GoinOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		fcdc7c2e9c [CI] Organize speculative decoding E2E tests by coverage (#50330) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-31 18:05:42 -07:00
Michael GoinandOpenAI Codex 90329913e9 [UX] Reduce startup log noise (#50590) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-31 13:04:34 -07:00
Michael Goin 0eec856cc3 Add Humming indexed-MoE regression test (#50468) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-30 16:26:24 -04:00
Michael Goin 1cb3fe5844 [CI Bugfix] Temp disable Humming wNa8 INT8 H100 CI (#50329) 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-07-29 23:14:12 +00:00
Michael GoinandOpenAI Codex d6247d7173 [Spec Decode][Perf] Replicate DSpark Markov head across TP ranks (#49731) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-29 11:43:51 -04:00
Michael GoinandOpenAI Codex 275556c35c [Bugfix] Detect mixed precision in packed KV cache specs (#49623) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-23 20:07:06 -07:00
Michael Goin d65acd83d8 [Model] Support llm-compressor Inkling NVFP4 weights (#49258) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-23 20:06:01 -07:00
Michael GoinandOpenAI Codex e18f0037a5 [Bugfix][KV cache] Support sparse-MLA targets with SWA drafts (#48776) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-23 11:30:50 -07:00
Michael GoinandNick Hill ac36a7a1e7 [MRV2][Spec Decode] Avoid rejection sampler OOM by chunking (#48630) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-23 18:13:13 +08:00
Michael GoinandOpenAI Codex 191146dba5 Add quantization label automation (#49492) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-22 19:59:15 -04:00
Michael Goin 917fdb5bf7 [Bugfix] Fix DeepGEMM warmup when using FlashInferFp8DeepGEMMDynamicBlockScaledKernel (#49467) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-22 16:28:49 -07:00
Michael GoinandCodex a7d00ec051 [Bugfix] DFlash fc sized wrong when num_target_layers != num_hidden_layers (#48524) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-07-21 14:42:52 -07:00
Michael Goin b8fb56d970 [CI] Add gemma-4-E4B-it-assistant to CI gsm8k for GemmaMTP (#49243) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-21 16:51:04 -04:00
Michael GoinandJoe Rowell 47f1b47a73 Ci/add laguna xs gsm8k (#49241) 
Signed-off-by: Joe Rowell <joerowell4@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Joe Rowell <joerowell4@gmail.com>
 2026-07-21 11:30:42 -04:00
Michael Goin c7ce03bcbd [Bugfix] Bump tml-fa4 for cutlass-dsl 4.6 API compatibility (#48988) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-18 05:59:33 -07:00
Michael Goinmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick Hill
 		425c4eafb0 [Sampler] Stop upcasting logits to fp32 in apply_sampling_params (#48641) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-17 18:15:20 -07:00
Michael Goin 02c01f442b [Model] Use standard ModelOpt config for Inkling NVFP4 (#48990) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-17 18:13:14 -07:00
Michael GoinandOpenAI Codex cc25f028b7 [Loader] Improve InstantTensor loading (#46868) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-17 16:30:02 -04:00
Michael Goin c95c663049 [Quant] Add nvfp4_per_token online MoE quantization (#48538) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-16 14:25:27 -07:00
Michael GoinandClaude Opus 4.8 8bfd683901 [Spec Decode] Add kv_cache_dtype to speculative_config to control separately from target (#48787) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-15 23:49:23 -06:00
Michael Goin 3a5e88e629 [Bugfix] Fix local speculators with dots in the name from classifying as custom_class (#48754) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-15 17:58:06 -07:00
Michael GoinandClaude Fable 5 ecf4aa5ce2 [Bugfix] Fix FlashInfer non-causal draft attention (DFlash/DSpark) on Blackwell (#48167) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-15 12:44:01 -07:00
Michael Goin 0b54201a04 [CI] Build macOS arm64 CPU wheel natively on the macmini queue (#48289) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-14 19:40:26 +00:00
Michael Goin b2f7d2560a [Bugfix] Make MLA+SWA check the layer's backend, not the model config (#48520) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-14 09:53:34 -07:00
Michael GoinandClaude Fable 5 29fd688892 Add VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS and skip CuTeDSL fp4_gemm autotuning by default (#48268) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-10 18:13:08 -07:00
Michael Goin 08dfd68610 [Model] Add LongCat-Flash-Lite (n-gram embedding) (#47857) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-10 07:17:50 -07:00
Michael GoinandClaude Opus 4.8 0d12618e98 [Spec Decode] Support hybrid (SWA + full attention) DFlash drafters (#47914) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-08 11:12:45 -04:00
Michael GoinandClaude Fable 5 41de1380c2 [BugFix] Derive FlashInfer Q dtype from resolved per-group builder state (#47485) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-02 19:33:28 -07:00