iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,273 Commits 3 Branches 0 Tags
0c5dd18e7a0ace18d99438fb16b8375572ea76b5
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick HillandClaude Opus 5.5 0c5dd18e7a [MRV2] Collect KV transfer results before the drafter runs (#62041) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-17 10:27:57 -07:00
Nick HillhjjqClaude Opus 5.5
 		de7d70676d [Perf] Low-latency CPU all-reduce for DP metadata syncs (#61767) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: hjjq <50634613+hjjq@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-15 15:29:13 -07:00
Nick HillandIlia Yastrebov eaf25f675a [NIXL] Prepare KV transfer descriptors as NIXL 1.5 strided lists (#61032) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Ilia Yastrebov <iyastrebov@nvidia.com>
 2026-10-14 18:07:24 +02:00
Nick HillandClaude Opus 5.5 78b32b3fdc Revert "[Build] DeepGEMM migration to torch Stable ABI (#59482)" (#61510) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-13 15:33:50 -07:00
Nick Hill 3fe917abb1 [Bugfix][MRV2] Fix FlashInfer attention and ReplaySSM warmups on MRV2 (#61013) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-11 16:04:41 -07:00
Nick Hill c059bee812 [MRV2] Switch CI tests still running on MRV1 unnecessarily (#61014) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-11 09:57:07 -07:00
Nick Hill f8471164bf [MRV2] Detatch weight-reload logic from MRV1 (#60937) 2026-10-10 21:15:05 -07:00
Nick Hill 7d0b4e57aa [ROCm] Remove MRV1 model exceptions (#56360) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-10 05:09:19 +00:00
Nick Hill 7d1d8660d6 [MRV2] Miscellaneous code structure updates (#60754) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-10-09 17:58:09 -07:00
Nick HillvllmellmClaude Opus 5.5
 		ed7ae705a1 [Bugfix][MRV2] Use int64 index mappings to fix row-offset overflow (#60736) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-09 14:38:11 -07:00
Nick HillandClaude Opus 5.5 c055c1e075 [Core] Include the LoRA path in prefix-cache block hashes (#59335) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 08:37:09 +08:00
Nick HillandClaude Opus 5.5 105a4e097b [Bugfix][Frontend] Reject LoRA adapters named after a served model (#59286) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 18:29:38 -07:00
Nick HillandRobert Shaw 42dbe4578c [Core] Rework scheduler skipped_waiting queue (#58947) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-09-28 20:03:16 +00:00
Nick HillNicolò LucchesiClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Yongye Zhu
 		e7c903609f [Perf][MRV2] Allow FULL decode graphs for one-token prompt tails (#58400) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-09-28 12:48:52 -07:00
Nick HillandKimi Code de2d478d9a [Bugfix][Frontend] Document 404 response for /generative_scoring (#58788) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-25 14:08:00 -07:00
Nick Hill 234df712ed [MRV2] Minor model_runner.py code cleanup (#58610) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-25 07:59:27 -07:00
Nick HillandClaude Opus 5.5 7dbd0a8d22 [Bugfix][MRV2] Treat padded prompt tails as spec-decode rows for hybrid models (#58434) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 18:29:53 -07:00
Nick Hill 36f94d5fe8 [Bugfix][Outlines] Fix EOS termination and unconstrained masks after rejected drafts (#58612) 2026-09-24 21:55:56 +00:00
Nick HillandKimi Code e30559b58f [Core] Skip JIT monitor when JIT warmup is disabled (#58590) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-24 12:08:16 -07:00
Nick HillandKimi 074d57fce1 [Bugfix] Keep JIT warmup under enforce-eager when fault tolerance is on (#58593) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi <noreply@moonshot.cn>
 2026-09-24 12:07:32 -07:00
Nick Hill 09fe178dba Revert "[DSpark] Support pipeline-parallel targets in aggregated serving (#56956)" (#58484) 2026-09-24 08:48:36 -07:00
Nick Hill 310f15d354 [Bugfix][MRV2] Align dummy idx_mapping dtype to avoid runtime jit (#58462) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-23 16:54:48 -07:00
Nick Hill afec265bca [MRV2] Miscellaneous code cleanup (#57980) 2026-09-23 01:51:52 -07:00
Nick HillKimi CodeRoger Wang
 		826e300cde [Bugfix] Skip VllmConfig re-validation for with_hf_config submodel views (#58212) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Roger Wang <rogerw@inferact.ai>
 2026-09-23 00:50:17 +00:00
Nick Hill 5efbf0886a [CI] Support FORCE_COLOR env var; enable colors in CPU CI output (#56913) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-16 22:40:21 -04:00
Nick Hill f2aad6aa70 [MRV2] Buffer util simplifications (#56888) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-15 20:36:51 +00:00
Nick Hill 7dbc3d62c3 [Cleanup] Remove vestigial tpu_input_batch.py (#56898) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-14 21:29:15 -07:00
Nick Hill f84b0c4bce [Model Runner V2] Tolerate lack of pinned memory support (#56908) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-14 21:28:32 -07:00
Nick Hill ceaa9f95dd [Minor] Invert MRV1 specdec method fallback logic (#56899) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-14 18:56:04 -07:00
Nick HillandAndreas Karatzas cc5dd0a857 [MRV1] Scope breakable cudagraphs to the piecewise path only (#56312) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-11 19:55:34 +00:00
Nick Hill e6cb56337b [Core] MRV2 support for fast-prefill (#56145) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-10 10:36:04 -07:00
Nick Hill 4aadb0f14d [Core] Enhance cpu<->gpu sync checking to include paged async copies (#53491) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-08 17:26:21 -07:00
Nick HillRoger WangKimi Code
 		6cbb3c154e [Perf][GDN] Build cudagraph-capture metadata without a device sync (#55404) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <rogerw@inferact.ai>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-04 17:40:17 -07:00
Nick Hill 8277c42e4c [Perf] Ensure async h2d copies are pinned in more places (#55202) 2026-09-04 17:17:01 -07:00
Nick Hill e55b93f296 [Core] Deprecate "all" mamba cache mode (#55041) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-03 08:56:59 -07:00
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> bf95f58d10 [Core] Triton kernel for small-batch top-p only masking (#54651) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-03 07:52:47 -07:00
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> a566ea7e8f [Perf] Avoid more h2d copies from non-pinned tensors (#54660) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-01 20:37:09 -07:00
Nick Hill a56e74afd6 [CI] Remove MRV2-specific tests (#54823) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-01 14:15:12 -07:00
Nick Hill c28feab989 [Core][MRV2] Freeze gc during V2 CG capture; skip per-descriptor cleanup (#54646) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-31 19:37:46 -07:00
Nick Hill 6c18a54648 [Perf] Avoid h2d copies from non-pinned CPU tensors (#54299) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-29 12:06:30 +08:00
Nick HillandKimi Code 96242aa50d [Bugfix][MRV2] Release layer-bound KV cache memory in shutdown() (#54246) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
 2026-08-28 21:27:09 +00:00
Nick HillandKimi Code 70732942c6 [Bugfix][ROCm] Pre-allocate wvSplitKrc static workspaces before KV init (#54247) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
 2026-08-28 14:22:53 -07:00
Nick Hill 8cfba696ee [Misc] Separate adaptive verification config validation (#54108) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-27 20:03:01 -06:00
Nick Hill 4aab2b0ebe [Model Runner V2] Use MRV2 for all models by default (#53183) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-27 11:59:34 -07:00
Nick Hill 76c0c6530e [Bugfix][Scheduler] Don't pad spec decode up to max_model_len (#53962) 2026-08-27 08:30:22 -07:00
Nick Hill e6cc0899b6 [Bugfix][MRV2] Run cudagraph memory profiling in a throwaway graph pool (#53682) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-25 18:52:15 +00:00
Nick Hill 09fddeb4ce [Bugfix] BailingMoeV3 KDA: skip absent metadata during CUDA graph profiling (#53593) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-25 01:02:50 +00:00
Nick HillAnh Trananhtra3889
 		a4d70bef37 [Model Runner V2] Reserve CUDA graph memory (#53306) 
Signed-off-by: Anh Tran <anh.tran.3889@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Anh Tran <anh.tran.3889@gmail.com>
Co-authored-by: anhtra3889 <anhtra@nvidia.com>
 2026-08-23 23:11:19 -07:00
Nick Hill ee11730751 [BugFix] Revert incorrect MM keep_on_cpu=True changes (#52881) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-19 07:07:40 +00:00
Nick Hill deeeae75d0 [MM] Keep more metadata tensors on CPU (#52827) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-18 19:56:33 -07:00
Nick Hill 8efa13b700 [Bugfix] Pick the DeepSeek V4 eager cudagraph region per model runner (#52401) 2026-08-15 21:04:08 -07:00
Nick HillandClaude Opus 4.8 615d4cfade [Core] Check for GPU<->CPU syncs during CI (#43107) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-14 17:42:05 -07:00
Nick Hill d6f17f3d53 [MRV2] Support attention-free models (#52374) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-14 23:54:50 +00:00
Nick Hill 3e3ceb1961 [Perf] Avoid more GPU<->CPU syncs in multimodal encoders (#52369) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-14 10:40:14 -07:00
Nick Hill f9538af28c [Core] Clearer comments in BlockPool.free_blocks() (#52076) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-12 20:36:08 -07:00
Nick Hill f97e502969 [Perf] Avoid more GPU<->CPU syncs on the model execution path (#51738) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-11 19:30:34 -07:00
Nick HillRahul ChalamalaJanelle Cai
 		0a94d85a66 [Bugfix][MRV2] Require all requests to be decoding for uniform-decode dispatch (#51865) 
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
 2026-08-11 17:15:06 -07:00
Nick Hill 513f83e7ec [Bugfix] Take the sliding window from the layer, not the KV cache group (#51756) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-11 09:10:46 -07:00
Nick Hill fbff187d59 [Core] Make the GPU sync check thread-local and fix its suppressors (#51455) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-08 17:13:40 -07:00
Nick Hill 9b0afeb4f6 [Perf] Avoid some more unnecessary GPU<->CPU syncs (#51458) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-08 17:13:26 -07:00
Nick HillRahul ChalamalaJanelle Cai
 		d608dfabfd [Bugfix][MRV2] Reserve spec-decode lookahead blocks in V2 warmup (#51438) 
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
 2026-08-08 23:58:43 +00:00
Nick HillandClaude Opus 5 9e6be4a72b [Perf][Sparse MLA] Drop the atomic contention in the index remap (#50365) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-08 16:32:40 -07:00
Nick Hill 9823714346 [Bugfix] Drop stale layer kwarg from online MXFP4 kernel creation (fix precommit) (#51442) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 13:49:00 -07:00
Nick Hill ac70ce96e0 [Frontend] Disable uvicorn signal handlers instead of racing them (#50916) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 13:40:24 -06:00
Nick Hill b1e12d142d [V1] Copy NaN-in-logits counts to host asynchronously (#51304) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 00:05:03 +00:00
Nick Hill c5d470ac4c [ModelRunner V2] Minor indexing optimizations (#51210) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-06 14:44:13 -07:00
Nick Hill 77b519912b [CI][Bugfix] Fix test_shutdown_on_engine_failure startup deadlock (#51050) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-04 20:29:35 -05:00
Nick HillandClaude 7635a9002b [Core] Explicitly manage torch CPU threads in workers (#49919) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-04 10:06:38 -07:00
Nick HillandClaude 74295e3bd4 [CI][Bugfix] Fix flaky test_store_orders_after_compute_write (#50926) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-04 00:54:49 +00:00
Nick HillesmeetuYifan Qiao
 		4e582c5b54 [PD][Bugfix] Rebase KV lease deadlines onto worker clock (#50326) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: esmeetu <jasonailu87@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-07-30 01:25:42 +00:00
Nick HillandClaude Opus 5 437e0b7f8e [BugFix] Fix P/D preemption race condition (#50297) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-07-30 01:11:18 +00:00
Nick Hill 2ecd8645d8 Revert "[Misc][Minimax-M3]add default video_processor (#50092)" (#50313) 2026-07-29 13:46:07 -07:00
Nick Hill 82553692b8 [KV Connector] Fix NIXL mamba state pairing for multi-slot block tables (#50153) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-07-29 09:45:53 -07:00
Nick HillChris EastwoodYifan Qiao
 		a0c092ee72 [BugFix] Fix num_output_placeholders preemption underflow (#48245) 
Signed-off-by: Chris Eastwood <chris.eastwood@pwn4g3.dev>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Chris Eastwood <chris.eastwood@pwn4g3.dev>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-07-29 06:41:54 -07:00
Nick HillandJared Wen f37f03db4a [KV Connector] Support NIXL P/D for hybrid MLA+SSM models (#49762) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Jared Wen <w13431838023@gmail.com>
 2026-07-29 11:50:30 +08:00
Nick HillandJeff Ma 6fbbcf2151 [BugFix] Stop dummy runs from writing mamba state through stale block-table rows (#49757) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: Jeff Ma <jeffjma@umich.edu>
 2026-07-29 01:09:41 +00:00
Nick HillandAndreas Karatzas 6453fc0b8c [Bugfix] Don't reuse engine core payload buffer while zmq is sending it (#50053) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-28 16:09:28 +00:00
Nick Hill 0d0504b54c [Core] Warm up runner-owned Triton kernels before the first request (#49903) 2026-07-28 07:58:02 -07:00
Nick HillandClaude Opus 4.8 94100b5915 [CI] Wire untethered test files into CI jobs (#49340) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-28 14:02:31 +00:00
Nick HillandClaude Fable 5 601fa9a74e [KV Connector] Support NIXL heterogeneous P/D block sizes for hybrid models (#49612) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-28 06:45:02 -07:00
Nick Hill d223c900d8 [Bugfix] Only pad transformers backend value when it is narrower (#50060) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-07-27 23:36:45 -05:00
Nick HillandWoosuk Kwon 8112b6c997 [MRV2] Always build attn metadata at capture time (#49364) (#49995) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-07-27 17:25:17 +00:00
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 59a6b0411d [Core] Fix internal LB load-balancing (#49204) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-27 14:00:46 +00:00
Nick Hill 74d3b799e1 [Bugfix] Fix mHC block-M prenorm GEMM cross-row reduction carry-over (#49429) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-07-27 04:42:47 +00:00
Nick HillandBenjamin Chislett 439f336212 [Core] Fix gpu<->cpu syncs in MRV2 mamba_hybrid.py (#49736) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-07-27 02:41:27 +00:00
Nick Hill 50aa830482 [BugFix][MRV2] Don't create dummy requests longer than max_model_len (#49751) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-07-27 02:04:20 +00:00
Nick HillandTaneem Ibrahim fdaa0d9e59 [ModelRunner V2] Support encoder-only attention (#49331) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-07-27 00:38:44 +00:00
Nick HillandClaude Opus 4.8 46f01a50ac [CI][Bugfix] Fix test isolation in block_int8/ptpc_fp8 MoE kernel tests (#49609) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-23 12:26:02 -05:00
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 521aa80f71 [Core] Simplify KVBlockZeroer index tensor handling (#48399) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-23 18:12:46 +08:00
Nick Hill 229e01e9e1 [BugFix] Handle per-group prefix-hit divergence for hybrid models with KV connector (#48425) 2026-07-22 17:19:11 -07:00
Nick Hill 910cc8543a [Bugfix] Restore gather_and_maybe_dequant_cache OOB guard (#49427) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-07-22 13:47:37 -07:00
Nick Hill 431934522b [CI] Fix stale/fragile untethered kernels-root tests (#49423) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-07-22 14:43:07 -06:00
Nick Hill b0d7875180 [CI] Increase timeout of pytorch-compilation-unit-tests (#49450) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-07-22 15:39:09 +00:00
Nick HillandClaude Opus 4.8 a1c15bcb0f [CI][Bugfix] Fix and wire streaming-input tests (#49356) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-22 00:35:23 +00:00
Nick Hill 4809de7317 [Misc] Fix terminal output logo coloring (#49344) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-07-22 00:21:29 +00:00
Nick Hill 08e5067561 [CI] Bump timeout of entrypoints-integration-api-server-openai-part-2 (#49359) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-07-21 14:54:03 -07:00
Nick HillandClaude Opus 4.8 1dca300653 [CI] Fix and wire encoder/manager cudagraph unit tests (#49339) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-21 20:41:16 +01:00
Nick HillandClaude Opus 4.8 de6ec294ef [Bugfix] Fix DSA crash under breakable piecewise cudagraphs (#49302) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-21 18:28:13 +01:00
Nick HillandClaude Opus 4.8 5aab491bc9 [CI] Wire tests/models/inkling into a B200 job (#49325) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-21 15:43:20 +01:00
Nick Hill af91f4b3e4 [Cleanup] Remove unused StructuredOutputRequest.status field (#49235) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-07-21 00:06:16 +00:00