iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,297 Commits 3 Branches 0 Tags
main
Commit Graph
89 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
aoshen02andClaude Opus 5.5 fe8f2140ee [Doc] Document /inference/v1/generate request/response fields and DP rank header (#62305) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 15:26:36 +00:00
aoshen02andClaude Opus 5.5 371745ebd6 [Doc][RL] Document routed experts return for MoE routing replay (R3) (#62302) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 21:11:45 +08:00
aoshen02andClaude Opus 5.5 2365177ddd [Doc][RL] Document the weight-transfer HTTP protocol for trainers without vLLM (#62267) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 09:26:56 +00:00
aoshen02andClaude Opus 5.5 237d99b4af Revert "[Docs] Add train-inference consistency agent skill" (#61551) (#62186) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-17 13:54:30 +00:00
aoshen02andClaude Opus 5.5 2d8efee9a7 [Docs] Add train-inference consistency agent skill (#61551) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-17 20:49:50 +08:00
aoshen02aoshen524Claude Opus 5.5
 		837e87e1c5 [Feature][LoRA][MoE] Expert LoRA for FlashInfer TRTLLM MXFP4 experts (#61979) 
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-17 20:17:56 +08:00
aoshen02fh3Claude Opus 5.5
 		a94c315711 [DP] Engines own wave wake-ups; drop the front-end FIRST_REQ (#59559) 
Signed-off-by: fh3 <a@b>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 19:41:54 -04:00
aoshen02Claude Fable 5.1Dakai AnDakai An
 		3ccc0a90b6 [KV Connector] Support sleep mode with MooncakeConnector over RDMA (#59625) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Dakai An <77474977+andakai@users.noreply.github.com>
 2026-10-14 04:18:47 +00:00
aoshen02Claude Opus 5.5Aaron Hao
 		c5bda658d2 [Model][MoE] Load MegaMoE experts directly into the DeepGEMM layout (#60096) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Aaron Hao <hao-aaron@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Aaron Hao <hao-aaron@users.noreply.github.com>
 2026-10-13 11:52:08 -07:00
aoshen02aoshen524Claude Opus 5.5linitra24
 		e2ad0ce5f5 [Bugfix][LoRA] Slice expert LoRA by the unpadded per-rank size under TP (#61235) 
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io>
 2026-10-13 14:25:16 +08:00
aoshen02andCodex 239a3fa3a8 [Sleep] Retain draft-private weights with existing parameter selector (#61046) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-11 23:30:56 +08:00
aoshen02 bd5bbce6e4 [RL] Release weight-transfer communicators without stopping inference (#61060) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
 2026-10-11 21:15:14 +08:00
aoshen02andOpenAI Codex 1789f09588 [Frontend] Explain disabled scale-out endpoints with opt-in hints (#61042) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-11 08:58:07 +00:00
aoshen02andClaude Opus 5.5 1dfe9fbdae [Core] Release FlashInfer allreduce workspace with --enable-nccl-comm-suspend (#59360) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 02:26:40 +00:00
aoshen02Claude Fable 5.1indianspeedster
 		a6f43e57d6 [ROCm] Enable the cuMem CUDA-graph pool offload on ROCm (#59523) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: indianspeedster <cspandey016@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: indianspeedster <cspandey016@gmail.com>
 2026-10-09 08:29:55 +00:00
aoshen02aoshen02ClaudeCodex
 		caff2b44cb [BugFix] Pick the DP world-group port at bind time via the coordination store (#51018) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-08 02:30:35 +00:00
aoshen02wang.yuqiClaude Opus 5.5
 		42ecc8eebe [CI][RL] Consolidate RL entrypoint tests under tests/entrypoints/rl (#59948) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 08:29:09 +00:00
aoshen02wang.yuqiClaude Opus 5.5
 		cbee82324c [Docs][RL] Move pause/resume and batch invariance examples to examples/rl (#59949) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 06:22:10 +00:00
aoshen02andClaude Opus 5.5 31e2443c90 [Model] LongCat-Flash: scale MLA norms while loading and drop the post-load sweep (#60080) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 08:47:02 +00:00
aoshen02andClaude Opus 5.5 9a1f6fd4cb [Model][DeepSeek-V4.1] Permute mega-attention wq_b / wo_a while loading (#60064) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:20:09 +00:00
aoshen02andClaude Opus 5.5 d09ff77637 [Bugfix] Give each data-parallel engine its own global RNG streams (#59788) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 17:38:18 +00:00
aoshen02andClaude Opus 5.5 4a30c4cad0 [Model][DeepSeek-V4] Make MegaMoE shared-expert finalize independent of linear post-load order (#59927) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 08:39:49 +00:00
aoshen02andClaude Opus 5.5 5e56e9af2a [Feature] Release the CUDA graph pool on sleep (#59160) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 01:13:10 +00:00
aoshen02aoshen02Claude Opus 5.5
 		e319f86f15 [Feature] Per-row candidate IDs for prefill token scoring (M2 of #56860) (#56984) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-03 14:17:09 +00:00
aoshen02andClaude Opus 5.5 c91dccc080 [Bugfix][Quark] Pass grouped-routing arguments to OCP MX monolithic kernels (#59752) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 13:46:48 +00:00
aoshen02andClaude Opus 5.5 8e58ac22af [Feature] Offload KV-init runtime state on sleep (#59158) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 09:49:21 +00:00
aoshen02andClaude Opus 5.5 866f02db8a [Feature] Release WorkspaceManager scratch on sleep (#59156) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 02:14:37 +00:00
aoshen02Codexfh3Claude Fable 5.1
 		4a9d6f9572 [Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs (#55935) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 14:43:38 +08:00
aoshen02andClaude Opus 5.5 643bbef55c [Refactor] Share workspace and model runner init between GPU and XPU workers (#59200) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 14:29:12 +00:00
aoshen02andClaude Opus 5.5 749192ff41 [CI] Skip the IPC weight-checker test on non-CUDA platforms (#59398) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 09:20:32 +00:00
aoshen02aoshen02Claude Opus 5.5
 		a7a9ac2745 [Core] Combine per-engine utility results like the Rust client (#59240) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 22:20:51 +00:00
aoshen02aoshen02Claude Opus 5.5
 		cd94b21264 [Bugfix][Frontend] Return 400 for malformed RL dev route bodies (#59236) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 16:46:56 +00:00
aoshen02andOpenAI Codex 4861833ae2 [Bugfix][Core] Allow AuxOutput reset after abort without running requests (#59060) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-29 09:23:32 +00:00
aoshen02aoshen02Claude Opus 5
 		b721a4c709 [Feature] Add fixed-token prefill scoring (#54335) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-28 07:50:09 -07:00
aoshen02 7f1a5398e9 Fix full logprobs in token-in/token-out responses (#58488) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-09-24 04:46:16 +00:00
aoshen02andYongye Zhu 973a3be780 [Refactor][Quantization] Make FP8 and MLA weight transforms reusable pure functions (#57732) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-09-23 01:16:20 -07:00
aoshen02 feb87b1934 [Bugfix] Narrow AuxOutput KV restrictions to known PD connectors (#58150) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-09-22 23:09:39 +08:00
aoshen02aoshen02Codex
 		639461eda4 [RL][Sleep] Retain frozen weights across level-2 sleep (#57891) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-22 12:36:55 +08:00
aoshen02andaoshen02 9ff08f9493 [Bugfix] Honor skip_reading_prefix_cache for KV connector hits (#57269) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-09-17 03:46:28 -07:00
aoshen02 b978df3299 Revert "[Frontend] Omit absent fields from /inference/v1/generate stream chunks" (#57302) 2026-09-16 23:31:59 -07:00
aoshen02andClaude Fable 5.1 8538017f4b [Frontend] Omit absent fields from /inference/v1/generate stream chunks (#57264) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-17 05:58:56 +00:00
aoshen02 0983aef8da [Frontend] Return prompt metadata from /inference/v1/generate (#53187) 2026-09-16 15:29:07 -07:00
aoshen02 eb61707851 [Bugfix] Fix stale HPC QK-norm weights after weight refit (#56706) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-09-14 21:34:41 -07:00
aoshen02Bhushan AsatiMegha Agarwal
 		84e5f3edac [Bugfix] Validate routed-expert prompt offsets before engine submission (#56844) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Bhushan Asati <bhushanasati25@gmail.com>
Co-authored-by: Megha Agarwal <19240983+meghaagr13@users.noreply.github.com>
 2026-09-15 09:17:41 +08:00
aoshen02 755541443f [Bugfix][ROCm] Mask paged attention V cache padding (#53856) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-09-08 14:39:23 +08:00
aoshen02andCodex 808f8cd3ac [Governance] Add aoshen02 as code owner for RL components (#55529) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-06 13:26:22 +00:00
aoshen02 039ea82660 [Core] Sync DP state on the first step of a wave (#52957) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-09-06 02:16:58 +00:00
aoshen02andaoshen02 7fbd44cbe0 [Bugfix][DSv4] Seed the -1 sentinel in the prefill sparse index workspace (#55299) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-09-05 18:31:04 +08:00
aoshen02aoshen02Nick Hill
 		685074cd61 [Bugfix][V2] Warm up kernels before capturing CUDA graphs (#55341) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-04 22:44:18 +00:00
aoshen02aoshen02aoshen02Codex
 		f19431e5c8 [Bugfix][MoE] Allow TRTLLM FP8 block-scale MoE with SwiGLU clamp (#55069) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen524@gmail.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-04 08:34:21 -07:00
aoshen02andaoshen02 69cf055936 [Performance][DSv4] Size dequant gather launch grid by rows (#55061) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen524@gmail.com>
 2026-09-03 20:06:34 -07:00
aoshen02Claude Fable 5.1vx120
 		560ef78bfe [Perf][Model Runner V2] Compact sampling masks on GPU instead of unpacking the full-vocab bitmask on CPU (#54901) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: vx120 <893600387@qq.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: vx120 <893600387@qq.com>
 2026-09-03 19:03:36 -07:00
aoshen02aoshen02Claude Fable 5
 		4ac452ad98 [Core] Release NCCL communicator memory in sleep mode (#51485) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-01 14:31:16 +08:00
aoshen02aoshen02Claude Fable 5
 		f4f3bcd4b4 [Test] Assert co-located RayExecutorV2 stores publish distinct ports (#54310) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-28 22:13:31 -07:00
aoshen02aoshen02ClaudeOpenAI CodexAndreas Karatzas
 		67e86d1e6f [BugFix] Bind RayExecutorV2 TCPStore before publishing its port (#50969) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-28 12:27:22 -07:00
aoshen02 17da48596c [Bugfix][DP] Synchronize the device on pause completion (#52914) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-08-26 09:50:33 -07:00
aoshen02OpenAI CodexOpenAI CodexRoger Wang
 		ee5ae2ae70 [RL] Add rank-local IPC weight updates (#52497) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-08-26 15:36:57 +08:00
aoshen02andClaude Opus 5 7ddb50788d [Bugfix][DP] Don't assume the engines started when forwarding a wake (#51481) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-18 13:49:35 -07:00
aoshen02OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7553aac77b [Frontend] Add routed-experts prompt offset (#51906) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 08:09:00 -07:00
aoshen02 4eef91c03c [Model] Support R3 capture with DeepGEMM MegaMoE (#51831) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-08-12 01:07:07 -07:00
aoshen02andCodex f2906b492e [Bugfix] Avoid repeated layerwise reload warning scans (#49505) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-08-12 13:53:50 +08:00
aoshen02andRoger Wang 99e62b802c [Bugfix][Model Loader] Defer post-load attention weight processing (#49519) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-08-10 19:16:57 -07:00
aoshen02Claude Opus 4.6mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		33584901ba [Frontend] Add content_parts to /inference/v1/generate for raw multim… (#51478) 
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 10:00:12 +08:00
aoshen02Parsa Mazaheriaoshen02Claude
 		11ba93f364 [BugFix] Use file:// rendezvous for single-node executors to eliminate startup port races (#50999) 
Signed-off-by: Parsa Mazaheri <p_mazaheri@apple.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Parsa Mazaheri <p_mazaheri@apple.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-09 22:00:04 -04:00
aoshen02andOpenAI Codex 22a175921c [BugFix] Reject invalid data-parallel RPC ports (#51469) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-08 12:23:26 +08:00
aoshen02RTCartistaoshen02Claude Fable 5
 		f7ef489e93 [Bugfix] Fix ZMQ port TOCTOU race in shm_broadcast MessageQueue (#50960) 
Signed-off-by: RTCartist <wangshengb@buaa.edu.cn>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: RTCartist <wangshengb@buaa.edu.cn>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-07 18:19:02 -07:00
aoshen02Chenglun Huferkans-amiraoshen02Claude Fable 5
 		448344c0e2 [Bugfix] Fix get_open_port() livelock on DP-reserved ports and cover get_open_ports_list (#50965) 
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Chenglun Hu <chenglunhu@gmail.com>
Co-authored-by: ferkans-amir <amir.rezaei@tu-berlin.de>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-07 22:39:33 +08:00
aoshen02andAndy Lo 8cfa01cdd2 [BugFix] Dense multinode DP rescope with regression test (#49212) 
Signed-off-by: Andy Lo <andy@mistral.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Andy Lo <andy@mistral.ai>
 2026-08-05 18:12:03 -07:00
aoshen02Matej SirovatkaRishi PuriHsiao-Yuan ChenClaude Opus 5
 		7c77868cdf [Bugfix] Size and iterate w13 by shard count for non-gated MoE (#51125) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Matej Sirovatka <matej.sirovatka@gmail.com>
Co-authored-by: Rishi Puri <riship@nvidia.com>
Co-authored-by: Hsiao-Yuan Chen <littlecircle0730@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-05 17:24:47 -07:00
aoshen02andyihengz 6153dbe036 [R3] Unify routed expert shape configuration (#50940) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: yihengz <yihengz@nvidia.com>
 2026-08-04 20:20:41 -07:00
aoshen02andaoshen02 7153fd70f6 [Bugfix][MoE] Filter packed expert weights during EP loading (#49558) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-08-04 16:32:03 +00:00
aoshen02andNick Hill 42ab184ea7 [MRV2] Enable routed-experts capture (#50721) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-03 21:00:13 +00:00
aoshen02aoshen02vx120Claude Opus 5Roger Wang
 		b09688a6e7 [Bugfix][Spec Decode] Preserve draft buffers across level-2 sleep (#49774) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Signed-off-by: vx120 <893600387@qq.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: vx120 <893600387@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-07-28 00:16:21 -07:00
aoshen02andaoshen02 9e50e1037e [Bugfix][CuMem] Make KV-cache wake cleanup tag-safe (#49857) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-07-26 12:09:13 -07:00
aoshen02andaoshen02 ba18929079 [Bugfix][SpecDecode] Scope MTP completeness checks outside bucketed updates (#49178) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-07-21 18:56:39 -07:00
aoshen02aoshen02Codex
 		d9aa35161d Update BGE-M3 token expectations for leading spaces (#49269) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-07-21 03:49:17 +00:00
aoshen02 9459fc6471 [Bugfix][RL] Set vLLM config during weight reload (#45989) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-07-20 15:02:56 +08:00
aoshen02mvanhornClaude Opus 4.6mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		dcfebf93f4 [Bugfix] Fix logprobs token-string collision from SentencePiece space… (#48674) 
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: mvanhorn <mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-20 12:17:18 +08:00
aoshen02Wojciech WaisFederico Kamelhar
 		41ea2dd44a [Bugfix][V1/V2] Fix prompt_logprobs to respect logprobs_mode (#47680) 
Signed-off-by: Wojciech Wais <wojciech.wais@gmail.com>
Signed-off-by: Federico Kamelhar <209537060+fede-kamel@users.noreply.github.com>
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: Wojciech Wais <wojciech.wais@gmail.com>
Co-authored-by: Federico Kamelhar <209537060+fede-kamel@users.noreply.github.com>
 2026-07-17 21:58:59 +01:00
aoshen02 17fdd42100 [Bugfix][Attention] Preserve post-load tensors across weight reloads (#48251) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-07-17 14:15:26 +08:00
aoshen02andOpenAI Codex 3b39fd284a [Bugfix][Spec Decode] Support heterogeneous QK fusion geometry (#48671) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-14 22:37:10 -07:00
aoshen02 3ca242d1b6 [Bugfix][R3] Exclude draft routers from expert capture (#48622) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-07-14 17:45:35 -07:00
aoshen02Claude Opus 4.8mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		83762b77b0 [Frontend] Add /abort_requests to the RLHF dev API router (#47173) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-12 14:21:02 +08:00
aoshen02Claude Opus 4.8Roger Wang
 		3e6e33526d [Disagg] return routed_experts on streaming generate responses (#44638) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-06-21 07:37:10 -07:00
aoshen02andMartin Hickey f57ac274b2 [Render] Add reasoning/tool parsing to /derender + fix byte-fallback FFFD (#45919) 
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-06-20 19:43:32 -04:00
aoshen02andClaude Opus 4.8 f87df1df9e [Bugfix][MoE] Snapshot max_cudagraph_capture_size into FusedMoEConfig (#44613) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-05 23:14:41 -07:00
aoshen02aoshen524Claude Opus 4.7
 		1fc2cee50a [KVConnector][Mooncake] Wire reset_cache cascade end-to-end (#42694) 
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Signed-off-by: Ao Shen <aoshen@inferact.ai>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 20:52:35 -07:00
aoshen02 8c79ad6580 Revert "[Core] Replace routing replay with device cache and async D2H pipeline" (#39917) (#42434) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-05-13 23:49:01 -07:00
aoshen02 006af4b956 [Bugfix] Skip routed-experts hot path when disabled (#42148) 2026-05-09 18:01:04 -07:00