Skip to main content
OrchestKit v8.58.0 — 112 skills, 37 agents, 211 hooks · Claude Code 2.1.183+
OrchestKit
Skills

Audit Activation

Audits OrchestKit sub-agent activation from real spawn telemetry — computes the generic-vs-specialist spawn split, flags dormant agents (never fired), and classifies each as fires/mis-triggered/niche. The agent-side analogue of audit-skills. Use when specialized agents feel under-used, before pruning the catalog, or after wiring new agent spawn paths.

Reference medium

Auto-activated — this skill loads automatically when Claude detects matching context.

Audit Activation Audits OrchestKit sub-agent activation from real spawn telemetry — computes the generic-vs-specialist spawn split, flags dormant agents (never fired), and classifies each as fires/mis-triggered/niche. The agent-side analogue of audit-skills. Use when specialized agents feel under-used, before pruning the catalog, or after wiring new agent spawn paths.

audit-activation

Reports whether OrchestKit's specialized sub-agents are actually being activated, from real spawn telemetry — not vibes. The agent-side analogue of audit-skills (which audits skill quality; this audits agent activation).

It answers: "Do my specialized agents get spawned, or does the model default to generic Explore/general-purpose?" Read-only — it never edits agents.

Why this exists: a 2026-06 audit found only ~14% of agent spawns hit the 37-agent catalog vs ~74% generic, with 17/37 agents dormant — and that agents fire ~1:1 with how often a high-traffic skill names them via subagent_type=. Description rewrites ("use proactively") were A/B-tested and gave Δ0, so this skill measures wiring + usage, not description prose. See docs/feat--activation-audit/.

Quick Reference

CategoryFileImpactWhen to Use
Activation Checks$\{CLAUDE_SKILL_DIR\}/rules/activation-checks.mdHIGHWhat to compute per agent
Classification$\{CLAUDE_SKILL_DIR\}/rules/activation-status.mdHIGHfires / mis-triggered / niche / dead buckets
Output Format$\{CLAUDE_SKILL_DIR\}/references/output-format.mdMEDIUMReport layout + the spawn-split summary

CRITICAL: Task Management is MANDATORY (CC 2.1.16)

TaskCreate(subject="Audit activation: agent spawn analysis",
  description="Computing generic-vs-specialist split + dormancy from spawn telemetry",
  activeForm="Auditing agent activation")
TaskCreate(subject="Read spawn telemetry", activeForm="Reading subagent-spawns.jsonl")
TaskCreate(subject="Compute split + dormancy", activeForm="Computing split and dormant agents")
TaskCreate(subject="Classify & render", activeForm="Classifying agents and rendering report")
TaskUpdate(taskId="3", addBlockedBy=["2"])
TaskUpdate(taskId="4", addBlockedBy=["3"])

Workflow

  1. Inventory — Glob src/agents/*.md (exclude README/INDEX/CONTRIBUTING) for the catalog.
  2. Read telemetry — the FRESH stream is .claude/logs/subagent-spawns.jsonl (writers: pretool/task/spawn-intent-logger + subagent-start/subagent-validator). The legacy ~/.claude/analytics/agent-usage.jsonl is dead (orphaned in a refactor) — see edge cases.
  3. Compute — run all checks from Read("$\{CLAUDE_SKILL_DIR\}/rules/activation-checks.md"): spawn split (generic / ork-catalog / other-plugin), per-agent fire counts, never-fired set, concentration (top-5 %).
  4. Classify — apply Read("$\{CLAUDE_SKILL_DIR\}/rules/activation-status.md"): each dormant agent → mis-triggered (wired but no real spawn path) / niche (legit rare) / dead (no references anywhere).
  5. Render — output using Read("$\{CLAUDE_SKILL_DIR\}/references/output-format.md").

Quick Start

bash "${CLAUDE_SKILL_DIR}/scripts/run-activation-audit.sh"

The script reads the spawn telemetry, joins it against src/agents/, and prints the split, top agents, never-fired list, and concentration. Then apply the classification rules to bucket the dormant agents (this step needs the model: it greps skills/agents for each dormant agent's real spawn path).

Key Decisions

DecisionRecommendation
Telemetry sourceUse .claude/logs/subagent-spawns.jsonl (fresh); ignore the dead agent-usage.jsonl
"never fired" caveatAbsence in the telemetry window is a strong signal, NOT proof of zero — state the window
Fix directionDormancy ⇒ wire a subagent_type= spawn from a high-traffic skill; do NOT rewrite descriptions (A/B Δ0)
Prune thresholdOnly "dead" (no references in any skill/agent) is a prune candidate; niche stays

Chain

After this audit, run the deeper experiment if you suspect descriptions: docs/feat--activation-audit/agent-routing-experiment.mjs (isolated A/B selection harness).

  • audit-skills — quality audit for skills (this skill's sibling, agent-side)
  • telemetry-inspect — validates the telemetry data-plane this skill reads from
  • analytics — raw usage queries across projects
  • doctor — broader plugin health (manifests, hooks, memory budget)

Rules (2)

Activation Checks — HIGH

Activation Checks

What to compute for the activation audit. All counts come from the FRESH spawn stream .claude/logs/subagent-spawns.jsonl (NOT the dead ~/.claude/analytics/agent-usage.jsonl).

1. Spawn split (the headline)

Classify every spawn's subagent_type into one of three buckets:

  • generic CCExplore, general-purpose, Plan, workflow-subagent, statusline-setup
  • ork catalog — name (minus any ork: prefix) matches a file in src/agents/
  • other-plugin — anything else (candlekeep-cloud:*, claude-code-guide, hq-ext:*)

Report each as count + % of total. A healthy catalog is NOT dominated by generic.

Incorrect — counting only ork spawns and reporting "20 agents fired" with no denominator (looks fine; hides that generic did 74%).

Correctgeneric 690 (74%) · ork 130 (14%) · other 107 (11%) — the ratio is the finding.

2. Per-agent fire counts + never-fired set

Tally ork-catalog spawns by agent. Then never-fired = inventory − fired. List both.

3. Concentration

top-5 agents / total spawns as a %. High concentration (e.g. top-5 = 83%) means a few agents carry the catalog and the rest are dead weight or mis-wired.

4. Skill-naming cross-check (the root-cause signal)

For each agent, count how many src/skills/**/SKILL.md reference it via a subagent_type= spawn for ork:<name> (or an agent: <name> field). Agents fire ~1:1 with this count — an agent named by 0-1 skills will be dormant regardless of its description. This is what distinguishes "mis-triggered" from "niche".

grep -rl "subagent_type=\"ork:${name}\"\|subagent_type: ork:${name}" src/skills/ | wc -l

Activation Status Classification — HIGH

Activation Status Classification

Bucket each agent. Fired agents are ACTIVE. Every dormant (never-fired in the window) agent goes into exactly one of the buckets below — decided by its references, not its description.

Buckets

BucketConditionAction
ACTIVEfired ≥1× in the telemetry windownone — it works
MIS-TRIGGEREDdormant, BUT only referenced as a table row / narrative mention / deep-team member — has a real subagent_type= spawn in 0-1 high-traffic skillswire a spawn from a busy skill (fix)
NICHEdormant, legitimately specialized + rarely-needed domain (design, multimodal, perf, IaC, security-LLM), and is wired into at least one relevant skillkeep — low use is expected
DEADdormant AND no references in ANY skill or agent (truly unreachable)prune candidate

Definitions (apply before the procedure)

  • RARE-DOMAIN set (low use is inherently expected): design (system/context/tokens), multimodal/media, frontend-perf, IaC/infra, deployment, security-LLM/AI-safety, demo, UI-annotation. An agent whose primary domain is in this set is a NICHE candidate.
  • WIRED = referenced by ANY skill or agent in a way that can surface it: a real subagent_type=/agent: spawn, OR membership in an agent-team / agent-selection map / escalation table. Narrative-only prose mentions do NOT count as wired.
  • REAL BUSY-SKILL SPAWN = a subagent_type=ork:<name> (or agent: <name>) issued by a dispatchable skill — NOT one whose frontmatter is user-invocable: false + disable-model-invocation: true (a doc skill can't dispatch).

Decision procedure (per dormant agent — STOP at the first match)

  1. Count references: grep -rl "ork:<name>\|<name>" src/skills/ src/agents/. If 0 references anywhere → DEAD (prune candidate).
  2. Is its domain in the RARE-DOMAIN set AND is it WIRED?NICHE (keep — low use expected). Check this BEFORE step 3: a rare-domain agent is niche even if its only wiring is team/selection-map membership, not a busy-skill spawn.
  3. Otherwise (non-rare domain) — has refs but no REAL BUSY-SKILL SPAWNMIS-TRIGGERED (wire one from a high-traffic skill).
  4. Otherwise (non-rare domain WITH a real busy-skill spawn but still dormant) → MIS-TRIGGERED if the spawn path is itself rarely-run; note the path.

Ordering matters: step 2 (rare-domain NICHE) is checked before the mis-triggered test, so niche-domain agents (IaC, multimodal, security-LLM, frontend-perf) are not mislabeled mis-triggered just because they lack a busy-skill spawn.

Hard rules

  • Never bucket on description quality. A/B testing showed description rewrites give Δ0 routing benefit — dormancy is a wiring problem, not a prose problem.
  • "Never fired" = absent from the available telemetry window, which may have gaps. State the window; treat as strong signal, not proof of zero.
  • Watch near-duplicates — two agents covering the same domain (e.g. security-layer-auditor vs security-auditor) trend toward DEAD; flag for consolidation before pruning.

Incorrect — "ai-safety-auditor never fired and its description is terse → prune."

Correct — "ai-safety-auditor never fired BUT is wired into the security-audit team and LLM-safety is a rare task → NICHE, keep."


References (1)

Output Format

Output Format

Render the activation audit as ASCII + semantic emojis (per OrchestKit visual style). Three sections.

1. Spawn split (headline)

📊 AGENT SPAWNS (N total · <window>)
  generic CC    ████████████████████████████████████░░  74%  🔴
  ork catalog   ███████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░  14%  ⚠️
  other-plugin  ██████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░  11%

2. Catalog table

AGENT                          FIRES  SKILL-REFS  STATUS
web-research-analyst             26       4        ✅ ACTIVE
monitoring-engineer               0       1        🔧 MIS-TRIGGERED
ai-safety-auditor                 0       1        🟦 NICHE
<dead-example>                    0       0        💀 DEAD

Sort: ACTIVE (by fires desc), then MIS-TRIGGERED, NICHE, DEAD.

3. Verdict line + concentration

  fired: 20/37 · dormant: 17 (fix 6 · niche 11 · dead 0) · top-5 = 83% of spawns
  ▶ Next: wire MIS-TRIGGERED agents into a high-traffic skill (subagent_type=).
    Do NOT rewrite descriptions (A/B Δ0). Prune only DEAD.

--json mode

When invoked with --json, emit \{ window, totals:\{generic,ork,other\}, agents:[\{name,fires,skillRefs,status\}], dormant:\{fix,niche,dead\} \} and skip the ASCII. Useful for chaining into a dashboard.

Edit on GitHub

Last updated on