--- topic: agentskills-optimizing-descriptions source_keys: - agentskills-optimizing-descriptions --- ## How triggering works At startup, agents load only the `name` and `description` of each available skill. When a user's task matches a description, the agent reads the full `SKILL.md` into context. The description carries the entire burden of triggering. Important nuance: agents typically only consult skills for tasks that require knowledge or capabilities beyond what they can handle alone. A simple one-step request may not trigger a matching skill because the agent can handle it with basic tools. Specialized knowledge — an unfamiliar API, a domain-specific workflow, an uncommon format — is where a well-written description makes the difference. ## Writing effective descriptions - **Use imperative phrasing.** "Use this skill when..." rather than "This skill does...". The agent is deciding whether to act, so tell it when to act. - **Focus on user intent, not implementation.** Describe what the user is trying to achieve, not the skill's internal mechanics. - **Err on the side of being pushy.** Explicitly list contexts where the skill applies, including cases where the user doesn't name the domain directly: "even if they don't explicitly mention 'CSV' or 'analysis.'" - **Keep it concise.** A few sentences to a short paragraph. Hard limit: 1024 characters (descriptions grow during optimization — check length). ## Designing trigger eval queries Test triggering with a set of ~20 eval queries — realistic user prompts labeled `should_trigger: true/false`. **Should-trigger queries (8–10):** vary along these axes: - *Phrasing*: formal, casual, abbreviations, typos - *Explicitness*: some name the domain directly ("analyze this CSV"), others don't ("my boss wants a chart from this data file") - *Detail*: terse prompts mixed with context-heavy ones (file paths, column names, backstory) - *Complexity*: single-step tasks alongside multi-step workflows The most useful should-trigger queries are where the skill would help but the connection isn't obvious — these are where description wording makes the difference. **Should-not-trigger queries (8–10):** use near-misses — queries that share keywords but need something different. Weak negative examples ("Write a fibonacci function") test nothing because there's no keyword overlap. Strong examples: ``` # For a CSV analysis skill: "I need to update the formulas in my Excel budget spreadsheet" # shares "spreadsheet" concept, but needs Excel editing, not CSV analysis "can you write a python script that reads a csv and uploads each row to postgres" # involves CSV, but the task is database ETL, not analysis ``` Include realistic context in all queries: file paths, personal context ("my manager asked me to..."), specific column names, casual language. ## Testing trigger rates Model behavior is nondeterministic. Run each query 3 times and compute a trigger rate (fraction of runs where the skill was invoked). A should-trigger query passes if its trigger rate is ≥0.5; should-not-trigger if <0.5. Example script structure using Claude Code: ```bash check_triggered() { local query="$1" claude -p "$query" --output-format json 2>/dev/null \ | jq -e --arg skill "$SKILL_NAME" \ 'any(.messages[].content[]; .type == "tool_use" and .name == "Skill" and .input.skill == $skill)' \ > /dev/null 2>&1 } ``` ## Train/validation split Split your ~20 queries to avoid overfitting: ~60% train, ~40% validation. Both sets must have proportional should-trigger/should-not mixes. Use only the train set to guide changes; use the validation set only to check whether improvements generalize. Keep the split fixed across iterations. ## The optimization loop 1. Evaluate on both train and validation sets. 2. Identify train-set failures: which should-trigger queries didn't? Which should-not-trigger queries did? 3. Revise the description: - Should-trigger failures → description too narrow: broaden scope, add context about when the skill applies. - Should-not-trigger false positives → description too broad: add specificity about what the skill does *not* do. - Avoid adding specific keywords from failed queries — that's overfitting. Address the general category those queries represent. - If stuck after several iterations, try a structurally different framing rather than incremental tweaks. 4. Repeat until train queries all pass or improvement stops. 5. Select the best iteration by validation pass rate — not necessarily the last iteration. Five iterations is usually enough. If not improving, the problem may be with the queries, not the description. ## Before and after ```yaml # Before description: Process CSV files. # After description: > Analyze CSV and tabular data files — compute summary statistics, add derived columns, generate charts, and clean messy data. Use this skill when the user has a CSV, TSV, or Excel file and wants to explore, transform, or visualize the data, even if they don't explicitly mention "CSV" or "analysis." ``` The improved version is more specific about capabilities (stats, derived columns, charts, cleaning) and broader about applicability (CSV, TSV, Excel; even without explicit keywords). ## Applying the result 1. Update the `description` field in `SKILL.md` frontmatter. 2. Verify it's under 1024 characters. 3. Run 5–10 fresh queries (never part of the optimization process) as a final generalization check.