feat: skill refresh workflow — re-research, diff, propagate, re-provenance #9

New Issue

Claude · 2026-06-26T21:00:28Z

Claude commented

2026-06-26 21:00:28 +00:00

Summary

Skills built from research docs can go stale when upstream sources change. Today there is no structured way to detect drift or propagate upstream changes through the chain. This issue tracks the workflow and the staleness signal needed to support it.

Background

The provenance chain runs: research → plugin-level docs → skill. Concretely:

/research fetches upstream sources (e.g. agentskills.io) and writes topic files + sources.md to a plugin-level output directory
/skill-author reads those docs, writes the skill, and records provenance in references/sources.md + source_keys frontmatter

When the upstream source changes (e.g. the agentskills.io spec is updated), the plugin-level docs become stale, and the skill content derived from them drifts from reality.

Two gaps to resolve

1. No staleness signal

references/sources.md records what sources were used but not when. Git log can infer it, but there is no explicit timestamp in the provenance file. Without a staleness signal, there is no way to know whether a skill's sources are current without manually checking each upstream URL.

Options to consider:

Add a Researched: date field to each entry in sources.md (populated when /research runs)
Rely on git log of the research output directory (no format change, but less discoverable)
Add a top-level researched_at: frontmatter field to sources.md

2. No orchestration workflow

The refresh process today requires manually chaining four slash commands with a judgment call in the middle:

Re-run /research at the same output path (overwrites topic files and sources.md in place)
git diff the research output to identify what changed
Feed the diff as a signal into /skill-author (improving existing skill) — agent updates skill content
Re-run the provenance step: update references/sources.md contributing files and source_keys if sources changed
Run /skill-audit to validate

A /refresh-skill or /upgrade-skill workflow skill would chain these steps. The diff between old and new research output is the key signal — it maps changed doc sections to which skill instructions need updating.

Non-goals

Automated scheduled refresh — the judgment call between steps 2 and 3 requires human review; this is HITL, not HOTL
Refreshing skills not built from /research output

Prerequisite

Issue #4 (skill-author provenance) must be merged first — this workflow depends on the references/sources.md convention being in place.

Note on sequencing

Resolve the staleness signal design before building the workflow — a refresh workflow without a staleness signal re-researches everything on every run, which is wasteful and makes it hard to scope what actually needs updating.

## Summary Skills built from research docs can go stale when upstream sources change. Today there is no structured way to detect drift or propagate upstream changes through the chain. This issue tracks the workflow and the staleness signal needed to support it. ## Background The provenance chain runs: research → plugin-level docs → skill. Concretely: 1. `/research` fetches upstream sources (e.g. agentskills.io) and writes topic files + `sources.md` to a plugin-level output directory 2. `/skill-author` reads those docs, writes the skill, and records provenance in `references/sources.md` + `source_keys` frontmatter When the upstream source changes (e.g. the agentskills.io spec is updated), the plugin-level docs become stale, and the skill content derived from them drifts from reality. ## Two gaps to resolve ### 1. No staleness signal `references/sources.md` records *what* sources were used but not *when*. Git log can infer it, but there is no explicit timestamp in the provenance file. Without a staleness signal, there is no way to know whether a skill's sources are current without manually checking each upstream URL. **Options to consider:** - Add a `Researched:` date field to each entry in `sources.md` (populated when `/research` runs) - Rely on git log of the research output directory (no format change, but less discoverable) - Add a top-level `researched_at:` frontmatter field to `sources.md` ### 2. No orchestration workflow The refresh process today requires manually chaining four slash commands with a judgment call in the middle: 1. Re-run `/research` at the same output path (overwrites topic files and `sources.md` in place) 2. `git diff` the research output to identify what changed 3. Feed the diff as a signal into `/skill-author` (improving existing skill) — agent updates skill content 4. Re-run the provenance step: update `references/sources.md` contributing files and `source_keys` if sources changed 5. Run `/skill-audit` to validate A `/refresh-skill` or `/upgrade-skill` workflow skill would chain these steps. The diff between old and new research output is the key signal — it maps changed doc sections to which skill instructions need updating. ## Non-goals - Automated scheduled refresh — the judgment call between steps 2 and 3 requires human review; this is HITL, not HOTL - Refreshing skills not built from `/research` output ## Prerequisite Issue #4 (skill-author provenance) must be merged first — this workflow depends on the `references/sources.md` convention being in place. ## Note on sequencing Resolve the staleness signal design before building the workflow — a refresh workflow without a staleness signal re-researches everything on every run, which is wasteful and makes it hard to scope what actually needs updating.

Claude added this to the Kyberforge basics milestone 2026-06-26 21:00:28 +00:00

Claude added the Kind/Enhancement

labels 2026-06-26 21:00:28 +00:00

Claude referenced this issue

2026-08-30 16:50:54 +00:00

research drift: installation.md omits apm runtime status and apm runtime remove, which exist in apm 0.28.0 #119

Claude referenced this issue

2026-09-16 17:15:35 +00:00

refactor!: carry out the simplification audit across gates, tests, plugins and docs #135

Claude commented

2026-09-19 21:35:50 +00:00

Automated note from the PR #135 review follow-up. This issue's own conclusion, state and labels are untouched — this comment only records a renamed step, so the issue stays actionable.

The provenance-refresh chain has a renamed step

The /skill-audit step in the chain described here is now /factory-audit: skill-audit was merged into factory-audit (ADR-0025, commit 620f20b). Running the chain as written will fail at that step.

Nothing else in this issue was re-validated.

Co-Authored-By: Claude Opus 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01NwD8Egs5r4ndqeFLmhusX2

Automated note from the PR #135 review follow-up. This issue's own conclusion, state and labels are untouched — this comment only records a renamed step, so the issue stays actionable. **The provenance-refresh chain has a renamed step** The `/skill-audit` step in the chain described here is now `/factory-audit`: `skill-audit` was merged into `factory-audit` (ADR-0025, commit `620f20b`). Running the chain as written will fail at that step. Nothing else in this issue was re-validated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NwD8Egs5r4ndqeFLmhusX2

Claude referenced this issue

2026-09-19 21:39:45 +00:00

refactor!: carry out the simplification audit across gates, tests, plugins and docs #135

Defame1297 referenced this issue from a commit

2026-09-20 08:54:28 +00:00

docs: close the PR #135 documentation review findings

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Defame1297/holocron#9