feat(skills): add research skill for web-sourced reference file generation

Standalone /research skill that scans the codebase, discovers canonical
sources via websearch, reads and deepens in parallel via subagents, and
writes structured topic files + sources.md to an explicit output path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

.agents/evals/research/research/eval.yaml

@@ -0,0 +1,95 @@
+skill_name: research
+
+trigger_tests:
+  - id: explicit-basic
+    name: "Explicit trigger — basic invocation with topic and path"
+    query: "/research stripe .agents/skills/stripe/references/"
+    should_trigger: true
+
+  - id: explicit-with-urls
+    name: "Explicit trigger — topic, path, and starting URLs provided"
+    query: "Research the FastMCP library and put refs in .agents/skills/mcp/references/ — start here: https://fastmcp.dev/llms.txt"
+    should_trigger: true
+
+  - id: implicit-pre-skill
+    name: "Implicit trigger — looking up docs before writing a skill"
+    query: "Before we write the GitHub Actions skill, can you look up the API and put the docs somewhere useful?"
+    should_trigger: true
+
+  - id: negative-write-docs
+    name: "Negative — documenting existing code goes to write-docs"
+    query: "Document the auth module for me"
+    should_trigger: false
+
+  - id: negative-diagnose
+    name: "Negative — debugging investigation goes to diagnose"
+    query: "Research why these integration tests are failing"
+    should_trigger: false
+
+  - id: negative-neuledge
+    name: "Negative — MCP server setup goes to neuledge-context"
+    query: "Install the neuledge context server and set it up"
+    should_trigger: false
+
+output_tests:
+  - id: sources-always-written
+    name: "sources.md is always produced at the output path"
+    type: deterministic
+    prompt: "/research redis .agents/skills/redis/references/"
+    expected_output: >-
+      A sources.md file written to .agents/skills/redis/references/sources.md
+      containing at least one source section with a URL and status field.
+    assertions:
+      - "Output path .agents/skills/redis/references/sources.md is created"
+      - "sources.md contains at least one ## <slug> section"
+      - "Each section in sources.md includes a URL: line and a Status: line"
+
+  - id: topic-frontmatter
+    name: "Every topic file has required frontmatter fields"
+    type: deterministic
+    prompt: "/research stripe .agents/skills/stripe/references/"
+    expected_output: >-
+      One or more topic .md files written to .agents/skills/stripe/references/,
+      each with YAML frontmatter containing topic: and source_keys: fields.
+    assertions:
+      - "Every .md file except sources.md contains a YAML frontmatter block delimited by ---"
+      - "Every topic file frontmatter contains a topic: field"
+      - "Every topic file frontmatter contains a source_keys: field with at least one entry"
+
+  - id: no-empty-files
+    name: "No topic files written when no content is found"
+    type: deterministic
+    prompt: "/research xyzzy-nonexistent-tool-404 /tmp/refs-test/"
+    expected_output: >-
+      No topic .md files written. Skill reports what was searched and asks user
+      to provide starting URLs. sources.md may still be written if sources were attempted.
+    assertions:
+      - "Skill does not write any topic .md files to the output path"
+      - "Skill reports the search terms or URLs it tried"
+      - "Skill asks the user to provide starting URLs or confirm the topic name"
+
+  - id: output-path-respected
+    name: "All files written inside the provided output path only"
+    type: deterministic
+    prompt: "/research openai .agents/skills/openai/references/"
+    expected_output: >-
+      All written files are under .agents/skills/openai/references/. No files
+      created outside that directory.
+    assertions:
+      - "Every file path in the write operations starts with .agents/skills/openai/references/"
+      - "No files are written to the current working directory root or any other path"
+
+  - id: content-quality
+    name: "Extracted content is relevant, concise, and well-organised"
+    type: llm-rubric
+    prompt: "/research fastapi .agents/skills/fastapi/references/"
+    expected_output: >-
+      A set of topic files covering FastAPI concepts relevant to skill authoring:
+      installation, API reference, configuration, and examples at minimum. Content
+      is concise summaries, not verbatim page dumps. sources.md accurately maps
+      each source to the files it contributed to.
+    assertions:
+      - "Topic files cover concepts relevant to FastAPI (routing, request handling, dependency injection, or similar) — not generic web framework content"
+      - "Body prose in each topic file is a synthesis or extraction, not a verbatim copy of the source page"
+      - "source_keys in each topic file match slugs present in sources.md"
+      - "sources.md contributing files list accurately reflects which topic files each source contributed to"

.agents/skills/research/META.md

@@ -0,0 +1,11 @@
+```yaml
+version: "1.0"
+updated: 2026-06-21
+
+when: >-
+  Invoked when the user wants to gather structured reference documentation for a
+  tool, library, or API from web sources. Typically run before writing a new skill
+  that wraps an external tool, or any time reference files are needed for a topic.
+  Triggered explicitly ("/research <topic> <path>") or implicitly when the user
+  asks to look up, gather, or pull docs for a topic before implementing something.
+```

.agents/skills/research/SKILL.md

@@ -0,0 +1,89 @@
+---
+name: research
+description: >-
+  Use when the user wants to research a topic and generate structured reference
+  markdown files from web sources. Handles: finding canonical docs for a
+  tool/library/API, reading and deepening into linked pages, organizing extracted
+  content into topic files (overview, installation, configuration, cli-reference,
+  api-reference, examples, troubleshooting). Do NOT use when the user wants to
+  write documentation from existing code or specs (use write-docs), install or
+  query a specific documentation MCP server (use neuledge-context), or research
+  a bug/incident (use diagnose).
+metadata:
+  category: research
+allowed-tools:
+  - WebSearch
+  - WebFetch
+  - Read
+  - Write
+model: sonnet
+---
+
+<requirements>
+
+## Required inputs
+
+- **Topic** — the subject to research (tool, library, API, concept); inferred from user description if clear, ask if ambiguous
+- **Output path** — directory where reference files will be written; must be provided explicitly — do not infer or default
+- **Starting URLs** — optional; if provided, skip discovery websearch and read these first
+
+## Constraints
+
+- Never write files outside the explicitly provided output path
+- Skip any default topic file if no relevant content is found for it — do not create empty files
+- Create additional topic files beyond the default list when content warrants it (e.g. `webhooks.md`, `rate-limits.md`)
+- Subagents handle parallel source reading and link deepening — the orchestrator writes all files; subagents return summaries only, never write directly
+- `sources.md` is always written, even if only one source was read
+- Each topic file must have frontmatter with `topic` and `source_keys`; body is prose only — no inline URLs
+- Source keys in `sources.md` must be kebab-case slugs derived from the source domain or page title
+- Default topic list and file format spec live in `references/` sub-files — read them at step 1
+
+</requirements>
+
+<steps>
+
+## Process
+
+1. **Scan codebase.** Search the working directory for existing usage of the topic — imports, config files, version pins, existing reference files. Use findings to narrow research scope (e.g. target the version already in use, skip topics already documented). Read `references/topics.md` for the default topic list and `references/file-format.md` for the output file format spec.
+
+2. **Discover sources.** If no starting URLs were provided, websearch for canonical documentation (prefer `llms.txt`, developer docs, official API references over tutorials or blog posts). Collect 3–5 candidate URLs before reading any.
+
+3. **Read sources in parallel.** Spawn one subagent per source URL. Each subagent fetches the page, extracts relevant content, identifies links worth deepening, and returns a structured summary (content by topic area + links to follow). Subagents do not write files.
+
+4. **Deepen.** For each subagent that returned links worth following, spawn child subagents per branch. Continue until content becomes repetitive or out of scope. Cap at ~10 additional pages total across all branches.
+
+5. **Consolidate.** Merge all subagent summaries by topic area. Identify which default topics have sufficient content and which custom topics emerged.
+
+6. **Write topic files.** For each topic with content, write `<output-path>/<topic>.md` using the format in `references/file-format.md`. Orchestrator writes all files — never delegate file writing to a subagent.
+
+7. **Write `sources.md`.** Write `<output-path>/sources.md` mapping each source slug to its URL, description, and list of topic files it contributed to. Include sources that yielded no content, marked `no content extracted`.
+
+## Output format
+
+- `<output-path>/<topic>.md` per topic with content — formatted per `references/file-format.md`
+- `<output-path>/sources.md` — always produced; maps slug → URL, description, contributing files
+
+</steps>
+
+<checks>
+
+## Failure handling
+
+- Output path not provided — stop and ask; do not infer or default
+- No sources found after websearch — report what was searched, ask user to provide starting URLs
+- Subagent returns no usable content — skip that source, log in `sources.md` as `no content extracted`
+- All topic files would be empty — stop, report what was searched, do not write any files
+
+## Self-check
+
+- [ ] Codebase scanned before any websearch was performed
+- [ ] Output path was explicitly provided — not inferred
+- [ ] `references/topics.md` and `references/file-format.md` read at step 1
+- [ ] No topic file written without content
+- [ ] `sources.md` written with all sources read (including those with no content extracted)
+- [ ] All file writes performed by the orchestrator, not subagents
+- [ ] Each topic file has `topic` and `source_keys` frontmatter fields
+- [ ] All source keys in topic files have a matching entry in `sources.md`
+- [ ] No files written outside the provided output path
+
+</checks>

.agents/skills/research/references/file-format.md

@@ -0,0 +1,39 @@
+# Reference file format
+
+Every topic file follows this structure.
+
+## Frontmatter
+
+```yaml
+---
+topic: <topic-slug>          # matches the filename without .md (e.g. "api-reference")
+source_keys:                 # kebab-case slugs of sources that contributed; must match sources.md entries
+  - <slug>
+  - <slug>
+---
+```
+
+## Body
+
+Plain prose organized into markdown sections (`##`, `###`). Extract the content most relevant to skill authoring or implementation — not a verbatim copy of the source. Focus on:
+- Decisions that affect how to call the API or tool
+- Options, flags, or parameters with non-obvious behavior
+- Constraints, rate limits, or gotchas
+- Canonical patterns the skill should follow
+
+No inline URLs in the body — all source traceability lives in `sources.md` via `source_keys`.
+
+## sources.md format
+
+```markdown
+# Sources
+
+## <slug>
+
+- **URL:** <full URL>
+- **Description:** <one-line summary of what this source covers>
+- **Contributing files:** <comma-separated list of topic files this source contributed to>
+- **Status:** `extracted` | `no content extracted`
+```
+
+Use one `##` section per source. Slugs are kebab-case derived from the domain or page title (e.g. `stripe-api-docs`, `openai-python-sdk-readme`).

.agents/skills/research/references/topics.md

@@ -0,0 +1,17 @@
+# Default topic list
+
+Create one file per topic when relevant content is found. Skip topics with no content. Add custom topics when content warrants it.
+
+| File | Covers |
+|---|---|
+| `overview.md` | What it is, key concepts, mental model, architecture summary |
+| `installation.md` | Setup, dependencies, prerequisites, version requirements |
+| `configuration.md` | Config files, options, environment variables, defaults |
+| `cli-reference.md` | Commands, subcommands, flags, exit codes |
+| `api-reference.md` | Endpoints, SDK methods, types, request/response shapes |
+| `examples.md` | Common usage patterns, recipes, quickstart walkthroughs |
+| `troubleshooting.md` | Known issues, error codes, gotchas, workarounds |
+
+## Custom topics
+
+Create additional topic files when content doesn't fit the defaults. Examples: `webhooks.md`, `rate-limits.md`, `authentication.md`, `migrations.md`, `security.md`. Use kebab-case filenames.