feat(research): integrate Context7 MCP as primary source channel

Adds Context7 resolution before websearch for library/framework/API topics,
reducing reliance on web crawling for well-indexed libraries. Falls back to
websearch for unresolved libraries, concept topics, or when the user provides
starting URLs. Subagents are explicitly prohibited from calling Context7 to
prevent tool inheritance from producing duplicate or conflicting summaries.

Includes trigger and output evals for the Context7 path (resolves, fallback,
skipped for non-library topics), stale description and constraint fixes, and a
concrete "sufficient content" threshold.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-06-21 10:38:34 +00:00
parent 78db4454b0
commit 542f6ce102
4 changed files with 86 additions and 20 deletions

View File

@@ -31,6 +31,26 @@ trigger_tests:
query: "Install the neuledge context server and set it up"
should_trigger: false
- id: negative-context7-direct-question
name: "Negative — direct doc question goes to context7-mcp, not research"
query: "What are the Next.js middleware options?"
should_trigger: false
- id: negative-starting-urls-skips-context7
name: "Explicit starting URLs — Context7 step skipped, user source respected"
query: "Research fastapi at docs/refs/fastapi/ — start here: https://fastapi.tiangolo.com/llms.txt"
should_trigger: true
- id: implicit-library-context7-attempted
name: "Implicit library research — skill triggers and should attempt Context7 before websearch"
query: "Before we write the Redis skill, grab the docs and put them in .agents/skills/redis/references/"
should_trigger: true
- id: concept-topic-still-triggers-research
name: "Concept topic — research triggers but Context7 step is skipped (not a named library)"
query: "/research 'JWT authentication concepts' docs/refs/jwt/"
should_trigger: true
output_tests:
- id: sources-always-written
name: "sources.md is always produced at the output path"
@@ -79,6 +99,40 @@ output_tests:
- "Every file path in the write operations starts with .agents/skills/openai/references/"
- "No files are written to the current working directory root or any other path"
- id: context7-resolves-source-recorded
name: "Context7 source recorded in sources.md with context7: URL when library resolves"
type: deterministic
prompt: "/research next.js docs/refs/nextjs/"
expected_output: >-
sources.md contains at least one entry with a URL starting with context7:
(e.g. context7:/vercel/next.js) and a slug matching context7-*.
assertions:
- "sources.md contains a ## context7-* section"
- "The URL: line for that section starts with context7:"
- "At least one topic file lists the context7-* slug in its source_keys frontmatter"
- id: context7-fallback-to-web
name: "Fallback to websearch when Context7 does not resolve the library"
type: deterministic
prompt: "/research xyzzy-internal-tool-abc docs/refs/xyzzy/"
expected_output: >-
No context7-* source in sources.md. Skill falls back to websearch.
If no web results found, skill reports failure and does not write topic files.
assertions:
- "sources.md does not contain a ## context7-* section"
- "Skill either writes topic files sourced from web URLs or reports no sources found"
- id: context7-skipped-for-non-library
name: "Context7 step skipped when topic is not a library/framework/API"
type: deterministic
prompt: "/research 'JWT authentication concepts' docs/refs/jwt/"
expected_output: >-
No resolve-library-id call made. Skill proceeds directly to websearch.
sources.md contains only web-sourced slugs.
assertions:
- "sources.md does not contain a ## context7-* section"
- "Topic files are sourced from web URLs, not Context7"
- id: content-quality
name: "Extracted content is relevant, concise, and well-organised"
type: llm-rubric