chore: move skills and evals to plugins/bin, remove legacy root configs
Skills and evals migrated from .agents/ to plugins/bin/ plugin directory. Remove .mcp.json, provider-manifest.sh, and skills-lock.json legacy artifacts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
97
plugins/bin/evals/cross-cutting/neuledge-context/eval.yaml
Normal file
97
plugins/bin/evals/cross-cutting/neuledge-context/eval.yaml
Normal file
@@ -0,0 +1,97 @@
|
||||
skill_name: neuledge-context
|
||||
|
||||
trigger_tests:
|
||||
- id: explicit-install-register
|
||||
name: "Explicit trigger — install and register"
|
||||
query: "install @neuledge/context and register it as an MCP server in Claude Code"
|
||||
should_trigger: true
|
||||
|
||||
- id: explicit-package-management
|
||||
name: "Explicit trigger — package management"
|
||||
query: "install the react documentation package using neuledge context"
|
||||
should_trigger: true
|
||||
|
||||
- id: implicit-offline-docs
|
||||
name: "Implicit trigger — offline docs for AI agent"
|
||||
query: "I need React and Next.js docs available to my AI agent without web searches"
|
||||
should_trigger: true
|
||||
|
||||
- id: negative-different-mcp
|
||||
name: "Negative — different MCP server"
|
||||
query: "Add the Gitea MCP server to Claude Code"
|
||||
should_trigger: false
|
||||
|
||||
- id: negative-query-existing
|
||||
name: "Negative — querying an already-running server"
|
||||
query: "How do I query React docs using the context server that's already running?"
|
||||
should_trigger: false
|
||||
|
||||
- id: negative-cursor-setup
|
||||
name: "Negative — different provider"
|
||||
query: "Set up context serve for Cursor"
|
||||
should_trigger: false
|
||||
|
||||
output_tests:
|
||||
- id: install-script-announced
|
||||
name: "Install — script announced before running, version verified after"
|
||||
type: deterministic
|
||||
prompt: "install @neuledge/context"
|
||||
expected_output: >
|
||||
The skill announces that it will run scripts/setup-neuledge-context.sh before executing it,
|
||||
then verifies the installation by running `context --version`.
|
||||
assertions:
|
||||
- "Output mentions 'scripts/setup-neuledge-context.sh' before any install command is run"
|
||||
- "Output includes a `context --version` call after the install step"
|
||||
- "Output does not contain `npm install -g @neuledge/context@latest` (no floating @latest)"
|
||||
|
||||
- id: mcp-list-before-add
|
||||
name: "MCP registration — list checked before add, skipped if present"
|
||||
type: deterministic
|
||||
prompt: "register @neuledge/context as a Claude Code MCP server"
|
||||
expected_output: >
|
||||
The skill runs `claude mcp list` and checks for an existing 'context' entry before
|
||||
running `claude mcp add`. If already registered, the add step is skipped.
|
||||
assertions:
|
||||
- "Output includes `claude mcp list` before `claude mcp add context`"
|
||||
- "Output states that registration is skipped when the server is already present"
|
||||
- "The `claude mcp add` command uses stdio form: `claude mcp add context -- context serve`"
|
||||
|
||||
- id: auth-chmod-paired
|
||||
name: "Auth — secure-context-config.sh run immediately after auth add"
|
||||
type: deterministic
|
||||
prompt: "add auth credentials for docs.example.com to neuledge context"
|
||||
expected_output: >
|
||||
The skill runs `context auth add docs.example.com` with an environment variable reference
|
||||
for the credential, then immediately runs scripts/secure-context-config.sh.
|
||||
No credential value appears in the output.
|
||||
assertions:
|
||||
- "Output references an environment variable (e.g. $TOKEN) rather than a literal credential value"
|
||||
- "Output runs `scripts/secure-context-config.sh` in the same step as or immediately after `context auth add`"
|
||||
- "No bearer token, cookie value, or other credential string appears in the output"
|
||||
|
||||
- id: git-check-url-source
|
||||
name: "context add — git prerequisite checked for URL sources only"
|
||||
type: deterministic
|
||||
prompt: "add documentation from https://github.com/prisma/prisma using context add"
|
||||
expected_output: >
|
||||
Before running `context add`, the skill checks `git --version` because the source is a
|
||||
GitHub URL. The check is present for URL/repo sources and absent for local .db file paths.
|
||||
assertions:
|
||||
- "Output includes `git --version` before the `context add https://github.com/...` command"
|
||||
- "If given a local .db file path instead, the git check is absent"
|
||||
|
||||
- id: install-flow-quality
|
||||
name: "Full install + register flow quality"
|
||||
type: llm-rubric
|
||||
prompt: "install @neuledge/context and set it up as my Claude Code MCP server"
|
||||
expected_output: >
|
||||
A complete, ordered install-then-register flow: (1) announce the install script,
|
||||
(2) run setup-neuledge-context.sh, (3) verify with context --version,
|
||||
(4) check claude mcp list, (5) run claude mcp add if not already registered,
|
||||
(6) confirm with claude mcp list. Steps are in the correct order with verification
|
||||
between install and registration.
|
||||
assertions:
|
||||
- "Install step comes before MCP registration step"
|
||||
- "A verification command (context --version) appears between install and registration"
|
||||
- "The output would leave a user with a working @neuledge/context MCP server in Claude Code"
|
||||
- "No step is skipped without an explanation of why it was skipped"
|
||||
Reference in New Issue
Block a user