Defame1297/holocron
0
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases 3 Wiki Activity

chore(kyberforge): move research docs under docs/research/ and update README

Browse Source 
Separates development-time reference material from live plugin docs.
agentskillsio/, agentsmd/, and examples/skill-write/ are not shipped with
the plugin — grouping them under research/ makes that boundary explicit.
README now lists all top-level files and explains what research/ is for.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Defame1297
2026-06-23 17:18:20 +00:00
parent 2777a834b2
commit 5ff78fc146
50 changed files with 24 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

188
plugins/kyberforge/docs/research/docs/agentskillsio/evaluating-skills.md Normal file
View File

@@ -0,0 +1,188 @@
---
topic: agentskills-evaluating-skills
source_keys:
- agentskills-evaluating-skills
---
## Designing test cases
A test case has three parts: a **prompt** (realistic user message), **expected output** (human-readable success description), and optional **input files**.
Store test cases in `evals/evals.json` inside the skill directory:
```json
{
"skill_name": "csv-analyzer",
"evals": [
{
"id": 1,
"prompt": "I have a CSV of monthly sales data in data/sales_2025.csv. Can you find the top 3 months by revenue and make a bar chart?",
"expected_output": "A bar chart showing the top 3 months by revenue, with labeled axes.",
"files": ["evals/files/sales_2025.csv"]
}
]
}
```
**Tips for test prompts:**
- Start with 2–3 test cases. Expand after seeing first results.
- Vary phrasing, formality, and detail level.
- Cover at least one edge case (malformed input, ambiguous request).
- Use realistic context: file paths, column names, personal context. "Process this data" tests nothing useful.
Don't define assertions yet — add them after seeing what the first run produces.
## Workspace structure
Each iteration gets its own directory. Each test case gets `with_skill/` and `without_skill/` subdirectories:
```
csv-analyzer/
├── SKILL.md
└── evals/
└── evals.json
csv-analyzer-workspace/
└── iteration-1/
├── eval-top-months-chart/
│ ├── with_skill/
│ │ ├── outputs/
│ │ ├── timing.json
│ │ └── grading.json
│ └── without_skill/
│ ├── outputs/
│ ├── timing.json
│ └── grading.json
└── benchmark.json
```
The only file you author by hand is `evals/evals.json`. Other JSON files are produced during the eval process.
## Running evals
Each run starts with a clean context — no leftover state. Run each eval **with the skill** and **without it** (or against a previous version as baseline).
When improving an existing skill, snapshot it before editing:
```bash
cp -r <skill-path> <workspace>/skill-snapshot/
```
Capture timing data when each run completes:
```json
{
"total_tokens": 84852,
"duration_ms": 23332
}
```
In Claude Code, the task completion notification includes `total_tokens` and `duration_ms` — save immediately.
## Writing assertions
Add assertions after seeing first-round outputs. Assertions are verifiable statements about what the output should contain:
Good assertions:
- `"The output file is valid JSON"` — programmatically verifiable
- `"The bar chart has labeled axes"` — specific and observable
- `"The report includes at least 3 recommendations"` — countable
Weak assertions:
- `"The output is good"` — too vague
- `"The output uses exactly the phrase 'Total Revenue: $X'"` — too brittle
Add assertions to `evals.json`:
```json
"assertions": [
"The output includes a bar chart image file",
"The chart shows exactly 3 months",
"Both axes are labeled",
"The chart title or caption mentions revenue"
]
```
## Grading outputs
Grade each assertion against actual outputs: PASS or FAIL with specific evidence. Evidence should quote or reference the output, not state an opinion.
```json
{
"assertion_results": [
{
"text": "Both axes are labeled",
"passed": false,
"evidence": "Y-axis is labeled 'Revenue ($)' but X-axis has no label"
}
],
"summary": { "passed": 3, "failed": 1, "total": 4, "pass_rate": 0.75 }
}
```
Grading principles:
- **Require concrete evidence for PASS.** Don't give the benefit of the doubt.
- **Review the assertions themselves** while grading — notice when assertions are always passing (too easy) or always failing (broken/too hard).
For holistic quality: try blind comparison — present both outputs to an LLM judge without revealing which came from which version. Complements assertion grading by catching differences that don't map to specific assertions.
## Aggregating results
```json
{
"run_summary": {
"with_skill": {
"pass_rate": { "mean": 0.83, "stddev": 0.06 },
"time_seconds": { "mean": 45.0 },
"tokens": { "mean": 3800 }
},
"without_skill": {
"pass_rate": { "mean": 0.33, "stddev": 0.10 },
"time_seconds": { "mean": 32.0 },
"tokens": { "mean": 2100 }
},
"delta": { "pass_rate": 0.50, "time_seconds": 13.0, "tokens": 1700 }
}
}
```
The `delta` tells you what the skill costs (more time, more tokens) and what it buys (higher pass rate).
## Analyzing patterns
- **Remove assertions that always pass in both configurations.** They inflate the with-skill pass rate without reflecting skill value.
- **Investigate assertions that always fail in both.** Either the assertion is broken or the task is too hard.
- **Study assertions that pass with-skill but fail without.** This is where the skill adds value — understand why.
- **Tighten instructions when results are inconsistent.** High `stddev` means flaky evals or ambiguous instructions. Add examples or more specific guidance.
- **Check time and token outliers.** If one eval takes 3× longer, read its execution transcript.
## Human review
After grading, review actual outputs. Assertion grading only checks what you thought to write assertions for. Record specific feedback:
```json
{
"eval-top-months-chart": "Chart is missing axis labels and months are alphabetical not chronological.",
"eval-clean-missing-emails": ""
}
```
"Missing axis labels" is actionable; "looks bad" is not. Empty feedback means the output looked fine.
## Iterating on the skill
Three sources of signal:
- **Failed assertions** — specific gaps: missing step, unclear instruction, unhandled case
- **Human feedback** — broader quality issues: wrong approach, poorly structured output
- **Execution transcripts** — why things went wrong: ambiguous instructions, unproductive steps
Feed all three — with the current `SKILL.md` — to an LLM to propose changes. Guidelines for the LLM:
- Generalize from feedback (don't add narrow patches for specific examples)
- Keep the skill lean (fewer, better instructions outperform exhaustive rules)
- Explain the why (reasoning-based instructions work better than rigid directives)
- Bundle repeated work into `scripts/` when agents reinvent the same logic each run
### The loop
1. Give eval signals + current `SKILL.md` to an LLM; ask for improvements.
2. Review and apply the changes.
3. Rerun all test cases in a new `iteration-<N+1>/` directory.
4. Grade, aggregate, and review with a human.
5. Repeat until satisfied, feedback is consistently empty, or improvement stops.

154
plugins/kyberforge/docs/research/docs/agentskillsio/examples.md Normal file
View File

@@ -0,0 +1,154 @@
---
topic: agentskills-examples
source_keys:
- agentskills-quickstart
- agentskills-spec
- agentskills-best-practices
- agentskills-optimizing-descriptions
---
## Minimal skill (quickstart)
```
.agents/skills/roll-dice/SKILL.md
```
```markdown
---
name: roll-dice
description: Roll dice using a random number generator. Use when asked to roll a die (d6, d20, etc.), roll dice, or generate a random dice roll.
---
To roll a die, use the following command that generates a random number from 1
to the given number of sides:
```bash
echo $((RANDOM % <sides> + 1))
```
Replace `<sides>` with the number of sides on the die (e.g., 6 for a standard
die, 20 for a d20).
```
This is a complete, working skill: one file, under 20 lines. The `name` matches the directory name; the description is specific and imperative.
## Extended frontmatter
```yaml
---
name: pdf-processing
description: >
Extract text and tables from PDF files, fill PDF forms, and merge multiple
PDFs. Use when the user mentions PDFs, forms, document extraction, or needs
to work with PDF files — even if they don't use the word "PDF."
license: Apache-2.0
compatibility: Requires python3, uv, and pdfplumber
metadata:
author: example-org
version: "1.0"
category: document
---
```
## Description before and after
```yaml
# Before — too vague, won't trigger reliably
description: Process CSV files.
# After — specific about capabilities, broad about when to apply
description: >
Analyze CSV and tabular data files — compute summary statistics,
add derived columns, generate charts, and clean messy data. Use this
skill when the user has a CSV, TSV, or Excel file and wants to explore,
transform, or visualize the data, even if they don't explicitly mention
"CSV" or "analysis."
```
## Gotchas section
```markdown
## Gotchas
- The `users` table uses soft deletes. Always include `WHERE deleted_at IS NULL`.
- User ID is `user_id` in the database, `uid` in the auth service, and
`accountId` in the billing API — they all refer to the same entity.
- `/health` returns 200 even when the database is down. Use `/ready` instead.
```
## Output format template inline
```markdown
## Report structure
Use this template:
\`\`\`markdown
# [Analysis Title]
## Executive summary
[One-paragraph overview of key findings]
## Key findings
- Finding 1 with supporting data
## Recommendations
1. Specific actionable recommendation
\`\`\`
```
## Multi-step checklist
```markdown
## Form processing workflow
Progress:
- [ ] Step 1: Analyze the form (`scripts/analyze_form.py`)
- [ ] Step 2: Create field mapping (`fields.json`)
- [ ] Step 3: Validate mapping (`scripts/validate_fields.py`)
- [ ] Step 4: Fill the form (`scripts/fill_form.py`)
- [ ] Step 5: Verify output (`scripts/verify_output.py`)
```
## Plan-validate-execute pattern
```markdown
## Database migration
1. Run `python scripts/migrate.py --verify --backup`
2. Review the migration plan output
3. If plan looks correct, run `python scripts/migrate.py --execute`
Do not modify commands or add flags.
```
## Conditional reference loading
```markdown
## Error handling
If the API returns a non-200 status code, read `references/api-errors.md`
for the full error code table and retry guidance.
```
## Eval test case (evals.json)
```json
{
"skill_name": "csv-analyzer",
"evals": [
{
"id": 1,
"prompt": "I have a CSV of monthly sales data in data/sales_2025.csv. Find the top 3 months by revenue and make a bar chart.",
"expected_output": "A bar chart image showing the top 3 months by revenue, with labeled axes.",
"files": ["evals/files/sales_2025.csv"],
"assertions": [
"The output includes a bar chart image file",
"The chart shows exactly 3 months",
"Both axes are labeled",
"The chart title or caption mentions revenue"
]
}
]
}
```

100
plugins/kyberforge/docs/research/docs/agentskillsio/optimizing-descriptions.md Normal file
View File

@@ -0,0 +1,100 @@
---
topic: agentskills-optimizing-descriptions
source_keys:
- agentskills-optimizing-descriptions
---
## How triggering works
At startup, agents load only the `name` and `description` of each available skill. When a user's task matches a description, the agent reads the full `SKILL.md` into context. The description carries the entire burden of triggering.
Important nuance: agents typically only consult skills for tasks that require knowledge or capabilities beyond what they can handle alone. A simple one-step request may not trigger a matching skill because the agent can handle it with basic tools. Specialized knowledge — an unfamiliar API, a domain-specific workflow, an uncommon format — is where a well-written description makes the difference.
## Writing effective descriptions
- **Use imperative phrasing.** "Use this skill when..." rather than "This skill does...". The agent is deciding whether to act, so tell it when to act.
- **Focus on user intent, not implementation.** Describe what the user is trying to achieve, not the skill's internal mechanics.
- **Err on the side of being pushy.** Explicitly list contexts where the skill applies, including cases where the user doesn't name the domain directly: "even if they don't explicitly mention 'CSV' or 'analysis.'"
- **Keep it concise.** A few sentences to a short paragraph. Hard limit: 1024 characters (descriptions grow during optimization — check length).
## Designing trigger eval queries
Test triggering with a set of ~20 eval queries — realistic user prompts labeled `should_trigger: true/false`.
**Should-trigger queries (8–10):** vary along these axes:
- *Phrasing*: formal, casual, abbreviations, typos
- *Explicitness*: some name the domain directly ("analyze this CSV"), others don't ("my boss wants a chart from this data file")
- *Detail*: terse prompts mixed with context-heavy ones (file paths, column names, backstory)
- *Complexity*: single-step tasks alongside multi-step workflows
The most useful should-trigger queries are where the skill would help but the connection isn't obvious — these are where description wording makes the difference.
**Should-not-trigger queries (8–10):** use near-misses — queries that share keywords but need something different. Weak negative examples ("Write a fibonacci function") test nothing because there's no keyword overlap. Strong examples:
```
# For a CSV analysis skill:
"I need to update the formulas in my Excel budget spreadsheet"
# shares "spreadsheet" concept, but needs Excel editing, not CSV analysis
"can you write a python script that reads a csv and uploads each row to postgres"
# involves CSV, but the task is database ETL, not analysis
```
Include realistic context in all queries: file paths, personal context ("my manager asked me to..."), specific column names, casual language.
## Testing trigger rates
Model behavior is nondeterministic. Run each query 3 times and compute a trigger rate (fraction of runs where the skill was invoked). A should-trigger query passes if its trigger rate is ≥0.5; should-not-trigger if <0.5.
Example script structure using Claude Code:
```bash
check_triggered() {
local query="$1"
claude -p "$query" --output-format json 2>/dev/null \
| jq -e --arg skill "$SKILL_NAME" \
'any(.messages[].content[]; .type == "tool_use" and .name == "Skill" and .input.skill == $skill)' \
> /dev/null 2>&1
}
```
## Train/validation split
Split your ~20 queries to avoid overfitting: ~60% train, ~40% validation. Both sets must have proportional should-trigger/should-not mixes. Use only the train set to guide changes; use the validation set only to check whether improvements generalize. Keep the split fixed across iterations.
## The optimization loop
1. Evaluate on both train and validation sets.
2. Identify train-set failures: which should-trigger queries didn't? Which should-not-trigger queries did?
3. Revise the description:
- Should-trigger failures → description too narrow: broaden scope, add context about when the skill applies.
- Should-not-trigger false positives → description too broad: add specificity about what the skill does *not* do.
- Avoid adding specific keywords from failed queries — that's overfitting. Address the general category those queries represent.
- If stuck after several iterations, try a structurally different framing rather than incremental tweaks.
4. Repeat until train queries all pass or improvement stops.
5. Select the best iteration by validation pass rate — not necessarily the last iteration.
Five iterations is usually enough. If not improving, the problem may be with the queries, not the description.
## Before and after
```yaml
# Before
description: Process CSV files.
# After
description: >
Analyze CSV and tabular data files — compute summary statistics,
add derived columns, generate charts, and clean messy data. Use this
skill when the user has a CSV, TSV, or Excel file and wants to
explore, transform, or visualize the data, even if they don't
explicitly mention "CSV" or "analysis."
```
The improved version is more specific about capabilities (stats, derived columns, charts, cleaning) and broader about applicability (CSV, TSV, Excel; even without explicit keywords).
## Applying the result
1. Update the `description` field in `SKILL.md` frontmatter.
2. Verify it's under 1024 characters.
3. Run 5–10 fresh queries (never part of the optimization process) as a final generalization check.

48
plugins/kyberforge/docs/research/docs/agentskillsio/overview.md Normal file
View File

@@ -0,0 +1,48 @@
---
topic: agentskills-overview
source_keys:
- agentskills-home
- agentskills-spec
- agentskills-quickstart
---
## What Agent Skills is
Agent Skills is a lightweight, open format for extending AI agent capabilities with specialized knowledge and workflows. A skill is a folder containing a `SKILL.md` file — metadata plus instructions that tell an agent how to perform a specific task. Skills can also bundle scripts, reference materials, templates, and other resources.
The format was originally developed by Anthropic, released as an open standard, and has been adopted by a large and growing number of agent products: Claude Code, GitHub Copilot, OpenAI Codex, Google Gemini CLI, VS Code, Cursor, JetBrains Junie, Block Goose, OpenHands, Roo Code, Spring AI, Databricks Genie, Snowflake Cortex, and many others — 35+ confirmed implementations as of June 2026.
## Why Agent Skills exist
Agents are increasingly capable but often lack the context to do real work reliably. Skills solve this by packaging procedural knowledge and project-specific context into portable, version-controlled folders that agents load on demand:
- **Domain expertise**: capture specialized knowledge — legal review processes, data pipelines, presentation formats — as reusable instructions and resources.
- **Repeatable workflows**: turn multi-step tasks into consistent, auditable procedures.
- **Cross-product reuse**: build a skill once and use it across any skills-compatible agent.
## Progressive disclosure
Agents load skills in three stages:
1. **Discovery** — at startup, agents load only the `name` and `description` of each available skill (~100 tokens per skill). Just enough to know when one might be relevant.
2. **Activation** — when a task matches a skill's description, the agent reads the full `SKILL.md` body into context (<5000 tokens recommended).
3. **Execution** — the agent follows the instructions, optionally executing bundled scripts or loading referenced files on demand.
Full instructions load only when a task calls for them, so agents can keep many skills on hand with only a small context footprint.
## Canonical directory
The canonical location for skills is `.agents/skills/` at the project root. Tool-specific locations (`.claude/skills/` for Claude Code, `~/.codex/skills/` for Codex) are thin adapters that map to this canonical path. Putting skills at `.agents/skills/` maximizes cross-tool portability.
## File structure
```
skill-name/
├── SKILL.md # Required: metadata + instructions
├── scripts/ # Optional: executable code
├── references/ # Optional: documentation
├── assets/ # Optional: templates, resources
└── ... # Any additional files or directories
```
The `SKILL.md` file must contain YAML frontmatter followed by Markdown content. There are no format restrictions on the body.

168
plugins/kyberforge/docs/research/docs/agentskillsio/scripts.md Normal file
View File

@@ -0,0 +1,168 @@
---
topic: agentskills-scripts
source_keys:
- agentskills-using-scripts
---
## One-off commands
When an existing package does what you need, reference it directly in `SKILL.md` without a `scripts/` directory. Use package runners that auto-resolve dependencies:
| Runner | Language | Notes |
|--------|----------|-------|
| `uvx package@version` | Python | Recommended. Ships with `uv`. Caches aggressively. |
| `pipx run 'package==version'` | Python | Mature alternative, broader OS availability. |
| `npx package@version` | Node.js | Ships with npm/Node.js. Pin versions for reproducibility. |
| `bunx package@version` | Node.js | Bun equivalent of npx. Only in Bun environments. |
| `deno run npm:package@version` | TypeScript | Requires permission flags (`--allow-read`, etc.). |
| `go run golang.org/x/...@version` | Go | Built into Go toolchain. |
Tips:
- Always pin versions (e.g., `npx eslint@9.0.0`) for consistent behavior over time.
- State prerequisites in `SKILL.md` (e.g., "Requires Node.js 18+") or in the `compatibility` frontmatter field.
- Move complex commands into `scripts/` when a one-off command is hard to get right on the first try.
## Referencing scripts from SKILL.md
Use **relative paths from the skill directory root**. List available scripts so the agent knows they exist, then instruct it to run them:
```markdown
## Available scripts
- **`scripts/validate.sh`** — Validates configuration files
- **`scripts/process.py`** — Processes input data
## Workflow
1. Run validation:
```bash
bash scripts/validate.sh "$INPUT_FILE"
```
2. Process results:
```bash
python3 scripts/process.py --input results.json
```
```
The same relative-path convention works in `references/*.md` files.
## Self-contained scripts with inline dependencies
Bundle scripts that declare their own dependencies so the agent can run them with a single command.
**Python (PEP 723 + uv):**
```python
# /// script
# dependencies = [
# "beautifulsoup4>=4.12,<5",
# ]
# requires-python = ">=3.12"
# ///
from bs4 import BeautifulSoup
# ...
```
```bash
uv run scripts/extract.py
```
**TypeScript (Deno):**
```typescript
#!/usr/bin/env -S deno run
import * as cheerio from "npm:cheerio@1.0.0";
// ...
```
```bash
deno run scripts/extract.ts
```
**TypeScript (Bun):**
```typescript
#!/usr/bin/env bun
import * as cheerio from "cheerio@1.0.0";
// ...
```
```bash
bun run scripts/extract.ts
```
**Ruby (bundler/inline):**
```ruby
require 'bundler/inline'
gemfile do
source 'https://rubygems.org'
gem 'nokogiri', '~> 1.16'
end
# ...
```
```bash
ruby scripts/extract.rb
```
## Designing scripts for agentic use
### No interactive prompts (hard requirement)
Agents run in non-interactive shells and cannot respond to TTY prompts. A script that blocks on input hangs indefinitely. Accept all input via command-line flags, environment variables, or stdin:
```
# Bad
$ python scripts/deploy.py
Target environment: _
# Good
$ python scripts/deploy.py
Error: --env is required. Options: development, staging, production.
Usage: python scripts/deploy.py --env staging --tag v1.2.3
```
### Document with `--help`
`--help` output is the primary way an agent learns your script's interface:
```
Usage: scripts/process.py [OPTIONS] INPUT_FILE
Process input data and produce a summary report.
Options:
--format FORMAT Output format: json, csv, table (default: json)
--output FILE Write output to FILE instead of stdout
--verbose Print progress to stderr
```
Keep it concise — it enters the agent's context window.
### Write helpful error messages
```
Error: --format must be one of: json, csv, table.
Received: "xml"
```
Say what went wrong, what was expected, and what to try.
### Use structured output
Prefer JSON, CSV, or TSV over free-form text. Separate data from diagnostics:
- **stdout** — structured data (JSON, CSV)
- **stderr** — progress messages, warnings, diagnostics
```
# Hard to parse
NAME STATUS CREATED
my-service running 2025-01-15
# Unambiguous
{"name": "my-service", "status": "running", "created": "2025-01-15"}
```
### Further design considerations
- **Idempotency.** Agents may retry. "Create if not exists" is safer than "create and fail on duplicate."
- **Input constraints.** Reject ambiguous input with a clear error. Use enums and closed sets where possible.
- **Dry-run support.** A `--dry-run` flag lets agents preview destructive operations.
- **Meaningful exit codes.** Use distinct codes for different failure types; document them in `--help`.
- **Safe defaults.** Destructive operations should require explicit confirmation flags (`--confirm`, `--force`).
- **Predictable output size.** Many harnesses truncate tool output beyond 10–30K characters. Default to a summary or reasonable limit; support `--offset` for pagination, or require `--output FILE` to opt in to large stdout.

169
plugins/kyberforge/docs/research/docs/agentskillsio/skill-authoring.md Normal file
View File

@@ -0,0 +1,169 @@
---
topic: agentskills-skill-authoring
source_keys:
- agentskills-best-practices
---
## Start from real expertise
A common pitfall: asking an LLM to generate a skill without domain-specific context. The result is vague, generic procedures rather than the specific API patterns, edge cases, and project conventions that make a skill valuable.
**Extract from a hands-on task.** Complete a real task in conversation with an agent, providing corrections and preferences along the way. Then extract the reusable pattern. Capture:
- Steps that worked — the sequence that led to success
- Corrections you made — "use library X not Y," "check for edge case Z"
- Input/output formats — what data looked like going in and out
- Context you provided — project-specific facts the agent didn't know
**Synthesize from existing artifacts.** Feed project-specific material into an LLM and ask it to synthesize a skill. Good sources: internal runbooks, API specs, code review comments, version control history (especially patches — reveals patterns through what actually changed), real-world failure cases.
## Refine with real execution
Run the skill against real tasks. Feed results — all of them, not just failures — back into the creation process. Even a single execute-then-revise pass noticeably improves quality.
Read agent execution traces, not just final outputs. Common causes of wasted work in traces:
- Instructions too vague — agent tries multiple approaches
- Instructions that don't apply to the current task — agent follows them anyway
- Too many options without a clear default
## Spending context wisely
Once a skill activates, its full `SKILL.md` body loads alongside conversation history, system context, and other active skills. Every token competes for the agent's attention.
**Add what the agent lacks, omit what it knows.** Focus on project-specific conventions, domain-specific procedures, non-obvious edge cases, and the particular tools to use. Don't explain what a PDF is or how HTTP works.
```markdown
<!-- Too verbose — the agent already knows what PDFs are -->
PDF files contain text and images. To extract text, use pdfplumber...
<!-- Better — jumps to what the agent wouldn't know on its own -->
Use pdfplumber for text extraction. For scanned documents, fall back to
pdf2image with pytesseract.
```
Ask about each piece of content: "Would the agent get this wrong without this instruction?" If no, cut it.
**Design coherent units.** Skills scoped too narrowly force multiple skills to load for one task; skills scoped too broadly are hard to activate precisely. A skill for querying a database and formatting results is one coherent unit; a skill that also covers database administration is probably too broad.
**Aim for moderate detail.** Overly comprehensive skills hurt — the agent struggles to extract what's relevant and may pursue unproductive paths. Concise, stepwise guidance with a working example outperforms exhaustive documentation.
## Calibrating control
**Give the agent freedom** when multiple approaches are valid. Explaining *why* is more effective than rigid directives — agents make better decisions when they understand the purpose.
**Be prescriptive** when operations are fragile, consistency matters, or a specific sequence must be followed:
```markdown
## Database migration
Run exactly this sequence:
```bash
python scripts/migrate.py --verify --backup
```
Do not modify the command or add additional flags.
```
**Provide defaults, not menus.** When multiple tools could work, pick one and mention alternatives briefly:
```markdown
<!-- Too many options -->
You can use pypdf, pdfplumber, PyMuPDF, or pdf2image...
<!-- Clear default with escape hatch -->
Use pdfplumber for text extraction. For scanned PDFs requiring OCR,
use pdf2image with pytesseract instead.
```
**Favor procedures over declarations.** Teach the agent *how to approach* a class of problems, not *what to produce* for a specific instance. The approach should generalize even when individual details are specific.
## Effective instruction patterns
### Gotchas sections
The highest-value content in many skills is a list of gotchas — environment-specific facts that defy reasonable assumptions:
```markdown
## Gotchas
- The `users` table uses soft deletes. Queries must include
`WHERE deleted_at IS NULL` or results will include deactivated accounts.
- The user ID is `user_id` in the database, `uid` in the auth service,
and `accountId` in the billing API. All three refer to the same value.
- The `/health` endpoint returns 200 even if the database is down.
Use `/ready` to check full service health.
```
Keep gotchas in `SKILL.md` where the agent reads them before encountering the situation. When an agent makes a mistake you correct, add the correction to the gotchas section.
### Templates for output format
Provide a template when the agent must produce a specific output format. Short templates live inline; longer ones go in `assets/` and are referenced conditionally.
```markdown
## Report structure
Use this template:
\`\`\`markdown
# [Analysis Title]
## Executive summary
[One-paragraph overview]
## Key findings
- Finding 1 with supporting data
## Recommendations
1. Specific actionable recommendation
\`\`\`
```
### Checklists for multi-step workflows
```markdown
## Workflow
Progress:
- [ ] Step 1: Analyze the form (run `scripts/analyze_form.py`)
- [ ] Step 2: Create field mapping (edit `fields.json`)
- [ ] Step 3: Validate mapping (run `scripts/validate_fields.py`)
- [ ] Step 4: Fill the form (run `scripts/fill_form.py`)
```
### Validation loops
```markdown
## Editing workflow
1. Make your edits
2. Run validation: `python scripts/validate.py output/`
3. If validation fails, fix and re-run
4. Only proceed when validation passes
```
### Plan-validate-execute
For batch or destructive operations: create an intermediate plan in a structured format, validate it against a source of truth, then execute:
```markdown
1. Extract form fields → `form_fields.json`
2. Create `field_values.json` mapping each field to its intended value
3. Validate: `python scripts/validate_fields.py form_fields.json field_values.json`
4. If validation fails, revise and re-validate
5. Fill the form: `python scripts/fill_form.py input.pdf field_values.json output.pdf`
```
### Bundling reusable scripts
If execution traces show the agent independently reinventing the same logic across runs — building charts, parsing a format, validating output — write a tested script once and bundle it in `scripts/`. See the scripts reference.
## Structure large skills with progressive disclosure
Keep `SKILL.md` under 500 lines and 5,000 tokens — the core instructions the agent needs every run. When a skill needs more content, move detail to `references/` and tell the agent *when* to load each file:
```markdown
If the API returns a non-200 status code, read `references/api-errors.md`.
```
"Read `references/api-errors.md` if X" is more useful than a generic "see references/ for details." The agent loads context on demand rather than up front.

57
plugins/kyberforge/docs/research/docs/agentskillsio/sources.md Normal file
View File

@@ -0,0 +1,57 @@
# Sources
## agentskills-home
- **URL:** https://agentskills.io/home.md
- **Description:** Agent Skills overview — what it is, why it exists, progressive disclosure model, ecosystem of 35+ implementing tools
- **Contributing files:** overview.md
- **Status:** `extracted`
## agentskills-spec
- **URL:** https://agentskills.io/specification.md
- **Description:** Complete SKILL.md format specification — frontmatter fields, constraints, body content, optional directories, progressive disclosure levels, file references, validation
- **Contributing files:** overview.md, specification.md, examples.md
- **Status:** `extracted`
## agentskills-best-practices
- **URL:** https://agentskills.io/skill-creation/best-practices.md
- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops)
- **Contributing files:** skill-authoring.md, examples.md
- **Status:** `extracted`
## agentskills-optimizing-descriptions
- **URL:** https://agentskills.io/skill-creation/optimizing-descriptions.md
- **Description:** How to systematically test and improve skill descriptions for triggering accuracy — eval queries, trigger rate testing, train/validation splits, optimization loop
- **Contributing files:** optimizing-descriptions.md, examples.md
- **Status:** `extracted`
## agentskills-evaluating-skills
- **URL:** https://agentskills.io/skill-creation/evaluating-skills.md
- **Description:** Eval-driven skill quality improvement — test case design, workspace structure, assertion writing, grading, benchmarking, human review, iteration loop
- **Contributing files:** evaluating-skills.md, examples.md
- **Status:** `extracted`
## agentskills-using-scripts
- **URL:** https://agentskills.io/skill-creation/using-scripts.md
- **Description:** Using scripts in skills — one-off commands, self-contained scripts with inline dependencies, designing scripts for agentic use (no interactive prompts, --help, structured output, idempotency)
- **Contributing files:** scripts.md
- **Status:** `extracted`
## agentskills-quickstart
- **URL:** https://agentskills.io/skill-creation/quickstart.md
- **Description:** Step-by-step guide to creating a first skill (roll-dice example), how discovery/activation/execution work in practice
- **Contributing files:** overview.md, examples.md
- **Status:** `extracted`
## agentskills-llms-txt
- **URL:** https://agentskills.io/llms.txt
- **Description:** Documentation index used for source discovery — lists all available pages with URLs
- **Contributing files:** (none — used for discovery only)
- **Status:** `extracted`

124
plugins/kyberforge/docs/research/docs/agentskillsio/specification.md Normal file
View File

@@ -0,0 +1,124 @@
---
topic: agentskills-specification
source_keys:
- agentskills-spec
---
## Frontmatter fields
| Field | Required | Constraints |
|----------------|----------|-------------|
| `name` | Yes | 1–64 characters. Lowercase letters, numbers, and hyphens only. Must not start or end with a hyphen. No consecutive hyphens (`--`). Must match the parent directory name. |
| `description` | Yes | 1–1024 characters. Describes what the skill does and when to use it. |
| `license` | No | License name or reference to a bundled license file. |
| `compatibility`| No | 1–500 characters. Environment requirements: intended product, required system packages, network access needs. Most skills don't need this. |
| `metadata` | No | Arbitrary key-value map. Use for author, version, category, and any project extensions not defined by the spec. |
| `allowed-tools`| No | Space-separated string of pre-approved tools. Experimental — support varies by client. |
## Minimal example
```yaml
---
name: skill-name
description: A description of what this skill does and when to use it.
---
```
## Extended example
```yaml
---
name: pdf-processing
description: Extract PDF text, fill forms, merge files. Use when handling PDFs.
license: Apache-2.0
metadata:
author: example-org
version: "1.0"
category: document
allowed-tools: Bash(python3:*) Read Write
---
```
## name field rules
Valid: `pdf-processing`, `data-analysis`, `code-review`
Invalid:
- `PDF-Processing` — uppercase not allowed
- `-pdf` — cannot start with hyphen
- `pdf--processing` — consecutive hyphens not allowed
The `name` value must exactly match the parent directory name. A skill at `.agents/skills/my-tool/SKILL.md` must have `name: my-tool`.
## description field guidance
The description carries the entire burden of triggering. Agents read only `name` and `description` at startup. Write it as a trigger — describe both what the skill does and when to use it.
Good:
```yaml
description: >
Analyze CSV and tabular data files — compute summary statistics,
add derived columns, generate charts, and clean messy data. Use this
skill when the user has a CSV, TSV, or Excel file and wants to explore,
transform, or visualize the data, even if they don't explicitly mention
"CSV" or "analysis."
```
Poor:
```yaml
description: Helps with PDFs.
```
Hard limit: 1024 characters. Descriptions tend to grow during optimization — check length before committing.
## Body content
No format restrictions. Write whatever helps agents perform the task effectively.
Recommended sections:
- Step-by-step instructions
- Examples of inputs and outputs
- Common edge cases
The agent loads the entire body once the skill is activated. Keep `SKILL.md` under 500 lines. Move detailed reference material to separate files in `references/` or similar directories.
## Optional directories
### `scripts/`
Executable code agents can run. Scripts should be self-contained, include helpful error messages, and handle edge cases gracefully. See the scripts reference for design guidance.
### `references/`
Additional documentation agents can read when needed. Keep files focused — agents load them on demand, so smaller files mean less context use. Common files: `REFERENCE.md`, `FORMS.md`, domain-specific files (`finance.md`, `legal.md`).
### `assets/`
Static resources: templates, images, data files (lookup tables, schemas).
## File references
Use relative paths from the skill root when referencing other files:
```markdown
See [the reference guide](references/REFERENCE.md) for details.
Run: scripts/extract.py
```
Keep references one level deep from `SKILL.md`. Avoid deeply nested reference chains.
## Progressive disclosure levels
1. **Metadata** (~100 tokens): `name` and `description` — loaded at startup for all skills
2. **Instructions** (<5000 tokens recommended): full `SKILL.md` body — loaded on activation
3. **Resources** (as needed): files in `scripts/`, `references/`, `assets/` — loaded only when required
## Validation
The `skills-ref` reference library validates `SKILL.md` frontmatter and naming conventions:
```bash
skills-ref validate ./my-skill
```
Source: `https://github.com/agentskills/agentskills/tree/main/skills-ref`

36
plugins/kyberforge/docs/research/docs/agentsmd/configuration.md Normal file
View File

@@ -0,0 +1,36 @@
---
topic: configuration
source_keys:
- agents-md-official
- context7-websites-agents-md
---
## General Setup
No installation is required. Create a file named `AGENTS.md` at the root of the repository. Most agents that support the standard read it automatically without any configuration.
## Tool-Specific Configuration
Some tools require an explicit opt-in to read AGENTS.md:
**Aider** — add to `.aider.conf.yml`:
```yaml
read: AGENTS.md
```
**Gemini CLI** — add to `.gemini/settings.json`:
```json
{
"context": {
"fileName": "AGENTS.md"
}
}
```
For all other tools in the compatibility list (GitHub Copilot, Cursor, Codex, Claude Code, etc.), AGENTS.md is read automatically when present at the repository root or in the nearest parent directory.
## Conflict Resolution
- Nearest file wins: the AGENTS.md closest to the file being edited takes precedence over files higher in the directory tree.
- User prompt wins over all: explicit instructions given in the chat interface override any AGENTS.md rule.
- No merge across files: only the nearest file applies; inner files do not inherit from outer ones unless the agent explicitly implements that behavior.

63
plugins/kyberforge/docs/research/docs/agentsmd/examples.md Normal file
View File

@@ -0,0 +1,63 @@
---
topic: examples
source_keys:
- agents-md-official
- context7-websites-agents-md
- context7-agentsmd-agents-md
---
## Minimal Project Example
A practical AGENTS.md covering setup, code style, testing, and PR conventions for a pnpm monorepo:
```markdown
# AGENTS.md
## Setup commands
- Install deps: `pnpm install`
- Start dev server: `pnpm dev`
- Run tests: `pnpm test`
## Code style
- TypeScript strict mode
- Single quotes, no semicolons
- Use functional patterns where possible
## Dev environment tips
- Use `pnpm dlx turbo run where <project_name>` to jump to a package instead of scanning with `ls`.
- Run `pnpm install --filter <project_name>` to add the package to your workspace so Vite, ESLint, and TypeScript can see it.
- Check the `name` field inside each package's `package.json` to confirm the right name.
## Testing instructions
- Find the CI plan in the `.github/workflows` folder.
- Run `pnpm turbo run test --filter <project_name>` to run every check defined for that package.
- From the package root you can just call `pnpm test`. The commit should pass all tests before you merge.
- Fix any test or type errors until the whole suite is green.
- Add or update tests for the code you change, even if nobody asked.
## PR instructions
- Title format: [<project_name>] <Title>
- Always run `pnpm lint` and `pnpm test` before committing.
```
## Monorepo Nested Structure
Place AGENTS.md files at each scope level. Agents read the nearest file automatically:
```
my-monorepo/
├── AGENTS.md # Root-level: applies to the whole repo
├── packages/
│ ├── api/
│ │ └── AGENTS.md # API-specific instructions; overrides root for this package
│ ├── web/
│ │ └── AGENTS.md # Web app-specific instructions
│ └── shared/
│ └── AGENTS.md # Shared library instructions
```
Each sub-AGENTS.md can focus on the specifics of that package (framework, test runner, build tool) without repeating root-level conventions.
## What Agents Do with AGENTS.md
Agents actively use the file — they attempt to run listed test commands and fix failures before completing a task. Treat the instructions as live directives, not documentation. Keep the file updated as tooling and conventions change.

41
plugins/kyberforge/docs/research/docs/agentsmd/overview.md Normal file
View File

@@ -0,0 +1,41 @@
---
topic: overview
source_keys:
- agents-md-official
- context7-websites-agents-md
- context7-agentsmd-agents-md
---
## What AGENTS.md Is
AGENTS.md is an open format that serves as a dedicated README for AI coding agents — a single, predictable file where developers place context and instructions that help agents work effectively on a project. It is now stewarded by the Agentic AI Foundation under the Linux Foundation, having emerged from collaboration between OpenAI Codex, Amp, Jules (Google), Cursor, and Factory.
The format intentionally complements rather than replaces README.md. README stays focused on human contributors; AGENTS.md carries the agent-specific detail — build steps, test commands, conventions, security gotchas — that would clutter a README or is irrelevant to humans.
Over 60,000 open-source projects use it.
## Format
AGENTS.md is plain Markdown placed at the root of a repository. There are no required fields, no schema, and no special syntax — agents parse the text. Developers use any headings appropriate to their project.
Common sections include:
- Project overview
- Build and test commands
- Code style guidelines
- Testing instructions
- Security considerations
- Commit message conventions
- Pull request guidelines
- Deployment steps
## Precedence Rules
When multiple AGENTS.md files exist (common in monorepos), the file closest to the edited file in the directory tree takes precedence. Explicit user chat prompts override all AGENTS.md instructions.
## Monorepo Support
Place additional AGENTS.md files inside individual packages or subdirectories. Agents automatically read the nearest file, so subprojects can carry tailored instructions that supplement or override the root-level file. OpenAI's main repository contains 88 AGENTS.md files as an example of this pattern.
## Cross-Tool Compatibility
A single AGENTS.md works across the majority of AI coding tools. Confirmed compatible agents include: GitHub Copilot, VS Code AI, Cursor, Aider, Devin (Cognition), Windsurf (Cognition), OpenAI Codex, Google Jules, Gemini CLI, Zed, RooCode, Kilo Code, Warp, opencode, Amp, Factory, Augment Code, Junie (JetBrains), Semgrep, goose, Phoenix, Ona, UiPath Autopilot & Coded Agents, and Claude Code (Anthropic). The standard is designed to be vendor-neutral.

29
plugins/kyberforge/docs/research/docs/agentsmd/sources.md Normal file
View File

@@ -0,0 +1,29 @@
# Sources
## agents-md-official
- **URL:** https://agents.md/
- **Description:** Official agents.md website — overview, how-to guide, supported tools list, sample file, FAQ, and tool configuration snippets
- **Contributing files:** overview.md, configuration.md, examples.md
- **Status:** `extracted`
## context7-websites-agents-md
- **URL:** context7:/websites/agents_md
- **Description:** Context7 index of the official agents.md website (High source reputation, benchmark score 93.78) — overview, governance, cross-tool compatibility, configuration examples, FAQ
- **Contributing files:** overview.md, configuration.md, examples.md
- **Status:** `extracted`
## context7-agentsmd-agents-md
- **URL:** context7:/agentsmd/agents.md
- **Description:** Context7 index of the agentsmd/agents.md repository (Medium source reputation, 26 code snippets) — format spec, nested monorepo patterns, file structure examples
- **Contributing files:** overview.md, examples.md
- **Status:** `extracted`
## agents-md-llms-txt
- **URL:** https://agents.md/llms.txt
- **Description:** Attempted fetch of llms.txt for machine-readable spec
- **Contributing files:** (none)
- **Status:** `no content extracted` — HTTP 404