holocron/docs/notes/archive/ai-ethics-security-principles.md

Exploration Note: Core Principles — AI Ethics & Security

Purpose

Brief for a /grill-with-docs session. Goal: define the scope, structure, and tier placement of core principles governing AI ethics and security before writing any content.

What we want to build

A set of guiding principles covering:

• Ethics — responsible use of AI in development (trust, transparency, human oversight, bias, accountability)
• Security — secure use of AI tooling (prompt injection, credential handling, supply chain, code review hygiene, data exposure)
• Other candidates — privacy, auditability, scope creep (AI doing more than asked), agent autonomy limits

These would live somewhere in this repo and be deployed to ~/.claude/ via install.sh.

Key question the grill must resolve

Tier placement — the two-tier CLAUDE.md model (ADR-0005) means this decision has architectural weight:

• Tier 1 (always-on) — universal rules in providers/claude-code/CLAUDE.md; loaded every session; must apply everywhere
• Tier 2 (on-demand) — topic files in core/instructions/<topic>.md; agent reads when task context is relevant

Getting this wrong either bloats every session's context (over-elevated rules) or silently drops rules that should always apply (under-elevated rules).

What the grill should challenge

1. Which principles are truly universal (always-on) vs. task-specific (on-demand)?
2. Should ethics and security be one file or separate files?
3. Is "security" about agent behavior, or developer practice, or both? They have different owners.
4. Does any of this duplicate what's already in the always-on section of CLAUDE.md (e.g. "require explicit confirmation for irreversible actions" is already there)?
5. How do project-level overrides interact — can a project suppress a global ethics rule? Should it be able to?
6. What's the enforcement model — these are instructions to an AI, not code. How do we verify they hold?

Context the next session needs

• Read CONTEXT.md — domain language and existing principles
• Read docs/VISION.md — long-term direction, especially the Management Application (these principles will eventually be configurable in the UI)
• Read providers/claude-code/CLAUDE.md — the always-on section, to avoid duplicating what's already there
• Read docs/adr/0005-two-tier-claude-md.md — rationale for the two-tier model
• Chunk dev workflow: grill → /to-prd → /to-issues → /tdd → architecture review

Session flow

Step 1 — Research sub-agent (run first, before the grill)

Spawn a general-purpose sub-agent to explore the space and return material for the grill to work with. Give it this brief:

Research AI ethics and AI security principles relevant to a software development context. Cover:

• Established frameworks and standards (EU AI Act, NIST AI RMF, OWASP LLM Top 10, IEEE Ethically Aligned Design, Google/Microsoft/Anthropic responsible AI principles)
• What practicing dev teams actually use — concrete rules, not abstract values
• Security-specific: prompt injection, credential exposure, supply chain risks, code review hygiene, data leakage from context windows
• Ethics-specific: human oversight, transparency, bias in generated code, accountability, scope creep / agent autonomy limits
• Examples of how other orgs have written these as actionable developer guidelines (not mission statements)

Do not just summarize — propose options. For each topic area, give 2–3 distinct approaches with a recommendation and the tradeoff. Flag anything creative or non-obvious. The output will feed a grill session where I'll iterate and refine.

Let the agent return its full output before starting the grill.

Step 2 — Grill session (/grill-with-docs)

Run the grill with both this note and the agent's research output as context. The grill covers two tracks:

Track A — Structure (what the grill always does) Challenges tier placement, file structure, and terminology against CONTEXT.md and existing ADRs (see "Key questions" above).

Track B — Content ideation (added scope) Uses the research output to iterate on the actual principles. The grill should:

• Challenge each proposed principle — is it actionable or a platitude?
• Force a choice when the agent offers options: pick one, name the tradeoff
• Identify gaps the research missed
• Stress-test principles against real scenarios ("a junior dev asks the agent to commit credentials — does this rule catch it?")

The grill closes when both tracks are resolved: structure is decided and a draft principle set is agreed.

---

Session start — paste this to begin tomorrow

I want to work on core principles for AI ethics and security in this repo. Start by reading docs/notes/ai-ethics-security-principles.md for full context.

Then follow the two-step session flow in that note:
1. Spawn a general-purpose research sub-agent with the brief from the note. Wait for it to finish.
2. Run /grill-with-docs with both the note and the agent output as context.

The grill has two tracks: structure (tier placement, file layout) and content ideation (actual principles). Both must be resolved before we close.