holocron/docs/notes/factory-research-gaps-conflicts.md

Exploration Note: AI Coding Factory Research — Gaps and Conflicts with Current Roadmap

Date: 2026-05-16 Status: Superseded — grill complete. Decisions in docs/notes/factory-integration-decisions.md. ADRs: 0008, 0009, 0010. Issues: 0013, 0014. Context: Before starting Chunk 4 (workflows), the roadmap requires grilling the AI coding factory research against the existing repo. This note identifies where the research diverges from the current vision/roadmap/ADRs to frame that session.

---

1. The Central Scope Question (must resolve first)

The factory research describes a project-level development environment — a repo where you build software with AI assistance. This repo is a global config repo that provides skills, instructions, and conventions to projects.

The question: Is this repo building the factory pattern for others to use in their projects? Or is this repo itself structured as a factory?

This changes almost every downstream decision. Until it's resolved, individual gap items below may be pointing at the wrong layer.

---

2. True Conflicts (one has to win)

2.1 Where coding conventions live

Factory research: Preferred/Avoid code blocks belong in CONTEXT.md. Convention lives in a single shared-vocabulary file the agent always loads. Chunk 2 plan: Coding conventions go in a separate core/instructions/coding.md file, loaded on-demand.

One of these wins. If conventions go in CONTEXT.md, coding.md becomes much thinner or unnecessary. If they stay in coding.md, the factory's "CONTEXT.md is the primary context artefact" principle weakens.

2.2 Skill path structure: flat vs. nested

Factory research: 34 skills in 9 nested categories — design/grill-me/SKILL.md, implement/tdd/SKILL.md, cross-cutting/session-handoff/SKILL.md, etc. (phase × domain matrix) Current state: 12 skills in a flat structure — grill-me/SKILL.md, tdd/SKILL.md, etc.

Changing to nested paths breaks any tool that discovers skills by path. The nested structure also implies a different deployment model from install.sh. This is a structural decision for Chunk 3 — if we're adopting the nested taxonomy, it needs to be decided before writing any more skill files.

2.3 Governance file naming

Factory research: AGENTS.md at repo root as the single governance file for agents. Current repo: core/instructions/governance.md loaded via @import in providers/claude-code/CLAUDE.md.

These serve the same purpose. The factory's naming is cleaner (one file, obvious name), but the current structure fits the provider-agnostic model (governance isn't provider-specific). Not a blocking conflict, but naming inconsistency will cause confusion in grill sessions if not resolved.

---

3. Gaps (factory has it; this repo doesn't plan it yet)

3.1 LESSONS.md (impacts Chunk 2)

Factory treats LESSONS.md as a required committed file — the mechanism for long-loop feedback and convention graduation. Without it, there is no defined path from "I noticed a pattern" to "this is now a standing rule in CONTEXT.md". Not in the Chunk 2 scope. Question: is this a Chunk 2 addition or deferred?

3.2 docs/spec/ living spec layer

Factory requires docs/spec/overview.md and docs/spec/architecture.md — a spec that is updated in the same PR as any behaviour change. The current docs structure has no equivalent layer; workflow artifacts go in docs/prd/, docs/ard/, etc. Adding docs/spec/ is additive (not conflicting), but it changes the docs convention and needs to be decided before Chunk 4 (workflows) defines workflow artifacts.

3.3 Session handoff skill (impacts Chunk 3)

Factory treats cross-cutting/session-handoff as a core cross-cutting skill — it explicitly prompts LESSONS.md extraction before closing a session. Not in the current skills roadmap. If LESSONS.md is adopted (see 3.1), session handoff becomes necessary. These are linked.

3.4 Eval infrastructure timing

Factory requires eval.yaml co-located with every skill. Current roadmap puts tooling in Chunk 6. Every skill built in Chunk 3 without an eval accumulates tech debt against this requirement. Chunk 6 either has to back-fill evals for all existing skills, or the decision is made now that evals are deferred deliberately.

3.5 factory/ meta-skills category

Factory plans 7 meta-skills (write-skill, write-eval, write-adr, write-workflow, validate-skill, upgrade-skill, write-issue-spec) that let the factory build and verify itself. Not in the current roadmap at all. These would logically fit in Chunk 3 (skills), but they require the nested taxonomy decision (2.2) first.

3.6 References directory and stack versions registry

Factory requires a references/ directory with stack-versions.md (authoritative version registry) and context files like references/proxmox-cluster.md. Not planned in the current structure. Low urgency unless IaC skills are being built.

3.7 Model routing

Factory defines an explicit model routing table (Haiku for classification/formatting, Sonnet for most coding, Opus for adversarial review). Not in current vision or chunk scope. The factory treats this as both a cost decision and a sustainability decision — using frontier models for tasks a smaller model can do is waste, and the governance constitution calls this out explicitly. Where does this decision live — in governance.md, in CONTEXT.md, or per-skill?

---

4. Questions the Grill Needs to Resolve

These don't have a "right answer" yet — they need explicit decisions.

Q1: Does this repo adopt the factory's nested skill taxonomy now (before Chunk 3 adds more skills), or does it stay flat and reorganize later? What does install.sh need to change to support nested paths?

Q2: Should LESSONS.md be added as part of Chunk 2? If yes, what's the graduation threshold and mechanism for this homelab/solo context?

Q3: What parts of the factory's proposed structure belong in the global config repo vs. in project repos that inherit from it? Specifically: IaC skills, Gitea integration skills, docs/spec/, and role skills (Architect, Developer, Reviewer, etc.) — are these global primitives or project-specific?

Q4: The factory's implementation guidance lists 6 grill-me sessions before building. Sessions 1 and 3 (repo vision and roadmap) are roughly done. Sessions 4 (documentation structure), 5 (governance file structure), and 6 (skill library scope) have not been formally grilled — these map directly to the decisions above. Should this grill session cover all three, or one at a time?

Q5: Where does the write-a-skill skill (already deployed as write-a-skill/SKILL.md) map in the factory taxonomy? Is it the same as factory/write-skill, or is it a different thing?

Q6: The factory says every skill needs trigger tests before production use. The current skills have no evals. Is this a hard blocker on declaring Chunk 3 "done", or is it consciously deferred to Chunk 6 with a known backlog?

---

5. What Is Already Aligned (don't re-litigate)

• .agents/skills/ as canonical skill path — consistent with the Agent Skills standard
• SKILL.md format with YAML frontmatter — consistent
• CONTEXT.md exists and is ≤200 lines — consistent with factory's context architecture
• Pull-based distribution model — consistent with factory's "no runtime dependency" principle
• Governance.md loaded via @import — matches factory's "always-on" governance pattern
• No third-party skill installs without reading them — consistent with OWASP AST01 guidance
• ADRs in docs/adr/ — consistent
• Conventional commits — consistent with factory's git-commit-message skill intent
• Multi-file skill structure (tdd/ is already multi-file) — consistent with factory's best practice
• Progressive disclosure (names load at startup, bodies on demand) — already how install.sh works
• HITL before irreversible actions — consistent with governance layer

---

Input for the grill-me session. These are framing questions, not conclusions. The grill should surface what was missed here and challenge the framing of the conflicts before any decisions are locked.