What changed after the kit met real work.

Knowledge vs. consumed docs

Spec Agent CLI came before the packaged source kit. It added an OKF-style knowledge bundle to an existing repository that already had buildable specifications, agent skills, packaging rules, and a substantial quality gate. The baseline case: could governing knowledge remain distinct from the Markdown documents the application itself consumes?

Findings

• Project specifications and runtime input specifications need separate locations and explicit indexes.
• Documentation structure can drift while application tests remain green. Missing metadata and broken local links need their own check.
• Repository knowledge should join the ordinary quality gate without adding dependencies to the shipped application.

The first bundle

The kit’s first version carried forward a repo-local knowledge bundle with separate spec and ADR indexes, a dated log, source-to-knowledge mappings, and local validation. It kept project knowledge outside application runtime concerns and left stricter document-schema checks to the target repository.

The first greenfield run

Repo Pulse was the first full greenfield run through the kit’s goal loop. It moved from an owner-defined problem through five verified milestones, application contracts, architecture decisions, a clean-checkout check, and a final acceptance pass.

Findings

• The milestone plan reached completion before a realistic pasted URL exposed an input gap. Mechanical tests had covered the contract, while the first-user path still had something to teach.
• Architecture decisions could remain proposed after the goal was reported complete unless the review inbox became part of completion.
• The installed kit became stale within hours of a source-kit improvement, with no version signal in the target repository.
• Environment-file conventions needed a mechanical read boundary as well as written secret-handling rules.

Acceptance and drift

The goal interview now asks for realistic and messy example interactions. New backlogs end with a clean-checkout quickstart check, and the goal closes only after a first-user acceptance pass. Completion reports include proposed ADRs awaiting review. The kit also gained a pending-decision inbox, spec and ADR scaffolds, environment-file ignore and read-denial rules, a release stamp in installed repositories, session-start version drift reporting, and safer refresh behavior for review candidates.

Scheduled, multi-service, dual-agent

Skywatch put the loop against an application with scheduled work, 2 upstream services, SQLite state, SMTP delivery, quiet hours, retention, and a manual mail-sink check. It also had both Claude Code and Codex configuration in the same repository.

Findings

• A milestone can depend on a local tool the agent cannot install. The plan needs to identify that boundary before work begins and mark the check as owner-gated.
• A broad source mapping creates noise. Mappings work better at module and contract boundaries.
• The stop hook could repeat after blocking once, and loose filename exclusions could miss real code changes.
• Second-agent configuration could look like application work and keep rearming the documentation check.
• Dogfood repositories had improved their installed hooks. A later kit update needed to preserve those owner edits.

Provenance and precision

The goal interview now checks whether verification tools are available and records owner-gated steps. The starter map asks for narrow mappings. The stop hook gained a repeat-block guard, exact filename exclusions, portable project-root handling, and tolerance for second-agent configuration. Installed kit files now carry local provenance: an unedited kit file can be refreshed safely; an owner-edited file stays in place while the new version waits beside it for review. The post-goal review also proposes ordinary repository hygiene when it is still missing.

The model without the kit’s tree

This repository carried the kit’s conceptual model, ported by hand into a different physical layout: chaptered specifications under docs/architecture/specification/, sixteen three-digit ADRs with body-style status bullets instead of frontmatter, JSON Schemas as first-class governing contracts, an AGENTS.md playbook with CLAUDE.md as a pure import shim, and no bundle root. Could the kit adopt the arrangement instead of bulldozing it?

Findings

• Every mechanic assumed the canonical tree: the ADR scaffold would have forked a parallel numbering sequence, and body-status decisions were invisible to the review scan.
• The version hook nagged every session because there was no stamp file to read.
• The updater would have planted empty parallel indexes beside the real knowledge and staged its playbook against the wrong file.
• The stop-time gate misclassified mapped machine-readable schemas as undocumented code.

Layout tolerance

Kit 0.2.0 (ADR 0018): a three-key layout: block in the source map relocates the knowledge homes; status and numbering detection read the conventions repositories actually use; missing indexes are seeded from the files beside them, never created empty; the playbook stages against AGENTS.md where that is the real playbook; the version check stays silent without a stamp; and once real mappings exist, the stale-map check — not a path prefix — is the authority on what counts as documentation.

The inverse brownfield

The opposite arrangement from the hand-ported service: canonical directories and a stamped bundle root, but its own file conventions — adr-0001-* and spec-000-* names, table-style indexes with prose after the tables, and its own frontmatter validator running in CI.

Findings

• The numbering scan was blind to alpha-prefixed names and would have forked a parallel sequence — the same failure the previous round fixed for bare numbers.
• Appending a bullet corrupted table-style indexes, and the last line was no signal: real indexes keep prose after the table.
• The updater staged a whole bundle-root candidate whose only real payload was the missing version stamp.
• The thorough first-time pass — find every piece of knowledge, map it, backfill the gaps — had no defined procedure and was re-derived per adoption.

Naming tolerance and a guided adoption pass

Kit 0.3.0 (ADRs 0019 and 0020): the scaffolds continue a repository’s own ID sequences instead of forking them; a table-style index gets no guessed row — the helper declines and prints the entry for a manual add; the updater stamps the version into an existing bundle root in place; and a fifth installed skill, okf-adopt, carries the adoption pass itself — inventory without moving anything, map, backfill, validate — with migration to the kit’s defaults only at the owner’s direction.

The live adoption

With 0.3.0 committed, the updater ran against the live repository — not a scratch copy. No candidate conflicts, nothing flagged for review; the adoption pass mapped every module from the specs’ own component lists, and the repository’s own CI validator, the stale-map check, and verify-install were green on the first pass.

Findings from the live pass

• The goal template shipped owner: [owner name] — a bracket placeholder that parses as a YAML list and fails a strict frontmatter validator — while the playbook template hardcoded the kit author’s name, wrong for every adopter but one.
• The goal template’s milestone-mechanics paragraph restated the playbook’s Goal iteration section, and both files preload into every session.
• A repository adopted at its target state has no honest backlog to write; the template assumed one and left that case to be re-derived.

Owner-rendered templates

Kit 0.3.1: both installers render the owner-carrying templates instead of copying them, substituting the target’s git config user.name wherever they are written; the goal template’s milestone mechanics shrink to a pointer at the playbook plus the one backlog-local rule; and the template and the okf-adopt skill now name the empty-backlog idiom for repositories adopted at their target state.

Two upgrades in one day

The repository crossed 0.3.5 → 0.3.10 and then 0.3.10 → 0.3.12, and its earlier friction was the stated reason 0.3.11 existed — so the release fixing that friction met the repository it came from.

Findings

• The mirror fix held: the hook mirror that had drawn a pointless candidate at every prior release refreshed in place, none staged.
• The filled-playbook suppression did not fire — on the very repository whose friction motivated it. It asks for a template delta, and diff exits zero when two files are identical, so “the template did not change” took the same branch as “the delta could not be computed” and the whole bracketed template was staged again.
• Those two cases want opposite handling: an uncomputable delta leaves a reviewer nothing, so the template is still worth staging; an empty one is proof there is nothing to review.

Empty is not uncomputable

The delta helper now reports the identical case distinctly, suppression accepts it as an answer in its own right, and the summary states the template is unchanged since the stamped release rather than pointing at a delta file that was never written. Smoke coverage gained the version-crossing, template-unchanged case; the uncomputable path keeps its own.

The tool turned on the kit

A bug from the kit’s own history supplied the test: a decision record declared a relocatable stamp file, the installer kept writing the default path, and eight days and ten commits later a full audit — every script line-read — found it. Reintroduced as an ordinary-looking change, it went to the tool with the governing records beside it.

Caught, with two limits

• One model call returned drift and described the same consequence the audit had recorded — a second stamp and a drift note that never clears.
• It judges the change in front of it. The original arrangement — a record adding a claim about code that never moved — has no diff to read, and no diff-driven reader can see it.
• It reasons only over records the map points at, and the map did not point the installer at that decision record. Without the entry, the record never reaches the reader.

Mapping is the prerequisite

The gap is the kit’s own map, not the tool: a decision that binds the installer is not mapped to it, so nothing mechanical carries it into review. That entry is still open — recorded here rather than quietly fixed, because the finding is the map discipline the kit asks of every adopter applying to the kit itself.