Artifact formats
Every file the pipeline writes, its schema, and what validates it on read.
Every artifact the toolchain reads or writes, in pipeline order. All JSON/YAML artifacts
carry a version field and are validated with zod schemas from @handbooks/core on read.
Paths are POSIX-relative to the analyzed source root unless stated otherwise.
Work-directory layout
<work>/
phase1/
graph.json the call graph (nodes + edges + selfAttrs + metadata)
functions.csv one row per internal function
graph.dot Graphviz view (files as clusters; await edges colored)
dropped-calls.json unresolved calls, categorized
scan-coverage.json files the scan could NOT turn into facts, and why
phase2/
cards/<rel>.json one card per source file (tree-mirrored paths)
cards/_coverage.json {nFiles, nDescribed, missing[]}
skeleton.yaml the stage skeleton
assignment.json file → stage
organization.yaml intra-stage groups + reading order
members.json (member strategy only) function → stage
phase3/
narration.json stage + system prose
registers.json cross-stage state registers
cache/ content-hash caches (safe to delete; costs a re-generation)phase1/graph.json
{
"version": 1,
"metadata": {
"generatedAt": "2026-08-02T10:00:00.000Z",
"language": "python | typescript | go | rust | shell | multi",
"sourceRoot": "/abs/path",
"scannedFiles": ["aggregate/rollup.rs", "…"], // only files that were actually read and parsed
"nInternalFunctions": 316,
"nBoundaryNodes": 45,
"nEdges": 903,
"policy": "Edges are emitted only when the callee resolves …",
"unparsedFiles": [
// optional; [] means every scanned file parsed cleanly
{ "file": "app/legacy.py", "reason": "partial", "detail": "…" },
],
},
"nodes": {
"app.main.main": {
// internal node (kind: "internal")
"id": "app.main.main",
"name": "main",
"qualname": "main",
"file": "ingest/collector.go",
"lineStart": 4,
"lineEnd": 9,
"signature": "def main()",
"isAsync": false,
"isMethod": false,
"className": null,
"decorators": [],
"kind": "internal",
"synthetic": false, // true = implied node (e.g. implicit constructor)
"selfAttrsRead": [],
"selfAttrsWritten": [],
"paramTypes": {},
"nCallees": 3,
"nCallers": 0,
},
"boundary:os.getpid": {
// boundary node (kind: "boundary")
"id": "boundary:os.getpid",
"name": "getpid",
"qualname": "os.getpid",
"module": "os",
"className": "",
"kind": "boundary",
"nCallees": 0,
"nCallers": 1,
},
},
"edges": [
{
"callerId": "app.main.main",
"calleeId": "ingest.collector.Source.Next",
"isAwait": false,
"callType": "internal_constructor",
"line": 6,
"raw": "c.source.Next",
},
],
"selfAttrs": { "Collector": { "dropped": { "readIn": ["…"], "writtenIn": ["…"] } } },
}callType ∈ self_method · self_attr_method · param_method · internal_func · internal_constructor · boundary · boundary_constructor (never unresolved — those live
in dropped-calls.json).
phase1/dropped-calls.json
{
"version": 1,
"metadata": {
"generatedAt": "…",
"totalDropped": 12,
"byCategory": { "builtin": 7, "bare_name": 3, "local_var_method": 2 },
},
"edgesByCategory": {
"builtin": [
{ "caller": "app.main.main", "calleeRaw": "print", "isAwait": false, "line": 9, "raw": "print" },
],
},
}Categories: inherited_method, self_attr_unknown, string_literal_method, builtin,
local_var_method, bare_name.
phase1/scan-coverage.json
The sibling of dropped-calls.json, one level up: that file accounts for every call
the analyzer refused to guess, this one accounts for every file it refused to claim it
had analyzed.
{
"version": 1,
"metadata": {
"generatedAt": "…",
"nScanned": 412, // files that reached the graph — i.e. graph.metadata.scannedFiles
"nUnparsed": 3,
"byReason": { "partial": 1, "unparsable": 1, "unreadable": 1 },
},
"files": [
// sorted by path, so an unchanged tree re-runs byte-identically
{ "file": "app/legacy.py", "reason": "partial", "detail": "the parse tree contains syntax errors…" },
{ "file": "ops/legacy.sh", "reason": "unparsable", "detail": "resolved is not a function" },
{ "file": "vendor/dangling.py", "reason": "unreadable", "detail": "ENOENT: no such file or directory…" },
],
}reason | What the parser got | In scannedFiles? | Gets a card? |
|---|---|---|---|
unreadable | nothing — the read failed | ❌ | ❌ |
unparsable | nothing — the grammar threw | ❌ | ❌ |
partial | real facts, but incomplete | ✅ | ✅ |
unreadable— discovery listed the path, but the read failed: a permission mode, a dangling symlink, a file the build deleted underneath the run.detailcarries the errno message.unparsable— the grammar threw, or returned no tree at all. Zero facts. This is what a shell script containingcaseproduces today.partial— the file parsed, butrootNode.hasError: tree-sitter parked the text it could not understand in an error node and carried on. Everything extracted from the rest of the file is real — what is missing is whatever sat inside that node, which is invisible from the outside. This is the reason to read the file yourself before trusting a page about it.
An empty files array is a positive claim — "every scanned file parsed cleanly". The
artifact being absent means the analysis predates this record, which is not the same
thing.
Why the first two are removed from scannedFiles
A file that yielded no facts used to stay in scannedFiles, so phase 2a wrote a card for it and
_coverage.json counted it as described. The handbook then asserted, as a parser fact, that a file nobody
had read contains zero functions. Dropping those paths here keeps one list meaning one thing: scannedFiles
is what the analyzer read, scan-coverage.json is what it could not.
phase2/cards/<rel>.json — FileCard
{
"version": 1,
"file": "ingest/collector.go",
"purpose": "Drains the queue and executes each task.", // "" = generation failed (backfilled)
"role": "domain_logic", // entrypoint|orchestration|domain_logic|io_transport|data_model|config|util|test|generated|other
"lifecycle": "main loop", // free-form short hint; "none" when not meaningful
"description": "…120-300 words…", // deep mode only
"functions": [
// deep mode only; facts from the graph, prose from the LLM
{
"id": "app.worker.Worker.run",
"qualname": "Worker.run",
"name": "run",
"className": "Worker",
"lineRange": [10, 13],
"signature": "def run(self)",
"calls": ["ingest.collector.valid"],
"calledBy": ["app.main.main"],
"extCalls": [],
"nCalls": 3,
"nCalledBy": 1,
"nExtCalls": 0,
"purpose": "…",
"dataFlow": "…",
"relations": "…", // may be empty; facts never are
},
],
}phase2/skeleton.yaml — Skeleton
metadata:
version: 1
archetype: demo task runner # one-phrase system shape
draftedBy: skeleton-synth # skeleton-synth | skeleton-doctor | user
stages:
- id:
stage-1 # any filename-safe id (^[A-Za-z0-9][A-Za-z0-9._-]*$);
# conventionally stage-N / stage-N.M / crosscut-N.
# Reserved page names (overview, index, register(s), …)
# are auto-suffixed by the normalizer.
title: Startup
description: Entry point wiring…
parent: null # substages point at their parent id
children: [stage-1.1] # derived; always rebuilt from parent on load
crosscut: false # true = cross-cutting infrastructureThis same schema is what you author by hand for --strategy member / --skeleton.
children may be omitted or stale — it is normalized on load.
phase2/assignment.json — Assignment
{
"version": 1,
"fileStage": { "ingest/collector.go": { "stage": "stage-1", "also": [] } }, // "unassigned" allowed
"buckets": { "stage-1": ["ingest/collector.go"] }, // primary stage only; disjoint
"coverage": { "nFiles": 5, "nAssigned": 5, "unassigned": [] },
}phase2/organization.yaml — Organization
metadata: { version: 1, nStages: 4 }
stages:
stage-2:
title: Task execution
groups:
- title: Core flow
summary: Everything this stage owns, in execution order.
files:
- { file: ingest/collector.go, purpose: '…', role: domain_logic, nFunctions: 5 }
orderedFiles: [ingest/collector.go, ingest/http_source.go] # flat reading order across groups
coverage: { nFiles: 5, nOrganized: 5 }phase3/narration.json — Narration
{
"version": 1,
"lang": "en", // en | zh
"systemOverview": "…200-350 words…",
"stageSummaries": { "stage-1": "…100-200 words…" },
}phase3/registers.json — Registers
{
"version": 1,
"registers": [
{
"id": "reg-task-queue", // ^reg-[a-z0-9-]+$
"semantics": "The FIFO list of pending tasks…",
"stages": ["stage-1", "stage-2"],
}, // only real stage ids
],
}Rendered handbook (handbook render)
<out>/
overview.md H1 title + 🗺️ system overview + see-also links
index.md recursive stage index (heading depth = tree depth)
register.md | State register | Semantics | Stages touched | (only when registers exist)
<sid>.md one page per content-bearing stage (summary, sub-stages,
organization groups, per-file cards with function details,
📊 state-registers section when touched)
agent/ (--agent-site) index.md · symbols.tsv · files.tsv · calls.tsv · stages/<sid>.md
html/ (--html) self-contained multi-page site (no external requests)
handbook.html (--html-single) one self-contained pageAgent index (--agent-site)
<out>/agent/
index.md the only file meant to be read whole: lookup recipes, the stage
table, the register table, coverage
symbols.tsv name → path:startLine-endLine, kind, stage, nCalledBy, signature
files.tsv path → stage, role, nSymbols, purpose[prose]
calls.tsv call edges: the caller always located, the callee located or
marked boundary:<import specifier>
stages/<sid>.md second hop: the stage's file list and its co-change pairsThe human artifact explains; the agent artifact locates. They are not two renderings of one text. Where an agent needs the explanation it is one hop away — each stage page links to the human page rather than copying it.
Why TSV and not markdown tables
- A markdown table would mangle 338 signature rows in this repo silently, because a
TypeScript union type contains
|. A tab does not collide with source text. - One fact per line survives truncation. Every grep recipe returns a complete answer on one line — name, location, kind, stage, callers and signature together — so a clipped result is still actionable.
- A tab anchors a whole column:
grep "^scan\t"matches the symbol namedscan, not every line containing the word.
Column order is value order, prose last, so a consumer that clips long lines eats prose before it eats a path.
The header lines
Every table opens with # comment lines naming the columns and the trust boundary — the
same disclosure the pipeline makes everywhere else, moved onto the artifact that carries it:
# name location kind stage nCalledBy signature
# parser facts. kind=fn is a function or method. kind=type:<class|interface|struct|record|enum|
# trait|alias|other> is a parsed type DECLARATION, span read off the declaration itself.
# kind=class-derived is the fallback where a language's adapter extracts no types: the SPAN is
# min..max of the class's METHODS, not of the declaration. Which languages are indexed and which
# fall back is stated in index.md under "coverage" — a miss here is not proof a name does not exist.
# nCalledBy counts callers inside the scanned set PLUS callers that reach it through an import
# (see calls.tsv boundary rows); a cross-package-only callee would otherwise read as dead code.calls.tsv states the matching one, and it names the difference between the two kinds of row
it carries:
# callerQualname callerLocation calleeQualname calleeLocation
# calleeLocation is path:line when the analyzer resolved it, or boundary:<import specifier>
# when the call leaves the scanned set — the name is known, the location is not and is not guessed.
# A call the analyzer could not pin down at all is in phase1/dropped-calls.json,
# never guessed here — so absence is not proof nothing calls it.Boundary edges, and why a monorepo needs them
A call that leaves the scanned set through an import gets boundary:<specifier> as its
callee location, never a path. The name is a fact; the location is not, and is not guessed.
This is not a footnote in a monorepo — it is most of what an agent wants to know. Measured on
this repository: 1,063 of 3,565 edges are boundary edges, 284 of them into
@handbooks/core. With resolved edges only, checkLanguage — called four times from another
package — appeared with zero callers, which an agent reads as dead code. That is a wrong
pointer, not a gap, and a wrong pointer is the failure this artifact exists to prevent.
For the same reason symbols.tsv's nCalledBy counts boundary callers as well as in-package
ones, and its header says so. boundary: cannot be mistaken for a path, so nothing is
invented by including them.
Type rows, and the fallback beneath them
symbols.tsv carries three kinds of row. fn is a function or method. type:<kind> is a
parsed type declaration — the span is read off the declaration itself — over a closed
vocabulary: class, interface, struct, record, enum, trait, alias, other.
record is not folded into struct because a Java or C# record is a reference type, and
struct is the one word in this vocabulary that also means value type. other is
load-bearing rather than a dustbin: a Go defined type (type Celsius float64) is not an
alias, a Rust union is not a struct, a Java @interface is not an interface — and
signature carries the declaration as written, so the native keyword is never lost.
Which languages extract real types is declared per adapter and disclosed in index.md,
the same way analysis fidelity is (invariant 3). AdapterCapabilities.typeKinds is a list
rather than a boolean, because an adapter could find classes and miss every interface; [] is
a positive claim, and the field being absent means the artifact predates it and is reported as
unknown, never as zero.
All twelve precisely-parsed languages extract them — C++, C#, Dart, Go, Java, PHP, Python,
Ruby, Rust, Solidity, Swift, TypeScript. Shell declares [] because it has no type
declarations at all. The five generic-tier languages (Kotlin, Objective-C, OCaml, Scala, Zig)
declare [] deliberately: their adapter matches patterns rather than parsing precisely, so a
type row from one would be indistinguishable in the IR from a precisely-parsed one at a
lower fidelity — which is exactly what invariant 3 exists to prevent. They keep the
class-derived fallback instead.
Measured against real repositories, counting rows against the declarations a grep can see:
PHP and Solidity 100%, C# 98.9%, Swift 97.0%, Dart 96.1%, Ruby 92.7%, C++ 87.5% (of files
that parsed cleanly; spdlog's macro-heavy headers defeat the grammar itself, which
scan-coverage.json records). Every shortfall is a declaration the adapter refused to guess
at — a type declared inside a function body, or a name that collides under the arity-free id
model — never a guessed span.
class-derived is the fallback where an adapter extracts no types: the span is min…max of
the class's methods — where the members are, not where the declaration is — so it is
labelled rather than presented as a parsed fact. On this repository, adding real type
extraction dropped class-derived from 45 rows to 19, and every remaining one is an object
literal (TYPESCRIPT_SPEC, silentLogger) rather than a type declaration — which is exactly
what the fallback should be catching.
One cost of taking the declaration's own span: where a declaration carries a leading attribute
or annotation, the span starts there, because that is where the grammar's node starts. The
signature is protected from it — if the cap would cut the type's name away, the attributes are
elided with a leading … instead, because a signature that does not name what it declares is
not a shorter signature but a useless one.
The disclosure matters more than the coverage: an agent that greps a type name, gets nothing,
and concludes the type does not exist is the wrong pointer this artifact exists to prevent.
Constants, variables and macros are indexed in no language, and index.md says so.
Freshness
index.md's header carries HandbookModel.provenance — { commit?, generatedAt }, read
from the run manifest. Line numbers are now the primary payload, and a stale line number is
the one fact that goes wrong silently, so the artifact says when it was made and against
what.
SKILL package (handbook skill)
<out>/
SKILL.md frontmatter: name (<slug>-handbook) + description
("Use when … Do not use …"); body = routing protocol
references/
overview.md index.md registers.md
stages/<sid>.md
agent/ (--agent-dir) index.md · symbols.tsv · files.tsv ·
calls.tsv · stages/<sid>.md
coverage.json (optional) {schemaVersion, summary, files:[{path,stage,sha256}]}Validation contract (handbook validate): frontmatter has exactly name + description;
description states use AND don't-use; body references references/index.md and directs to
the actual source; overview/index/registers/stages present; index links every stage page;
no duplicate coverage paths; with --source, hashes must match the live tree. A
references/agent/ directory is optional, but when it exists it must carry index.md and
all three tables — the index and its fact tables ship together or not at all.
Planner output (handbook plan)
A markdown plan: prose summary → EDIT blocks → one declarations JSON block.
### EDIT 1
- file: `app/engine.py`
- where: `Engine.spin (~5)` — add retry
```old
<byte-exact current text, ≥3 context lines each side, unique in the file>
```
```new
<replacement text>
```
```json
{ "will_modify": ["Engine.spin"], "will_add": [], "will_remove": [] }
```Resync case directory (handbook resync --case)
<case>/
edited/ the changed source tree (required)
plan.md change description; its ```json declarations block
(will_modify/will_add/will_remove) sharpens scope (optional)
change.diff unified diff; PRESENT AND EMPTY = "nothing to resync" (optional)
resync-report.json written by resync: {skipped, changedFiles, addedFiles,
deletedFiles, affectedStages, cardsRegenerated, narrated}Environment variables
Every variable Handbooks reads, the naming rule that generates them, the .env cascade, and which ones must never enter a config file.
Language support
18 languages across two analysis tiers — which extensions each claims, what the generic tier gives up, and the two caveats worth knowing before you hit them.