Storage Architecture

See the permanent storage assessment for source-backed ownership baselines, lossless codec experiments and regression budgets.

Status: v0.5.0 — Parquet project storage shipped Last Updated: 2026-08-30

---

Overview

The native directory authority, lifecycle policy, and adversarial checks are documented in Directory capabilities.

graphforge-storage owns project generations and participants, admission and recovery, Parquet catalogs and write paths, derived indexes, and portable interchange. Runtime scans use GraphCatalog and DataFusion TableProvider implementations over the selected project’s data. Query-language semantics remain in the compiler and execution layers.

Shared logical and physical Arrow schema definitions live in graphforge-ir::arrow_schema; storage reexports the existing public schema names. Relational compilation consumes neutral schema data and has no production storage dependency. Execution owns path hydration through retained catalog providers.

The unused StorageProvider, StorageRow, and ParquetProvider stubs were removed in #1006. They had no consumers and their scan path returned NotImplemented; they never provided backend selection.

---

Lifecycle implementation ownership

Research Version and Proposal publication use the same Project authority under research@4 and registry@4. ADR 0039 records the Version foundation; ADR 0043 extends its single CURRENT publication to accepted parent content, review history, selected proof, exact contribution mappings and receipts. Context restoration preserves unrelated heads and permanent receipt/identity/acceptance history. Selected CAS content can outlive its source generation; an origin Version citation does not itself retain that Version’s entire payload. Frozen Proposal root release permits explicit obsolete-payload deletion and cleanup, while independent accepted proof roots retain required content. The registry admits only its exact current contract version; incompatible older records fail closed, without an implicit upgrade or lossy history conversion. See the research workspace contract for capacity limits and the separate receipt and deduplication lifetimes.

Object-store manifest updates, materialization, installation, and garbage collection have private modules under graph_object_store. Read/publication leases and shared lifecycle locking remain in the parent. project_publication retains commit order, CURRENT replacement, and reconciliation; its participants and control modules own staging and bounded journal/atomic-file operations. Checkpoint lifecycle and locks remain in project_checkpoints, with registry pair recovery and restoration participant construction as private children.

The filesystem crate keeps common capability types and public exports in its root. cache_io, platform, windows, and windows_cas own the corresponding native implementations and direct tests. See the source inventory.

Value contract and current dependencies

graphforge-storage currently depends on both graphforge-ir and graphforge-ontology in production. Its writer and GFDR journal consume IrLiteral; catalog and topology paths consume IR’s runtime catalog and tagged IDs; ontology/composition adapters supply semantic binding validation. Storage is therefore not yet independent of compiler-owned value definitions.

ADR 0025 chooses graphforge-value for the shared value/ID/catalog encoding contract; extraction is pending in #1011 and #1012. Storage retains physical schemas, I/O, and project admission. Moving definitions must preserve existing bytes and public Arrow layouts. graphforge.ir_version metadata is descriptive and does not admit a project: container, participant and value compatibility use their own checked contracts. See project format compatibility.

Parquet persistence

Parquet is the graph data format used by the Rust storage implementation. It:

• Stores graph tables and opaque domain-owned participants as columnar Parquet files; graphforge-storage does not define provenance or knowledge semantics
• Carries GraphForge metadata at the file level (ontology version, IR version, query ID)
• Persists derived compiled ontology runtime tables for rapid startup; the CURRENT-selected workspace’s canonical ontology JSON remains durable authority

Parquet file-level metadata:

graphforge.dataset_kind      = "topology_nodes"
graphforge.ontology_version  = "core-2026.05"
graphforge.writer_version    = "0.5.0"
graphforge.ir_version        = "0.1.0"
graphforge.query_id          = "01J..."
graphforge.provenance_policy = "conservative_min"

Project metadata and manifests use JSON. A future storage backend would need to satisfy the actual generation, admission, publication, and scan contracts; the removed StorageProvider stub did not establish that support.

---

Identity and Surrogate Keys

GraphForge uses a dual-key pattern for all first-class objects:

Key	Type	Purpose
UUID (*_uuid)	FixedSizeBinary(16) — UUIDv7	Canonical stable identity. Globally unique. Immutable. Survives project merges, offline generation, and cross-analyst exchanges.
Surrogate (*_id)	UInt64	Execution-time optimization. Assigned at ingest/load time. Used for DataFusion join operations. Never exposed in public API results.

Why UUIDv7

UUIDv7 (RFC 9562) is time-ordered within a millisecond, globally unique without coordination, fits in Arrow FixedSizeBinary(16), and supports offline generation on mobile devices or air-gapped systems. See UUID identity authority for identity lookup, including what changed for projects written before the membership index was removed.

UUID byte order, accepted text form, content-derived UUIDv8 records, canonical Arrow bytes, and domain-separated SHA-256 fingerprints follow the frozen canonical fingerprint v1 contract.

UUID→Surrogate mapping

The relational lowering layer maps node_uuid → node_id once at scan time. All DataFusion join operators use integer surrogates (node_id, edge_id, src_id, dst_id) for performance. Results project back to UUID columns before returning to the caller.

Rule: UUIDs appear in every public API result schema. Surrogates are execution-internal and must never appear in API outputs.

The v4 reverse authority makes node_id → node_uuid a generation-pinned, disk-bound read rather than a graph-sized process cache. Its manifest names typed immutable artifacts: packed UUID payloads for contiguous ordinal ranges, UUID-sorted (UUID, surrogate) forward records, plus sorted surrogate tombstones. Current ordinal descriptors use wire version 5 while retaining the v4 logical authority and API names. Admission checks each retained file in one streaming pass that jointly derives its required seed-zero XXH64 checksum, block fences, structural checks, and mapping commitment. Payload SHA-256 identities are computed at publication; the domain-separated mapping commitment remains a required contract identity. Aggregate admission counters report the exact artifact bytes, sequential read calls, and peak bounded buffer. A lookup accepts only a bounded request batch, sorts and deduplicates it, applies newest tombstones, and reads authenticated fixed-size ordinal blocks while restoring caller order. Storage grows with retained identities rather than the largest sparse surrogate, and anonymous buffers do not grow with graph cardinality. Version-3 state is an explicit rebuild-required disposition; a v4 reader never interprets or mixes v3 reverse records.

Fixed-hop consumers such as projection-aware expansion retain one authenticated generation handle and submit bounded destination-ID batches to it. They do not reopen or reauthenticate the full authority for each chunk. The lookup restores caller order while exposing sanitized requested/unique/found, selected-range, logical-byte, coalesced-call, and peak-buffer evidence. Typed failure evidence counts authentication failure without containing graph identities or paths. The explicit migration disposition is CanonicalTopology: rebuild scans canonical topology under the durable rewrite and never treats a v3 reverse run as v4 authority. Its temporary-byte peak covers every coexisting external-sort run, merge output, surrogate projection, and final artifact through storage-owned lifecycle accounting, including retained staging copies and control-record temporaries. It is not an artifact-only approximation or a recursive observation of the active scratch tree.

Admission also streams both forward and ordinal authorities through the same domain-separated mapping commitment and count. Strict forward UUID ordering and that bounded reconciliation reject duplicate, missing, or cross-generation identity mappings without constructing a graph-sized in-memory map.

The reader holds ordinal-v4.lock shared only during admission, then releases it after pinning the immutable files; every v4 publisher holds that same stable file exclusively across artifact installation and manifest replacement. Each admitted file’s stable identity, length, and high-resolution modification time are retained as a fast change detector. That cooperative ownership does not replace corruption refusal: every selected ordinal or tombstone block is checked against its required manifest XXH64 checksum before its bytes can produce a result. A non-cooperating mutation therefore fails closed even if it preserves the inode and restores the original modification time. Adjacent selected ordinal blocks may share one bounded read only when their intervening gap and combined span fit the configured cap; every constituent slice is still checked against its own checksum. Version-3 rebuild recognition uses the existing canonical v3 manifest and run-descriptor validator, so a minimal version tag or a v3 document mixed with v4 fields fails closed.

Objects requiring UUID identity

Object	UUID column
Node (entity)	node_uuid
Edge (relationship)	edge_uuid
Document	doc_uuid
Provenance event	provenance_uuid
Analyst/User	analyst_uuid
Project	project_uuid
Workflow	workflow_uuid
Embedding	embedding_uuid
Source reference	source_uuid
Ranking output row	rank_uuid
Clustering output row	cluster_uuid
Generated artifact	artifact_uuid

---

Storage Layout

GraphForge is pre-v1 and does not support older project formats. The normative pre-v1 compatibility policy permits only the v0.5 project/container contract and rejects historical inputs without mutation.

GraphForge organises a project as immutable, complete generations. CURRENT is the only publication authority; graph, provenance, and knowledge participants become visible together through one atomic pointer replacement. The normative layout, fsync order, reader leases, recovery rules, and failpoints are frozen in ADR 0013. The public acknowledged-durable boundary and isolation honesty rules are frozen in ADR 0018.

Named checkpoints, lease-pinned historical reads, logical diff, and complete-workspace revert are defined by ADR 0014. A revert publishes a new complete generation; it never moves CURRENT backward. Active checkpoint references add explicit retention roots, while deletion releases only that root and cannot invalidate an already leased reader.

Research Version retention (M11).

The research workspace contract adds semantic Branch, Version, and Proposal state above these generation/checkpoint primitives. A Branch’s immutable base and every retained Version pin the selected graph, required ontology/local Artifact dependencies, and comparison baselines. Genealogy identifiers alone must not pin the entire ancestor; explicitly retaining a whole Project Version does. Child Branches, Proposals, and acceptance provenance retain their required closure across cleanup, compaction, and reopen with a compatible reader. Explicit deletion cannot invalidate that closure.

External-only evidence preserves references and integrity metadata, with missing or unverifiable bytes disclosed. It does not gain an archival guarantee merely by appearing in a Version. A conceptual Branch overlay does not authorize an unmanifested state store or reads from a mutable parent. Reuse complete, authenticated publication and shared immutable payloads; do not duplicate a billion-edge Project to create a small Branch or pin its unrelated content indefinitely. Measure retained bytes after parent evolution and cleanup as well as copy cost; repack shared physical units when necessary to reclaim unrelated content. Historical expansion outside the retained closure can be unavailable.

Branches share the Project container and its sole CURRENT authority. Parent integration and its acceptance receipt publish in the same generation; Branch acceptance status derives from that receipt, not a second commit. The existing ADR 0018 failure semantics remain authoritative: before linearization preserve prior state; after linearization reconcile/report committed state, never roll it back. Forks provide independent Project authority. M11 adds no pre-v1 migration guarantee.

The layout below documents current storage. Research participants and retention roots are registered by their native owners; the acceptance evidence maps real Branch/Proposal retention, cleanup and recovery tests.

project/
├── FORMAT
├── CURRENT             # sole committed-generation pointer
├── locks/writer.lock
├── transactions/
├── generations/
│   └── <generation-uuid>/
│       ├── lease.lock
│       ├── manifest.json
│       ├── graph/              # file-backed graph workspace (optional; with graph/files)
│       │   └── deltas/         # authoritative mutation runs (ADR 0019; not adjacency)
│       └── participants/
│           ├── graph/...       # files.json (inventory) and semantic bindings
│           ├── workspace/
│           │   ├── configuration.json
│           │   └── ontology.json
│           ├── provenance/...
│           └── knowledge/...
├── cache/               # derived and source-fingerprint keyed
└── trash/

The graphforge-generation manifest requires wire version 2. Every participant descriptor binds exact length, SHA-256 publication identity, and mandatory fixed-width seed-zero XXH64. Data-bearing Arrow/Parquet participant reads check length and checksum; JSON control participants retain SHA-256 authentication. Old generation manifests and missing or malformed checksum metadata are refused. The unchanged project FORMAT and CURRENT envelopes keep their version 1.

Graph publication and the confidence-assessment mutation select replaced participant families from manifest descriptors before opening sibling payloads. Unchanged immutable participants in those paths are carried into the child’s complete manifest by a stable-path hard link to the pinned parent file; publication rechecks file identity, length, XXH64 and SHA-256 before advancing CURRENT. This narrow hard-link allowance applies only to manifest-authenticated participant payloads. Project control files remain single-link files, and JSON participants retain their SHA-256 content check. Other knowledge publication paths may still materialize sibling participants. Each generation remains independently readable and owns a flat participant inventory, so retention and garbage collection do not follow manifest chains.

A minimal committed generation declares graph@1 and workspace@1. workspace@1 contains canonical JSON records for explicit ontology absence (or an adopted advisory/strict ontology) and authoritative registered project configuration. Project open validates these records before opening graph data. New publications store graph workspace files under the generation-owned graph/ tree with a graph/files inventory participant. Persisted legacy graph/snapshot Arrow envelopes are refused. Root YAML/JSON and environment settings are inputs only and cannot override the selected generation. Version 2 of graph/files replaces the expanded per-generation inventory with a compact authenticated Patricia/radix root. Immutable payload and manifest objects are addressed by SHA-256 in the project object store, so an update writes only changed payloads and the compressed path-copy root path while unchanged objects are reused byte-for-byte. A storage-owned root-bound state authenticates an existing inventory once per publication sequence; individual updates cannot substitute a caller-owned cache and do not rescan all prior descriptors. Open resolves the bounded manifest and verifies every selected payload before exposing the generation. Ordinary API open and restore dispatch on the declared participant version. Retired expanded inventory versions 1/3, compact root versions 2/4, and persisted Arrow graph snapshots are refused; pre-v1 projects in those formats must be recreated. Current expanded inventories use versions 5/7 and compact roots use versions 6/8, with a required seed-zero XXH64 checksum beside every SHA-256 payload name. The descriptor and payload must agree. Immutable generations remain unchanged while a writable facade materializes a private workspace.

The graph-files role is inferred from the relative path. It does not select a different integrity policy: current-format admission checks XXH64 and exact length of every payload it admits, without a payload SHA-256 pass: bulk payloads on first touch, the rest at open (see below). SHA-256 still names CAS objects and authenticates control metadata and existing trust boundaries. The standalone graphforge verify command and whole-store forensic scanner are retired. Portable package verification remains required. See ADR 0049 for the pre-v1 format policy and trust assumptions.

Opening a compact generation is O(files), not O(payload bytes). The open authenticates the manifest and route table and checks each payload’s presence and exact length, then hard-links the payloads into the private workspace. A payload’s content is checked (exact length and XXH64) in one of three ways, chosen by path in graphforge_storage::graph_admission:

• First touch. Nodes, edges, property fragments, search segments and the v4 forward and ordinal identity runs are checked when the first reader opens each object, memoized for that hydration, and a refusal is memoized too. Readers reach the check through ReadPathFile::admitted, open_admitted, and current_search_artifact. Anything that would bless a payload’s current bytes (inventory capture for publication, rewrite baselines, appends) admits it first, so corruption is refused rather than republished under a fresh digest. (The identity handle authenticates the blocks it reads itself, so for the identity runs only that capture check applies.) The property inventory of a compact generation holds each fragment’s manifest entry and reads nothing at open: the exact length is checked, the footer (schema and row count, which the manifest does not carry) is read the first time a route is used, after checking the content directly on the read-only content-store inode. Schema and row-count failures propagate through planning; only an absent route receives a base-schema default. Footer admission is memoized for the inventory’s life. A route is “used” only by a plan that reads property values: the catalog names property tables without opening them, and a read plan that references no property value (graphforge_ir::property_demand returns None) compiles against key-only route schemas. Its scans still join each route by node_uuid/edge_uuid, so a route such a plan actually reads is admitted and authenticated at execution. Each property payload read copies and authenticates its bytes into a private snapshot before decoding, so an in-place change after a successful read is refused and a concurrent change cannot alter the bytes already authenticated for the decoder. A checksum refusal under indexes/search/ is a hard validation error, never a reason to rebuild the index. Text find takes its freshness identity from the same manifest (node and node-property objects by name, length and XXH64), so it reads no edge object and re-reads no source to recheck freshness.
• Self-authenticating. Ordinal identity runs (block checksums in their manifest), CSR shards (per-shard XXH64) and delta runs (verified at replay) carry their own authority; hydration neither reads nor registers them.
• Eager. Every other payload is small metadata or a sidecar with many by-name readers (generation counters, label encoding, catalogs, adjacency build records, CSR shard manifests, unclassified files). Hydration hard-links it from the object store and checks its length and XXH64 as it links, so an unforeseen reader fails closed. Only the route table and the small mutable ordinal-identity controls (manifest, receipt, lock, tombstones) are copied into single-link private files. The forward and ordinal identity runs are hard-linked read-only; the identity handle admits a shared inode only when it is read-only and authenticates each block it reads.

GraphFilesOpenEvidence.bytes_validated is declared length validated, and bytes_checksummed is content actually read and checksummed while hydrating. ResolvedProjectGeneration::graph_files_inventory still checks every payload (memoized); unadmitted_graph_files_inventory is the length-only form an open uses. Every mutating commit publishes a compact root, so a mutated project opens as cheaply as a constructed one. Expanded (V1) and delta-bearing generations that already exist are still readable: they verify and copy the whole tree at open, and convert to a compact root on their next commit.

The same-inode, same-length corruption tests in graphforge-api/src/workspace_hydration/tests.rs cover every role, including real adjacency and search index publications:

Role	Inventory source	Corruption coverage
Topology	topology/, including the current topology/runtime_catalog.parquet	Nodes and edges are refused on first touch (open or the first query/recount); sidecars (generation counters, label encoding, surrogate tails, runtime catalog) are refused at open; a mutating commit refuses a corrupted edge payload instead of republishing it
Properties	properties/ and edge_properties/	Real property payload; open checks each fragment’s exact length only, and every read (projection, filter, find, portable export, SET) refuses a corrupted fragment on first touch
Index	Published indexes/adjacency/ CSR and indexes/search/ artifacts	Both real build paths reach the compact inventory; adjacency build records and shard manifests are refused at open, CSR shards by their own checksum on read, search artifacts on first touch
Delta	deltas/ journal runs	Role-level full-admission test uses an opaque fixture; journal replay has separate validation
Catalog	Top-level semantic-routes.json	Real control payload, authenticated by SHA-256 with the manifest
Other	Any admitted graph workspace file outside the named prefixes	Opaque fixture is copied and checked while hydrating, so it is refused at open

runtime_catalog.parquet at the graph root would be Catalog-role, but the current compact writer places it under topology/. The role classifier permits that top-level path without relying on a writer to produce it. Search artifacts are also derived, but an explicit search build publishes them under indexes/search/ in the graph-files inventory. The cache outside a published graph workspace is separate from this inventory.

Semantic routes and portable filenames

Node labels and edge relations are semantic UTF-8 identifiers, not filesystem components. Rust owns their translation at every node-property, edge-property, and topology-relation path boundary. A physical route component is r- followed by the 64 lowercase hexadecimal digits of SHA-256 over the exact semantic UTF-8 bytes. Its length is always 66 bytes, including for long semantic names.

The graph-owned semantic-routes.json control record reverses each component to its exact semantic route. It belongs to the authenticated graph-files inventory and travels with publication and portable export/import. Decoding verifies the table digest, canonical serialization, unique entries, component derivation, and resource budgets. Every physical route must resolve through the table; unreferenced table entries are rejected. A digest collision between distinct semantic routes is a typed refusal, never an overwrite. The table is bounded to 64 MiB and 100,000 entries; exceeding a budget fails explicitly.

No case folding or Unicode normalization changes a semantic identifier. Names such as CON, AUX, trailing-dot names, normalization variants, and literal r- prefixes retain their original meaning in public results and metadata. Readers use authenticated layout authority, never filename-prefix guessing. Current raw versions 5 and 6 cannot contain the reserved mapping record; mapped versions 7 and 8 require it. Retired versions 1–4 are refused.

Raw-to-mapped route conversion operates only in an owned private workspace. It authenticates source identity and bytes, stages the encoded files and complete mapping in the same durable rewrite, and recovers that rewrite before reading route authority. Physical migration preserves graph UUIDs, ranks, values, and semantic generation counters. An authenticated legacy publication must be translated while copying into the private workspace, so Windows never needs to create an intermediate raw reserved filename. Published trees and CAS objects remain immutable.

Mapping updates share the graph rewrite transaction. Writers authenticate the current mapping under the rewrite lock before extending it, and transformations that remove or rename physical routes rebuild the mapping from their emitted inventory. Path containment, no-follow opens, native identity, link-count, collision, and exact-inventory checks still apply to encoded paths.

Construction checkpoint versions 8 and 9 select mapped output explicitly. Version 9 also reclaims accepted payloads after authenticated shape and shaped payloads after authenticated encoding; see construction supersession. Versions 6 and 7 retain their original encoding when resumed; versions 7–9 use the same compact detail codec. Mapped construction publication combines the retained parent routes with newly emitted routes and verifies the complete mapping before installing the version-4 manifest. Legacy parent payloads can retain their authenticated CAS objects while their logical route paths change.

Initial builds. An initial build (an empty project, from an import session or the chunk API) produces this same encoded inventory without staging or shaping. The bulk builder ranks nodes and edges, and every canonical artifact is a projection of the ranked arrays: sorted node UUIDs, sorted edge UUIDs, and (src, dst) per edge. Each artifact is emitted once and hashed as it is written (ADR 0058, resumable import). Appends to a non-empty project keep the staged path above, and the builder never reads a parent generation.

• Restart, not resume. A crash, cancellation or error discards the build’s scratch, and the next validate reruns from the sources. A rerun produces the same graph as an uninterrupted run (ADR 0038 property 1, amended for initial builds).
• Sources are read in place. A registered Parquet source is pinned by path, device, inode, size and modification time and is not copied. The source SHA-256 is folded from the bytes the build itself reads. A source that changed is refused; a rewrite that preserves the whole pin is not detected.
• Chunk-API builds spool. On an empty project each accepted chunk is one durable Arrow IPC file, and sealing builds from the spool. A crash leaves the spool intact, and every decode is authenticated against the digest acknowledged at acceptance.
• Over budget, scratch. When the estimate exceeds the memory budget, compact edge records, adjacency entries and normalized property rows use disposable CRC32C scratch without SHA-256 or fsync. Property runs sort exact schemas by UUID, then stream the existing logical windows, owner projections and 4 MiB fragments. Physical scratch frames do not change catalog IDs, cuts, ordinals or published bytes.
• Node tables use scratch when needed. When about 56 bytes per node plus fixed workspace exceeds the budget, node identities, endpoint resolution, degrees and CSR key ranges use bounded node-UUID range partitions on scratch (#1929).
• Route. The route is a function of the pinned parent: an empty project is an initial build and runs the bulk builder, any other is an append and stages. Nothing is recorded, so a later change in free memory cannot send a started build to another route.

Registered-source decoding has a separate pre-allocation envelope (#1918): Parquet and IPC inventory is admitted before native decoding, and source tasks reserve page, Arrow-array and normalization workspace from a shared pool. That reservation is distinct from the normalized-property scratch bound and is not a whole-process RSS guarantee; see resumable import.

Scratch passes run concurrently (#1938). The partitions in flight are the threads that run every scratch pass; the planner picks the largest worker count, up to the available construction lanes, whose per-worker reservations (and, for property-bearing input, an 8 MiB property run each) fit the budget. Partitions are claimed in order and admitted by a memory gate, so the reservations in flight never exceed the budget. The build reports the count as scratch_concurrency; it is 1 only when the budget admits no more.

The staged pipeline serves appends only; no initial-build plan reaches it. Publication semantics are unchanged.

Publication does not copy the encoded files. On unix it links the same inode to graph-objects/sha256/<2>/<62> and acknowledges the bucket directory, so each published byte is written once. Each encoded file was synced once by its producer before the checkpoint that pins the inventory, and the inventory records that (seal_protocol), so linking its objects runs no second file barrier; an inventory without the record is sealed at install (ADR 0013 ordering unchanged). The encoded name stays until the session’s private tree is retired, so a publication that stopped at any point reruns by deduplicating against the existing object. Nothing reads the file back at install: exact length and XXH64 are checked on first read (ADR 0049) and at the commit boundary before CURRENT moves. If the object store cannot take a link (EXDEV), the install copies explicitly; Windows always copies. Every writer of the encoded tree creates a fresh file exclusively or by temp-and-rename, and the shared inode is sealed read-only, so no later write reaches a published object.

Read sessions retain one admitted route inventory with their catalog and adjacency provider. Publishing a later generation replaces the facade’s provider; an existing lazy stream keeps its original inventory and private index artifacts through completion.

Immutable property snapshots

Node and edge properties use full-snapshot-v1 fragments under properties/<component>/<generation>-<ordinal>.parquet (fixed-width decimal identity) and the corresponding edge_properties tree. Each row is the complete property state for one UUID; an explicit tombstone deletes the whole row. Admission authenticates every canonical generation and ordinal named by the committed graph-files inventory. Readers merge all authenticated fragments by UUID and descending (generation, ordinal) authority; the first row for a UUID is its complete current map or tombstone. Unchanged UUIDs remain authoritative in older immutable fragments, while a newer tombstone prevents a deleted UUID from resurfacing. A write window composes repeated SET/REMOVE operations once and publishes only its changed UUID snapshots without a full-route decode or any prior-fragment rewrite. A PATCH/REMOVE producer performs at most one authenticated targeted batch lookup for the window’s UUID set, with the same zero-per-record-seek scanner; sealing consumes those complete staged rows and does not read historical fragments again.

Writers cut logical fragments in UUID order at MAX_PROPERTY_FRAGMENT_ROWS (65,536) and MAX_PROPERTY_FRAGMENT_BYTES (4 MiB of Arrow value, offset and presence bytes). These boundaries give dense ordinals and disjoint UUID ranges within a generation. A row larger than the logical target stays intact: the target does not impose a new limit on valid property values.

The separate physical bound is MAX_PROPERTY_OBJECT_BYTES (4 MiB), including all Parquet headers, pages, schema metadata and footer. Writers check the final encoded size; compression is not assumed to shrink its input. An oversized logical Parquet stream is split into fixed-size byte segments, each stored in a valid Parquet envelope with the graphforge.property_object=1 marker, logical length, part count and part index. The first object uses the canonical fragment filename; later objects append .part-<20-digit-index>.parquet. The envelope has a fixed schema and is uncompressed, so it adds no second compression pass. Every completed envelope is checked against the physical bound before publication.

Parts share one logical fragment identity. The authenticated inventory owns the complete ordered part set, and the shared property byte reader exposes the original Parquet stream to existing Arrow decoders. It loads only requested ranges, retains at most one decoded part, and verifies every physical object through existing manifest checksums. Missing, duplicate or inconsistent parts are corruption. Logical fingerprints and query results use decoded values, never envelope rows. Reading a large requested value necessarily reads all of its parts; storage chunking does not remove existing query resource budgets.

Construction, SET/REMOVE windows, delta replay and graph projection use this publication boundary. Oversized projected flat snapshots become canonical generation-zero fragments. Existing plain Parquet files remain readable, including oversized files written before the cap; portable copies preserve their representation. Newly encoded objects use the bounded representation. Projects containing envelopes require an envelope-aware reader; older readers do not interpret the envelope’s binary payload as property values.

Each fragment also carries the authenticated graphforge-property-live-schema/1 route summary: an exact live-UUID count for every currently present property key. Mutation preparation updates those counts from the targeted UUIDs’ old complete maps to their new complete maps; work is bounded by the mutation window and schema width, never total rows or fragment history. Repeated writes in one window consume the already-staged summary. The newest fragment summary is the logical schema authority: keys with a positive count retain their authenticated physical type, while historical keys whose count reached zero surface as Arrow Null. Physical historical schemas remain available for decoding older UUID snapshots. Malformed counts, underflow/overflow, a live key without an authenticated physical field, or conflicting semantic/type metadata fail closed. Legacy routes without this summary retain their historical schema union until rewritten by an explicit migration; targeted writes never invent counts for untouched UUIDs.

Readers derive route authority from the committed graph-files inventory, validate canonical fragment identity, schema/semantic metadata, strictly sorted unique non-null UUIDs, and tombstone invariants, then perform a bounded disk-backed newest-wins merge. SQL and direct APIs share that scanner. SQL emits bounded Arrow batches; LIMIT changes emission only after authority validation. Decoded rows enter fallible bounded scratch runs. The final emitting merge does not begin until every authenticated fragment reaches clean EOF and all page, Arrow, row-order, tombstone, and live-byte checks succeed. A late decoder or resource failure therefore discards the runs and returns a typed error with zero rows observed by direct callbacks or DataFusion—even for a projected LIMIT 1 plan.

The query scan (PropertyOverlayExec, shared by Cypher and SQL) does not spool. Every fragment is already a run sorted by UUID, so the newest snapshot of each UUID is a k-way merge of the fragments. A fragment opens when the merge frontier reaches its smallest UUID (from its footer statistics) and closes at EOF: disjoint base fragments are read one after another and an overlapping mutation fragment stays open beside the current one. The merge shares the same live-byte budget; a route whose overlapping fragments cannot be open within it is refused with that budget’s error rather than spilled, because a read must not write. A scan that carries a LIMIT holds its rows until the whole route has validated, as before. The merge reads rows in UUID order, so results keep the order of the spooled merge.

A completed join frontier can nominate destination UUIDs before a property scan opens its fragments. A LEFT enrichment whose scan projects property payload nominates from its build side, whatever produced that frontier: the join still decides matches, NULL preservation, and output multiplicity, and a LEFT join keeps every build row, so pruning the scan cannot change a result. Key-only destination scans require proof that the frontier comes from a traversal destination (or the exact expansion source an equality join selects); a topology scan’s own UUID is never such a frontier, so count(*) still authenticates the whole route it joins. When an embedded LEFT join projection hides the UUID, the rewrite retains it through repartitioning and removes it afterward; rewrites that cannot lift the key reject unknown partition keys rather than turning optimizer metadata into an executable hash expression. The join keeps its partitions and its build rows stream: a tap on the preserved side records only the UUID key column (one 16-byte key per distinct row, charged to the query pool) and forwards every batch, so no build row is concatenated and a frontier of wide rows cannot overflow the 32-bit offsets of a variable-width column. Nothing has to bound the frontier: no equality anchor or row fetch is required to nominate. The nomination bounds itself instead, because a frontier that names much of its route is cheaper to scan than to point-read. It abandons, and the scan then reads the whole route exactly as the un-nominated join would, once the frontier holds more than half of the route’s planned rows (a footer upper bound, or the manifest estimate when footer statistics are off; an inexact estimate can abandon too early or too late but never changes a result) or once its key set would outgrow an eighth of the query pool. The plan has the same shape either way, and a pool that refuses a reservation below that cap is still an error.

The Rust facade regression property_read_pushdown compares fixed-fanout anchored queries while unrelated destination rows and fragments grow, at one, two, and four partitions. It warms authentication metadata, checks read-path byte growth and zero application write counters, and preserves result counts. enrichment_frontier_reads does the same for enrichments whose frontier no equality selects (after ORDER BY ... LIMIT, a further hop, or a label scan of a few nodes with nothing bounding it) and whose projection drops the node UUID, on a route large enough to plan partitioned hash joins; it also checks that a frontier past half the route abandons its nomination and returns every row. Run them with cargo nextest run --locked -p graphforge-api --test property_read_pushdown --test enrichment_frontier_reads in an isolated CARGO_TARGET_DIR. SF1 syscall evidence must distinguish query file-content writes from startup materialization and empty spill-directory creation; application counters alone do not cover all process filesystem activity. Measurement results belong on the producing issue.

A node-property scan also accepts column = literal for Int64, Utf8 and Boolean columns, offered by the plan as a hint (the filter stays in the plan). Row-group statistics select the fragments and row groups that can hold the value; their rows nominate candidate UUIDs; the candidates are then resolved against every fragment whose UUID range can hold one, so a newer snapshot or tombstone still decides the value. More than 262,144 candidates, or a column stored with another type in any fragment, declines the hint and the scan streams the route. A lookup on a column whose values rise with the UUID therefore authenticates the fragment that holds the value; a scattered column reads that column of every fragment. Admission retains the stable root capability plus each fragment’s authenticated path, native file identity, length, XXH64 checksum, and schema—not one OS handle per historical fragment. A scan opens fragments on demand without following links, requires the admitted device/file identity, and checksums the complete file while reading those exact bytes into memory (an object is at most 4 MiB, the physical cap), so reading writes nothing. A larger legacy plain fragment is authenticated completely to build a budgeted in-memory index of 4 MiB block checksums. Subsequent requests verify the admitted file identity and each owned block against that index before Parquet sees its bytes. The index and two block buffers are reserved before allocation; no read-path scratch file is created. Replay and mutation readers, which bound their memory separately, retain the exclusively created, unnamed scratch snapshot. Identity, length, and checksum must match before decoding on every path. Consequently live fragment handles are bounded by max_open_runs rather than total history, and same-name replacement, transient in-place mutation (even if restored), symlink, scratch planting, and path substitution all fail closed. Parquet page headers are then parsed through a bounded compact-protocol reader before Arrow allocation. Declared compressed/uncompressed sizes must remain within the authenticated chunk/file ranges and the configured live-byte limit. One shared budget covers Arrow batches, decoded rows, spill buffers, and merge cursors; rolling fan-in levels keep run references logarithmic and unlink merged inputs immediately. Operational evidence separates raw graph-files authority authentication from retained property-fragment authentication. Each has distinct byte totals, 64 KiB block-equivalents (ceil(file_bytes / 64 KiB) per file), and actual non-empty read-call counters; block-equivalents are not read calls. Aggregate authentication bytes, equivalents, and calls equal their authority plus property components. physical_blocks is an actual-operation count: authentication read calls plus decoder read calls. Evidence also distinguishes validation and selected-value decoder bytes/calls, authenticated scratch bytes written, and the largest single-snapshot coexistence peak admitted against live scratch free space, plus range seeks and physical row-decode visits (a row decoded by validation and selected-value passes contributes once to each pass), shadowed rows, fragments and row groups considered/selected, and the shared live-byte peak. External-merge evidence reports first-level encoded spool input separately from total spill bytes, runs, and passes so amplification is a checked ratio rather than a worst-case row-size estimate. The invariant per_record_seeks = 0 remains exact.

Property-only commits reserve a checked monotonic property generation under the durable-rewrite lock. Legacy state initializes it from the maximum topology and search generation. Portable fingerprints consume the logical overlay once per route, so immutable and flattened projections have the same semantic identity.

Pure CAS reads open the existing graph-objects/sha256 namespace and existing lifecycle.lock with read-only capabilities. They create neither lifecycle state nor tmp/active. The authenticated regular lifecycle.lock is the cross-platform coordination authority: reads and publications hold it shared, while collection holds it exclusively. On Unix, the same operations also lock the retained graph-objects directory so replacing the still-open lifecycle pathname cannot split cooperative coordination. Windows instead relies on the retained lifecycle handle’s delete/rename denial because directory handles do not support byte-range locking. Post-lock identity and link validation closes namespace substitution races on both platforms. Publication, materialization, lease cleanup, and GC use the distinct mutable open-or-create capability; materialization remains there because installing hard links mutates the source inode’s link state.

The node-v3 canonical shape uses bounded buckets of one to eight exact-path entries. Maximal lowercase-hex SHA-256 prefixes are compressed into nodes; branches have at least two distinct nibble children. Empty inventory is the sole one-node empty-branch exception. Each entry authenticates its logical path, byte length, role and payload SHA-256. Bucket order, unique paths and the entire ancestral hash route are checked, including when a targeted lookup is absent. For F > 0 entries, the structural bound remains 2F - 1 nodes. Node v1/v2 and mixed/future formats are refused; this node-format change does not change the separate graph-files root version and adds no compatibility reader or migration machinery.

Every production manifest-object read admits at most 256 KiB before allocating the encoded buffer. Decoding admits at most eight entries, 4096 UTF-8 bytes per path, 64 bytes per digest, and sixteen branch children. The explicit field visitor avoids an unbounded flattened JSON intermediate and rejects the ninth entry. A 64-KiB decoded representation charge covers the node, bounded entry or child slots and their string contents. This is a logical charge, not a hard native-memory bound: parser scratch, canonical re-encoding, allocator overhead, read caches, and retained copy-on-write ancestors can overlap. Process RSS and filesystem peaks must be measured separately; route-table payload limits are independent of these manifest-node limits.

Insertion into a bucket is local; a ninth entry partitions at its first SHA-256 divergence into at most seventeen nodes. An unsplittable ninth full-digest collision is refused. Replacement copies only the selected path. Deletion removes empty nodes, collapses unary branches, and coalesces a small subtree using a probe limited to sixteen nodes and eight entries. A lower bound from pending nonempty children stops oversized probes without scanning the subtree. Published roots and payloads remain immutable, so retained generations and active snapshots keep their exact authenticated inventory. The 128/256/512 entry regression ladder bounds representative successful and absent lookups and replacements to four reads, and checks deletion work separately. These are application I/O counters, not OS I/O or native memory measurements.

GFDR run envelopes and records require version 2 with seed-zero XXH64 checksums; version-1 framing is refused. Control-chain identities and publication SHA-256 names remain required, while default replay performs corruption checks without cryptographic payload rehashing.

No commit publishes delta runs any longer (#1388): a delta-bearing open verifies, copies and re-streams the whole graph, which a bounded open cannot afford. Runs that generations already carry stay readable and compactable. Authoritative small-write delta runs, when present, live under graph/deltas/ inside the same generation and are inventory-verified (ADR 0019). GFDR’s binary framing and JSON payload schema are a permanent, versioned exception to the default Parquet graph-data rule; see ADR 0024. Compaction and ordinary opens decode the compact base from these canonical Parquet files; there is no duplicate JSON graph-state authority. A delta-bearing open verifies the contiguous typed GFDR chain, materializes a contained private Parquet view, and exposes that view only after replay succeeds within its declared limits. Checkpoint views use the same path, so a checkpoint remains pinned to its exact generation. Routing stems and canonical openCypher value types are retained; routing-free or string-only prototype GFDR payloads fail with GF_UNSUPPORTED_PROJECT_FORMAT rather than being guessed. Compaction folds a verified contiguous prefix back into canonical Parquet via a new immutable generation (compact_graph_delta) and reclaims unreachable inputs only through the shared retention/GC oracle. They are distinct from rebuildable indexes/adjacency/deltas/ accelerators.

GFDR admits only node/edge property set and removal. All node/edge topology upserts and deletes, including records with full-width identity metadata, fail with GF_UNSUPPORTED_PROJECT_FORMAT. Canonical publication owns topology. Admission checks precede preparation and transaction-retry shortcuts; decoding rejects persisted topology records, and direct in-memory replay prevalidates the complete input before changing caller state. Internal overlay writers refuse topology before creating target authority. No backward reader or migration is provided. See the current publishing contract.

Replay materializes only UUIDs touched by property operations; unchanged UUIDs remain in prior immutable fragments. During overlay construction the memory ceiling charges decoded runs, idempotency payloads, typed operation values, and the overlay simultaneously. Runs are released before materialization. Materialization then charges the retained overlay, node endpoint/identity authority, target references, baseline and output rows, Arrow arrays, and schema-width × row-group column metadata plus the active Parquet writer buffer. Replay writers use the shared permanent Zstd policy, retain their justified dictionary-off setting, and bound row groups by max_batch_rows. Flat generation-zero properties enter this same authenticated, sparse-fragment materialization path. max_records_per_run and max_work_rows independently bound mutation and physical work. Limit failures use the typed GF_RESOURCE_LIMIT code. Removing an absent key or setting an identical value is a no-op and creates no new property fragment.

For explicit bounded composite property set/remove requests, the Rust facade selects GFDR before mutating its private workspace. Storage prepares an owning child graph tree but cannot publish it independently; the facade combines its authenticated graph/files participant with every unchanged or updated parent participant and stages one complete generation. Creates, deletes, Cypher, bulk/algorithm writes, optimistic multi-writer requests, unsupported values, and journal capacity exhaustion select canonical full-Parquet publication before staging. Bindings and the CLI do not implement a second routing engine.

Optional capability absence is recorded in the generation manifest; it is not inferred by scanning folders. Graph-only readers validate the mandatory workspace control records and graph participants but never open provenance, knowledge, or epistemic tables. Semantic table ownership remains with the domain crates defined by ADR 0012.

Mutable topology rewrite recovery

Before a graph workspace becomes an immutable project participant, topology, property, search, and index maintenance may replace several fixed-path files. Those files advance through one authenticated durable rewrite, never through a sequence of independently committed renames. The engine retains the admitted project-root identity and each destination parent directory, holds the named rewrite lock exclusively, and binds every staged/final relative path, temporary-file identity, exact length, and SHA-256 digest in a checksummed intent. Paths must be canonical descendants; substitution, traversal, duplicate names, and cross-root state fail closed, while the named rewrite lock also requires one link.

A hydrated destination may be a read-only hardlink to a published CAS payload. After authenticating its prior identity and contents, the rewrite uses an explicit shared-destination replacement capability. The temporary source must remain a private single-link regular file; the replacement checks both expected identities and preserves the old inode, bytes, attributes, and ordinary query readers. Windows retained authentication guards still deny deletion and must be released before replacement; their sharing protections remain unchanged. On Windows this path alone uses FILE_RENAME_IGNORE_READONLY_ATTRIBUTE; it never clears attributes shared with CAS aliases. An absent prior destination uses no-replace installation. Ordinary filesystem replacement stays strict.

The intent is bounded to 16,384 entries and 8 MiB. Its sole generation-authority entry is topology/generation.json, whose JSON is bounded to 4 KiB and must encode the exact next topology/search pair. Data files are installed and authenticated first, directory namespace barriers are completed, retained root/lock identities are revalidated, and generation authority is installed last. Only then is the intent removed durably. This makes an existing matching destination an idempotent completed step while refusing a missing or changed temporary instead of accepting a partial batch.

An interrupted preparing intent cleans up only identity-matched retained temporaries. An interrupted durable intent always rolls forward from either the exact prior or exact next generation; any other generation state is corruption. The #931 UUID-to-surrogate index must participate through a typed auxiliary receipt that names and authenticates one exact staged receipt entry, so topology shards and index authority recover atomically. This internal topology/search generation is not project publication authority: a recovered workspace is still invisible to new project readers until the complete generation is selected by CURRENT.

Unless a root is shown explicitly, graph paths in the sections below are relative to the pinned generation’s participants/graph/; primary workbench paths are relative to participants/workbench/; derived index paths are relative to root cache/.

embeddings/ — primary vector generations

Unlike indexes/, caller-, algorithm-, or provider-produced vectors are primary workbench data and are never reconstructed or discarded as a cache. Names do not enter paths; compatibility and generation SHA-256 digests do:

embeddings/
├── aliases.json                         # display name -> compatibility digest
└── spaces/<compatibility-sha256>/
    ├── space.json                       # compatibility descriptor + refresh policy
    ├── active.json                      # checksummed active generation pointer
    └── generations/<generation-sha256>/
        ├── vectors.parquet              # node_uuid + FixedSizeList<Float32, N>
        └── manifest.json                # source fingerprint, counts, digests, state inputs

Builders use a collision-resistant private sibling directory, validate the complete UUID/vector batch, write and fsync vectors, write the checksummed manifest last, fsync the tree, then atomically replace active.json. The prior active generation remains visible until that final pointer swap. Incomplete private trees are ignored and recoverably removed on open. Alias replacement is separate from generation publication, so an incompatible producer cannot take over a name accidentally.

Generation manifests require wire version 2, exact Parquet byte length, a seed-zero XXH64 file checksum, and a checksum of canonical UUID/vector rows. Default open checks both physical bytes and canonical rows without recomputing their publication/content SHA-256 identities. Missing, malformed, or older manifest metadata fails closed; no legacy hashing fallback is provided.

Every open recomputes fresh, stale, substantially_stale, incompatible, or corrupt from the persisted descriptor/source fingerprint and current graph metadata. The exact identity fields, mutation thresholds, forced-stale boundary, retention, refresh coalescing, and provider privacy rules are normative in Embedding v1. Deleting a node or removing its selected label makes it ineligible immediately; corrupt or incompatible bytes always fail closed. Credentials, raw input text, provider payloads, and knowledge-layer fields are never stored here.

---

Derived Indexes

The indexes/ folder holds derived, rebuildable acceleration structures. Nothing here is canonical: every file under indexes/ can be reconstructed from topology/ (and, for FTS, properties/) alone. An absent index is not an error — it means the accelerator has not been built yet, and the engine falls back to building in memory on demand. See ADR 0004.

indexes/adjacency/ — graph-native adjacency index

The adjacency index is a derived CSR (compressed sparse row) representation of the topology, used by both the Cypher traversal path (variable-length Expand) and the analyst verbs (rank/cluster/paths/analyze/similar). It is optional: absent ⇒ build in memory on demand (today’s behavior); present ⇒ load from disk. It is surrogate-keyed and never changes results — only speed.

indexes/
└── adjacency/
    ├── index_manifest.parquet
    ├── WORKS_AT.out.csr.json       # versioned shard-set manifest
    ├── WORKS_AT.out.csr.shards-<digest>.d/
    │   ├── 00000000000000000000.csr
    │   └── ...
    ├── WORKS_AT.in.csr.json
    ├── OWNS.{out,in}.csr.json
    └── _all.{out,in}.csr.json      # union across relation types

The builder (graphforge_storage::adjacency::build_adjacency_index) writes one {out, in} pair per relation type plus the _all union pair, then the manifest last. Relation names unusable as file stems (path separators, .., the reserved _all) are skipped — those relations are served by scan-build, but their rows still flow into the union index. The manifest is stamped with the topology_generation counter read before the edge scan. A concurrent topology mutation can therefore make the result stale, never falsely fresh.

index_manifest.parquet

Column	Arrow type	Notes
relation_type	Utf8	Relation type name, or _all for the union index
direction	Utf8	"out" | "in"
topology_generation	UInt64	Counter pinned before the source scan
built_at	Timestamp(Microseconds, UTC)
node_count	UInt64	Number of source nodes covered (CSR row count)
edge_count	UInt64	Number of (edge, neighbor) entries

Sharded CSR (<REL_TYPE>.<dir>.csr.json) — a versioned JSON manifest names an immutable, content-addressed shard directory. Each bounded shard is Arrow IPC with one column and covers a contiguous local surrogate range:

Column	Arrow type	Notes
adjacency	LargeList<Struct { edge_id: UInt64, neighbor_id: UInt64 }>	Row i holds the adjacency entries of surrogate node_id = i, in CSR order

Within each shard this is the CSR structure in its idiomatic Arrow encoding — the two logical arrays cannot be two top-level columns because a RecordBatch requires equal column lengths. The list’s offsets buffer is the CSR offsets array (length node_count + 1, Int64, starting at 0, monotone), and the flattened struct child is the targets array (length edge_count): neighbors of node_id = i are targets[offsets[i]..offsets[i+1]]. The reader authenticates and decompresses one bounded Arrow batch, then copies its values into the shared shard cache.

Conventions:

• Empty graph: a zero-row batch — logical offsets == [0], empty targets. The offsets array is never empty.
• Node with no neighbors: an empty list (offsets[i] == offsets[i+1]).
• The shard manifest records format/version, total node/edge counts, ordered boundaries, per-shard counts, encoded/decoded byte lengths, SHA-256 publication identities, and required seed-zero XXH64 corruption checksums. A row may span consecutive shards when a high-degree vertex exceeds the configured hard edge cap; readers concatenate those fragments in deterministic (key, edge_id) order.
• Logical CSR rows cover exactly node_id ∈ 0..node_count; surrogates beyond node_count have no entries. Empty interior rows need no physical shard bytes.
• In-memory consumers (graphforge_exec::AdjacencyProvider) keep a ShardedCsrIndex on a persisted hit and materialize only the requested logical row from its bounded shard fragments. Only current version 3 manifests and Zstd IPC shards are admitted. Earlier manifests and standalone legacy files have no compatibility reader or migration path. Scan-build fallback still materializes a hash map for oracle parity.

indexes/property/ — property-equality index

A node anchor (:L {id: $x}) (or WHERE n.id = $x on one label) is answered by a persisted equality index instead of a scan of the label’s property column (#1983). Every property named id is indexed (the import session declares no key columns). One pair exists per node property route and indexed column whose stored type is integer or string:

indexes/property/
├── <stem>.eq                 # manifest: XXH64 line, then JSON
└── <stem>.shards/00000000.eqs  # Arrow IPC (value, node_uuid), sorted, 4096 rows each

<stem> is a hash of the route and column. The manifest carries the route, column, value type, a digest of the route’s property objects (path, length, XXH64), and each shard’s first and last value, length and XXH64. A lookup binary-searches the manifest and reads only the shards whose range holds the value, so its reads do not grow with the route. The planner pushes column = literal into the property scan as before; the scan names the index in EXPLAIN (index=property_equality(id)) when a current one answers it. Bytes are a pure function of the route’s content.

• Built at publish. An initial bulk construction (also an import session onto an empty project, in memory or durable) writes the index inside its own generation. GraphForge::rebuild_property_indexes rebuilds it from the current generation and publishes a new one.
• Current or not used. The index answers only while its source digest equals the route’s. A project published before the index, or a route changed by a mutation since, is answered by the scan until the next rebuild. A read never builds or repairs an index.
• Authenticated. A manifest or shard that fails its checksum is refused with GF_VALIDATION; the scan is not substituted for it.
• Semantics. The index is a candidate source: the newest live snapshot of each candidate is still checked against the predicate, exactly as the footer-statistics path does. Float, boolean and null comparisons are not pushed down and use the scan.
• The node topology is anchored too. The labelled node scan that feeds the join to the property route would otherwise decode every node fragment (array_has(type_ids, L) over the whole topology) to meet the one index result. StoredEqualityHints therefore also offers the equality to that node scan as an anchor, through column projections and total predicates only. Where a current index answers it, GraphForgeParquetExec (anchor=property_equality(id) in EXPLAIN) resolves the index’s UUIDs to node_ids through the topology identity columns, pruned by row-group and page statistics, and decodes exactly those rows by ordinal from the fragments that hold them. The label filter and the equality stay in the plan and decide which rows survive, so the answer is the unanchored scan’s answer; a node the index names under another label, a deleted node, and a node with several labels behave as they do without the anchor. A stale or absent index leaves the scan whole. Its reads grow with the matches, not the graph.

indexes/edge-properties/ — edge property index

A traversal that returns or filters an edge property reads the properties of the edges it crossed from a persisted index instead of opening and checksumming every fragment of the edge property route (#1984, ADR 0059). One manifest and one directory of packs exist per edge property route:

indexes/edge-properties/
├── <stem>.epx                      # manifest: XXH64 line, then JSON
└── <stem>.packs/
    └── 00000000.epk                # up to 4 MiB of consecutive, individually sealed blobs

A shard holds at most 4,096 rows and at most 1 MiB per column. Each shard stores its sorted edge_uuid column and each property column as separate blobs (a Zstd Arrow IPC stream of one batch), so a statement reads only the properties it demands. The values are the route’s own Arrow values, exactly as the fragments store them. The manifest records the route, a digest of the route’s property objects (path, length, XXH64), the route’s property columns, and for every shard its first and last edge UUID, row count, and the pack, offset, length and XXH64 of each blob. A lookup reads the UUID blob of the shards whose range holds a requested edge and the demanded column blobs of the shards that hold a match, each a byte range of a pack checked against its seal in the manifest, so its reads are bounded by the frontier and not by the route. Blobs are packed because a durable open hard-links every artifact the generation inventories; a file per blob made the open of an indexed project several seconds slower. ExpandExec and VarLenExpandExec both resolve their edge properties through it.

• Built at publish. An initial bulk construction cuts each fragment into shards as it encodes it and writes every pack once, as it fills, inside its own generation, with the manifest last (never reading the published fragments back). GraphForge::rebuild_property_indexes rebuilds it, with the node equality index, from the route’s newest-wins state in UUID order and publishes a new generation.
• Current or not used. The index answers only while its source digest equals the route’s. A project published before the index, one built by a staged append, and a route changed by a mutation since are answered by the property overlay until the next rebuild. A read never builds or repairs an index.
• Authenticated. The manifest is checked on first touch; packs are self-authenticating, and every blob a lookup reads is checked against its seal in the manifest. A failure is GF_VALIDATION; the overlay is not substituted for a corrupted index, and a damaged column blob refuses only the statements that demand it.
• Not indexed. A route with a tombstone, unsorted fragment rows, a column stored under two types, or a single value over 1 MiB. Bytes are a pure function of the route’s content: no clock, UUID or process identity, and no Arrow field metadata.

Bounded CSR compression

Every production shard publisher uses Arrow IPC Zstd (the pinned Arrow default compression level 3) with the existing full-width UInt64 IDs and Int64 offsets. The standard IPC raw-buffer marker is permitted when compression would expand a buffer; it is part of the current codec, not a legacy file fallback. Configured shard limits remain upper bounds and are capped at 1,048,576 rows and 1,048,576 edges. High-degree rows continue across shards; smaller configured limits retain their existing behavior.

For admitted shard counts N and E, the seven decoded buffers total D = 8*(N+1) + 16*E + ceil(N/8) + 3*ceil(E/8) bytes, including validity bitmaps. The encoded file is capped at D + 16,384 bytes. The manifest carries both exact encoded length and D; lookup and stable-shard reuse validate these counts, bound reads, check seed-zero XXH64, then check the IPC footer/message and every buffer before Arrow allocates decoded arrays. Preflight requires the exact fixed schema, one batch, no dictionaries, zero null counts, matching field lengths, Zstd buffer compression, and one exact current Zstd frame with matching content size. Invalid schema declarations return an error before Arrow’s schema converter is invoked. Decoded offsets must start at zero, remain monotone, and end at the edge count.

Opening checks shard metadata without reading payloads. Cloned readers share one cache. Standalone opens use STANDALONE_DECODED_SHARD_CACHE_BYTES (64 MiB). Execution providers share one cache across their indexes within the B / 4 topology allowance derived from query budget B; the node-label index and edge-property ID maps draw on that same allowance. Retained decoded shards are bounded by bytes, not by count: a least-recently-used policy evicts within the available cache allowance, so sequential traversal still pays one decode per shard while random-access frontiers (for example a two-hop ExpandExec frontier in neighbour order, #1518) stop paying one decode per row. A shard larger than the available allowance caches alone. Retained decoded bytes therefore never exceed max(available allowance, one shard) and never scale with the graph (#1094). Cursor walks retain no entries in this cache; each cursor’s active decoded shard is a separate transient buffer. On a miss, the prior cached shards stay alive until the replacement fully validates; a failed read evicts nothing. Resource accounting must include the retained cache, the encoded file, Arrow’s body buffer and decoded arrays, the new CSR vectors, parser/alignment allocations and the fixed Zstd decoder context. The pinned bulk Zstd decoder uses the supplied bounded destination rather than a separate streaming window buffer. D is a deterministic buffer-content bound, not a process RSS bound. Allocator overhead and other graph/query owners remain separate. A whole logical hub row can span many shards; only row_chunk bounds the returned row portion by its limit.

Rebuild and versioning semantics

• Source of truth. A CSR is always reconstructable from topology/edges/<REL_TYPE>.parquet alone, deterministically.
• Generation identity. The adjacency manifest records the topology counter pinned before the source scan. A complete delta chain may advance an older base to the current topology counter without copying the base CSR.
• Publication rule. A graph mutation and its source fingerprint publish in the same immutable generation. CURRENT changes only after every participant is durable and validated.
• Crash-safety invariant. A reader sees either the prior complete graph generation or the new complete graph generation. A failed or interrupted write never exposes a counter/data mismatch or committed prefix.
• Staleness detection. The provider compares the manifest’s topology counter with the current counter and validates any required bounded delta chain. A corrupt accelerator is never served as a hit.
• Fallback. On mismatch (or absent index), the provider scans the typed edge tables and builds the adjacency in memory — yielding identical results, only slower. A stale or missing index can therefore never cause incorrect output.
• Corruption is refused, not rebuilt (#1388). An index object that fails its checksum or length (a CSR shard against its shard manifest; index_manifest.parquet and each *.csr.json against the project manifest on first touch), or whose checksummed bytes do not decode, is refused by the query that touches it with GF_VALIDATION. Nothing is rebuilt or written: a rebuild would answer by scanning the edge table, so a bounded query would silently cost O(E) and the damage would never be reported. An index in a format this release does not read, or an I/O failure, is a storage error. The explicit index_adjacency rebuilds from the authenticated edges and replaces a corrupted index.
• Explain and query evidence. Traversal plans retain adjacency=hit, miss, or building. A stale index adds adjacency_rebuild=stale, including the ordered one-hop and two-hop plans. Explain observes the pending rebuild without performing it. Public query execution evidence records adjacency_rebuilds: a construction-published generation reads its current index with zero rebuilds; the first traversal after a stale-making mutation records the rebuild it performs. Corruption refuses rather than rebuilding.
• Rebuild triggers. Lazy on first traversal when the indexes/adjacency/ capability is present, or explicit via forge.index("adjacency", ...). Append-only commits publish bounded delta segments; a full rebuild compacts them into sharded bases.
• Determinism (R-ADJ-2). Full rebuild streams each typed edge file once; out entries sort by (src_id, edge_id) and in entries by (dst_id, edge_id) — the edge_id tie-break makes shard bytes reproducible from topology/ alone. The _all relation’s indexes use the same sorts over the union of typed and exploratory edges. Cache filenames use portable route components; the manifest retains exact semantic relation names. The manifest’s built_at is excluded from the determinism guarantee.
• Bounded build. Projected Parquet batches feed sorted spill runs. Bounded-fan-in merge passes (64 runs by default) emit rows directly into hard-capped shard sinks; they never reconstruct complete edge/neighbor arrays. Both edge entries and local offset rows have hard shard caps. AdjacencyBuildMetrics exposes source rows, spill runs/bytes, shard count, and peak shard entries/rows for scale evidence.
• Build ordering. Builders write immutable shard directories, atomically publish each shard-set manifest, and write index_manifest.parquet last. The public facade builds in a same-filesystem private directory, validates it, then swaps the complete adjacency directory under its visibility lock. Cancellation or failure leaves the prior directory active and removes unpublished spill/build state.
• Loader semantics (graphforge_exec::PersistentAdjacencyProvider). Freshness requires a non-empty manifest whose topology generation is current directly or through a complete bounded delta chain. Fresh + row present ⇒ load (adjacency=hit); stale or torn ⇒ lazy rebuild, then serve; fresh but no row for the requested relation ⇒ scan-build without rebuild (rebuilding cannot add an unknown relation — prevents a rebuild-per-query loop); a missing shard manifest for a manifest row, or one that disagrees with the index manifest’s counts ⇒ lazy rebuild; a corrupt accelerator ⇒ GF_VALIDATION, never a rebuild; an unreadable generation counter ⇒ scan-build; capability absent ⇒ scan-build (adjacency=building). Typed-mode "*" bypasses the index entirely (reported as building, never a false miss). A failure to write a missing or stale index never fails the query — only its speed.
• Direction. out and in CSRs are stored separately; undirected traversal unions them. In exploratory mode, _exploratory.parquet rows are routed by their rel_type_name column.

indexes/<LABEL>/tantivy/ — full-text search index

Full-text indexes (Tantivy) are also derived and rebuildable from properties/. See the Find / index (forge.find / forge.index).

---

Graph Fact Schema

Topology layer (hot path)

Graph traversal reads only the topology layer. No property columns are read unless the query explicitly projects them.

topology/nodes.parquet

Column	Arrow type	Notes
node_uuid	FixedSizeBinary(16)	UUIDv7 — canonical stable identity
node_id	UInt64	Local surrogate — DataFusion join key
type_id	UInt32	Immutable primary label used for property-file routing
type_ids	List<UInt32>	Authoritative complete label set; scans use membership in this column
created_at	Timestamp(Microseconds, UTC)
updated_at	Timestamp(Microseconds, UTC)

The first label in a node’s creation pattern is its immutable primary label. The property route continues to use that primary label, encoded through the semantic-route mapping for its physical path; adding secondary labels therefore cannot orphan or relocate properties. Unlabelled nodes route to _untyped. A v0.5 node participant must contain both fields with the frozen schema; an earlier development schema is unsupported.

topology/label_membership_counts.json stores the positive per-label node membership counts used by write-context initialization and last-use DELETE accounting. The versioned control file is included in the generation’s authenticated graph-files inventory and is replaced in the same atomic rewrite as label and node changes. A generation without the summary establishes it once from node topology during its next successful write; the write result reports that establishment’s decoded rows and opened fragments separately from steady state initialization work.

Filtered node lookup

Canonical node files assign node_id densely and monotonically, so physical row ordinal n - 1 contains node_id = n. A filtered node read proves that layout from non-null row-group statistics plus ascending column and offset page indexes, then supplies Parquet with an exact RowSelection for the requested ordinals. Scattered destination ids therefore decode only their selected rows, instead of every page between their minimum and maximum.

The optimization is fail-closed. Missing page indexes, deleted/gapped IDs, malformed statistics, or unordered ranges use the row-group plus membership-predicate reader. Exact output keys are validated after ordinal selection; any mismatch is discarded and retried conservatively. This accelerator fallback applies only to a valid v0.5 participant; it is not a project-format compatibility path.

Filtered edge lookup

The bulk builder ranks edges by UUID and gives each canonical edge file a contiguous window of edge_ids, so a file that holds one route’s edges has physical row ordinal n - first_id for edge_id = n. The traversal’s edge read (ExpandExec, VarLenExpandExec) proves that layout per file from the same footer statistics and page indexes as the node read, and selects the requested rows by ordinal. The decision is made file by file: a file that holds none of the requested ids selects no row group and costs its footer, and a request is never compared with the size of the whole route. A file that is not dense (it shares its id window with another route, has no page index, or is a legacy single-file relation with gaps) keeps the row-group plus membership-predicate reader, as does a dense file when more than half of its rows are requested. The selected rows are validated against the ids requested of that file; any mismatch is discarded and the file is read conservatively. A read of a fixed frontier therefore decodes the pages that hold the frontier, so its bytes do not grow with the relation.

topology/edges/TYPENAME.parquet (one file per relation type)

Column	Arrow type	Notes
edge_uuid	FixedSizeBinary(16)	UUIDv7
src_uuid	FixedSizeBinary(16)	References node_uuid
dst_uuid	FixedSizeBinary(16)	References node_uuid
edge_id	UInt64	Local surrogate
src_id	UInt64	Local surrogate — DataFusion join key
dst_id	UInt64	Local surrogate — DataFusion join key
created_at	Timestamp(Microseconds, UTC)

Typed edge tables replace the unified edge_facts table. A relation is a logical union of its legacy flat fragment and ordered immutable range fragments. Each construction flush encodes only accepted rows; it never decodes and rewrites prior edge topology. Ordinary catalog, traversal, adjacency, UUID-index, mutation, projection, and delta-replay paths enumerate the same fragments. This keeps aggregate fresh-import edge encoding O(N) and resident topology bounded by the configured construction window while preserving direct single-relation scans.

Node topology follows the same immutable layout: the first compatible write may retain topology/nodes.parquet, while later appends create ordered topology/nodes/<first>-<last>.parquet fragments. Counts, filtered reads, surrogate recovery, UUID identity probes, semantic validation, projection, export, label mutation, and deletion operate over the logical union. A localized rewrite replaces only the fragment containing a changed row; untouched node fragments retain their filesystem identity.

topology/surrogate_tails.parquet is a one-row control record containing the monotonic maximum node and edge surrogates. It is staged in the same commit as every topology append. Writer reopen reads this bounded record rather than enumerating or decoding the accumulated topology fragments; legacy projects without it use the bounded tail migration path once.

Append validation and bulk endpoint resolution read the published topology Parquet through TopologyIdentityProbe; no derived UUID index is written or read (#1902). A probe skips every fragment row group whose node_uuid/edge_uuid min/max statistics exclude all requested UUIDs, then decodes only the UUID column (plus node_id for node lookups) of the rest. Live counts come from footer row counts. A UUID is refused when it names a live node, a live edge, or a deleted entity: topology/deleted_identities.parquet keeps the sorted UUIDs of every deleted node and edge, is rewritten only by a generation that deletes, and is an ordinary authenticated graph file. Work evidence reports row groups and compressed bytes read, fragments considered, and zero per-record seeks. Projects built before #1902 still carry topology/uuid-membership/manifest.json, identities-v5-* and node-surrogates-v5-*; those files open, export and verify as ordinary entries and are never read. See UUID identity authority.

Node ordinal resolution is a distinct, additive authority facet in the same directory. Its ordinal-v4-manifest.json, ordinal-v4-receipt.json, and ordinal-v4.lock live beside the ignored legacy membership files and never interpret them. The v4 logical ordinal facet requires wire version 5 with file/block XXH64 checksums; earlier wire-version-4 descriptors are refused. If the ordinal facet is absent, discovery returns a typed rebuild requirement. A present ordinal path must pass authenticated open and never falls back to v3 when malformed or substituted.

New mapped publications use a compact version-8 graph/files root with explicit mapped-route authority and required payload corruption checksums. Payloads, node-v3 branches and node-v4 checksum buckets live in the content-addressed object store; a generation stores only its root reference and logical totals. Updates copy a bounded SHA-256 nibble path and split or collapse bounded buckets, so publication never scans or recopies the entire prior file inventory. The private workspace commit boundary records revision-identified sealed and tombstone descriptors before mutations become visible; they are acknowledged only after CURRENT advances. Reopen authenticates radix control objects and checksums every selected payload. Current updates retain bounded path-copy publication; no legacy checksum upgrade exists. Post-CURRENT GC traces every remaining generation root and defers while an optimistic attempt or CAS publication lease is live. Retired expanded and compact inventories are refused. A writable facade converts current raw routes in its private workspace before publishing the mapped layout under the CAS publication lease.

The authoritative write census is executable: topology node and edge shards, node and edge properties, graph deltas, catalog records, extension-owned graph records, and the generation/runtime-catalog/runtime-label control files must all appear in the revision descriptor journal and resolve to the same authenticated logical inventory. Rebuildable adjacency artifacts live under .graphforge-cache/ and are rejected as graph authority. Parquet write sites share RewriteBatch plus commit_topology_aware; the three control-file writers record their descriptor before making replacement bytes visible.

Terminal buckets retain sorted exact paths even when distinct path bytes share a SHA-256 digest, up to the eight-entry bound. A real SHA-256 collision is not a test fixture. Tests cover exact-path replacement/deletion, bucket split/collapse, maximum escaped path fields, oversized encodings, duplicate paths, malformed routes, unsupported versions and authenticated old-root reads.

Properties layer (warm path)

properties/<component>.parquet (flat first fragment) and properties/<component>/<generation>-<ordinal>.parquet (immutable construction fragments). The mapped component resolves to the semantic entity type; legacy raw layouts use the entity type directly at the same route position.

Column	Arrow type	Notes
node_uuid	FixedSizeBinary(16)	Join key back to topology/nodes.parquet
(property columns)	(per ontology)	e.g. name Utf8, age Int64, email Utf8

Property access joins the logical node-topology shard union to the logical property overlay on node_uuid. Catalog, direct, SQL, export, verification, and import readers authenticate the legacy flat fragment and every canonical immutable fragment, then expose one newest-authority complete snapshot per UUID; they do not concatenate physical rows. A newest tombstone suppresses the UUID, and the newest live-schema summary determines which property keys remain logically present. Edge properties use the identical authority model under edge_properties/. Construction and ordinary SET/REMOVE mutations append only the changed bounded window without rewriting prior fragments. The separation still lets topology-only traversal skip property I/O entirely.

Provenance and knowledge participants

provenance/ and knowledge/ belong to the knowledge layer, but generic storage does not own their records or schemas. Under ADR 0012:

• graphforge-provenance owns provenance events and lineage;
• graphforge-knowledge owns knowledge assertions, assertion graph references, confidence assessments, evidence links, algorithm runs/events, and every additive epistemic epistemic record;
• graphforge-api validates cross-domain UUID references and assembles composite writes; and
• graphforge-storage receives validated Arrow batches as opaque generation participants and owns only their paths, checksums, persistence, publication, and recovery.

The exact knowledge schemas are frozen and are generated from the two owning Rust registries in the checked Knowledge schema inventory. The epistemic layer adds separate append-only status, amendment, reasoning, supersession, hypothesis, selection, and valid-time record families; it does not add mutable fields to knowledge assertions.

The legacy pre-knowledge PROVENANCE_EVENTS_SCHEMA, PROVENANCE_LINEAGE_SCHEMA, and graph-embedded edge confidence/provenance_uuid fields have been removed from graphforge-storage. They are not the knowledge-layer contract, and no historical project data is imported or converted.

Graph-only readers resolve the committed generation and graph-required manifest fields without opening either participant. A future or corrupt knowledge record blocks its owning knowledge API, never Cypher or neutral analyst-verb/find execution.

---

Ontology Runtime

The ontology is a runtime-loadable knowledge schema, not Rust structs generated into the binary. Three representations serve different purposes:

Format	Purpose
YAML / JSON	Human-authored ontology definitions (Serde-based load)
Arrow tables	Compiled execution format — cheap joins during binding and planning
Parquet	Derived compiled-runtime snapshot for rapid startup or reproducible deployments; discard and recompile on ontology-checksum mismatch

Ontology authoring format (YAML)

ontology_id: core
version: "2026.05"
entity_types:
  - name: Person
    abstract: false
  - name: Employee
    parent: Person
relation_types:
  - name: MANAGES
    src: Employee
    dst: Employee
    inverse: MANAGED_BY
    semantic:
      transitive: false
      symmetric: false
      functional: false
properties:
  - owner: Person
    name: name
    type: utf8
    nullable: false
constraints:
  - owner: Employee
    kind: unique_property
    expr:
      property: employee_id

At load time this compiles into Arrow lookup tables keyed by integer type IDs. String-heavy lookups during planning become O(1) integer comparisons.

Ontology runtime tables

Table	Purpose
ontology_meta	Identity, version, IR compatibility range, checksum
entity_types	Node classes and inheritance DAG (acyclicity enforced at load)
relation_types	Edge classes, endpoint type constraints, inverse pairs
property_types	Name, owner, value type, nullability, cardinality
type_constraints	Validation rules (unique, required, range)
cardinality_rules	Endpoint multiplicity (min/max per relation type)
semantic_flags	transitive, symmetric, reflexive, functional, acyclic
aliases	Human-facing and deprecated names

Versioned migration transforms remain part of the authoritative OntologyDoc; they are not a ninth compiled runtime table or Parquet snapshot file.

Ontology versioning

Two independent version axes:

Axis	Meaning
ontology_version	Meaning of types and rules — changes when the schema evolves
ir_version	Runtime/compiler contract — changes when the IR format changes

A new ontology version does not require a new IR version, and vice versa. Persisted datasets record the ontology_version used to write them. Arrow schema metadata carries both versions through IPC and Parquet round-trips.

Validation model

Level	When	Examples
Ontology-load	On file/table load	Duplicate names, missing parents, inheritance cycles, bad inverse references
Write-time	On CREATE, MERGE, batch ingest	Unknown property, wrong value type, illegal endpoint type, cardinality overflow
Query-time	During binding/planning	Unknown labels/types/properties, illegal pattern shape, ambiguous property resolution

---

Serialization Systems

Default rule (with two named exceptions):

System	Purpose	Format
Arrow / Parquet (graphforge-storage)	Graph topology/properties and generic persistence of domain-owned participants	Binary columnar (Arrow IPC / Parquet)
JSON / YAML (graphforge-ontology)	Ontology definitions and metadata	Text (human-readable, validatable)

Graph data → Arrow/Parquet. Ontology definitions and metadata → JSON or YAML. The permanent exceptions are (1) authoritative, versioned GFDR binary delta runs with schema-qualified JSON mutation payloads, and (2) derived compiled ontology runtime tables persisted as Parquet. The latter never supersede the CURRENT-selected workspace’s canonical ontology JSON and are discarded/recompiled when its checksum differs. External YAML/JSON files are authoring/import inputs, not alternate project commit pointers. ADR 0024 defines both compatibility and migration boundaries. Arrow schema metadata carries version and provenance annotations across language boundaries.

---

Two-Mode Graph Instances

// In-memory (fast, volatile)
let forge = GraphForge::new(None)?;

// Persistent (project directory)
let forge = GraphForge::new(Some("path/to/project/"))?;

The storage layer is transparent to all API surfaces.

---

References

• Architecture Overview — workspace layout and provider trait
• Execution Model — how providers connect to DataFusion
• ADR 0001: Rust Core — Parquet-as-primary and provider strategy

Current publishing contract

Rust owns publication behavior. The public facade selects the operation path; storage owns authenticated representations and atomic generation selection. A publishing path is complete only when the selected generation and the facade’s readers agree. Table-valued data remains Arrow at the API and Parquet at rest; versioned GFDR is the explicit property-journal exception.

Every applicable graph publisher preserves these authorities together:

• Exact node/edge UUIDs, full-width surrogates, endpoint identity, global edge identity uniqueness, and persistent consumed-ID high-water marks. Deletion cannot make an ID available to a subsequent ordinary CREATE.
• Complete typed or exploratory schemas, null/concrete property types, latest values and tombstones; immutable primary routes and label memberships remain distinct. Physical path components never substitute for semantic route names.
• Authenticated graph-file ownership, route authorities, deleted identities, ordinal receipts and applicable adjacency/search generations. Staging reads the admitted inventory rather than discovering authority from filenames.
• All graph, catalog, ontology/composition and other declared publication participants. A graph-only update retains unrelated participants; changing ontology cannot strand retained semantic bindings or reinterpret numeric IDs.
• Parent conflict, operation identity, cancellation, active reader leases and crash/returned-error recovery. CURRENT selects the complete old or new generation. A returned error after selection must reconcile actual selected authority; retries cannot install an unrelated private candidate.
• permanent_parquet::writer_properties for every verified permanent Parquet publisher. Per-path dictionary, row-group, streaming and memory settings remain local. Replay and compaction retain the accepted codec; private IPC/spill, portable containers and query sinks have separate contracts.

Producer → publisher → consumer	Operation boundary and applicable proof
Public construction → canonical graph generation → facade/reopen	Typed/exploratory topology and properties, sharded nodes, routes, all graph identity authorities and continuation tails. Construction, CAS ownership and publishing-budget facade regressions cover exact reopening and portable interchange.
Ordinary Cypher/analyst mutation → MutationTransaction/GraphWriter → generation readers	Canonical topology and property mutation; complete participant publication. Public CREATE/DELETE/SET, active streams, fault recovery and next-ID tests apply.
Composite property mutation → compact root publication (GFDR preparation retained for existing generations)	Composite property mutation publishes a compact root and no delta run (#1388). The GFDR storage APIs and their verified replay/compaction remain for generations that already carry runs: only the four property operations are admitted, and ownership fixtures cover sparse latest values, removals, nulls, route identity and shared immutable base payloads.
Composite topology mutation → canonical GraphWriter publication → facade	Never GFDR. Qualified create and owner-routing regressions cover identities and subsequent property mutation.
Storage GFDR APIs → framed runs → direct replay, open, checkpoint, compaction/import	All topology operations are unsupported, including checksum-valid records, duplicate operation IDs and matching transaction retries. Refusal precedes authority changes; direct replay leaves the entire supplied state unchanged.
Public compaction → complete new generation → refreshed facade	Full verified property chain, same-facade subsequent mutation, exact retry, retained streams and imported continuation. Private hydration and selected permanent ownership are measured separately.
Ontology adoption/clear and retained semantic transformations → workspace generation → prepared readers	Same-name promotion stages graph and ontology together; disjoint metadata-only adoption reuses payloads. Unsupported clear/type-ID reinterpretation refuses before selection. Existing semantic transformation and promotion lifecycle tests own applicable retained-data proofs.
Graph/belief/portable projection → selected artifact → verifier/import	Exact selection and endpoint closure, schemas, routes and rebuilt applicable controls. An exported artifact is not a selected live generation; clean import must validate it before subsequent mutation.
Portable clean import/checkpoint restoration → complete generation → facade	Complete participant authentication and corruption refusal, exact identities/properties and continuation. Persisted unsupported topology GFDR cannot be imported as supported current state.
Catalog/vector/knowledge/epistemic/provenance/restore markers → declared participant → domain reader	Domain schemas, references, authentication and atomic participant ownership apply. These writers do not allocate graph topology IDs; graph ordering/tails are inapplicable to the participant payload itself.

Private accepted construction/merge streams, decoder spools, standalone ontology persistence without a graph publisher, external query sinks, and test/benchmark fixtures are not permanent graph publishers. They cannot establish production support for a topology journal operation.

The four permanent_storage_budgets tests publishing_contract_alternates_supported_paths_and_refuses_topology_journals, publishing_contract_flat_ontology, publishing_contract_sharded_exploratory and publishing_contract_sharded_ontology use exercise_publishing_contract to check flat/sharded, exploratory/ontology-promoted, two-route graphs. The flat cases use public composite CREATE on an empty project; the sharded cases use public construction sessions. All four exercise actual property GFDR, compaction, canonical topology mutation, reopening, export/full verification, clean import and subsequent mutation. It compares UUIDs, routes, endpoints, nullable typed values and node/edge allocation continuity. The neighboring compaction/promotion/owner fixtures add active snapshots, interrupted publication, cancellation, retries and full-width identity coverage. Storage journal tests additionally cover forged persisted records, direct-run bypasses, byte-preserving refusal, corruption and exact resource thresholds.

The conformance work reuses the source-bound measurements and deterministic budgets from the permanent-storage assessment, including #1213 encoding, #1219 CAS replay, #1224 property ownership, #1231 facade refresh and #1229 promotion. Rejection creates no graph payload or publication; its fixture compares every retained file digest and allocation before/after. The added preflight is a constant-work discriminant check per operation and retains no second graph or decoded value collection. Existing replay work, metadata, batch and memory limits remain enforced. Logical counters do not claim process/native RSS bounds; sampled overlapping allocation is not a hard peak bound. Source-bound integrated S20/S22 accounting remains the epic’s later measurement, not an outcome inferred from this contract test.

Publishing-contract acceptance evidence

The frozen four-case measurement records 55.58 s user CPU, 5.36 s system CPU and 162,652 KiB observed process peak RSS. Separate syscall tracing recorded 882,417,829 bytes read and 157,666,163 bytes written; separate pathname sampling observed 21,864,448 allocated bytes at its largest sample, deduplicating shared inodes across retained, private and portable files. These are complete lifecycle observations, not isolated rejection costs or hard temporary-disk limits. The evidence records OS block counters, sample gaps, overlapping validation activity and failed superseded fixture attempts. No incomparable baseline improvement is claimed.

Contract outcome	Direct evidence
Supported producers, complete authorities and encoding selection	The producer matrix above; shared-policy and per-path budgets in the permanent-storage assessment. Participant-only paths explicitly exclude topology allocation.
Unsupported topology refuses before authority changes	topology_payloads_reject_before_encoding_preparation_or_publication, topology_cannot_bypass_published_retry_or_mutate_direct_replay_state, committed_checksum_valid_invalid_memberships_reject_without_authority_mutation, and topology_overlay_refusal_preserves_routes_and_full_width_ids. These cover all topology variants, retry/direct-replay bypasses, authenticated persisted records and unchanged target bytes.
Flat/sharded and exploratory/ontology public lifecycle	The four publishing_contract_* cases above assert the selected topology layout after optional adoption. They preserve exact node/edge UUIDs, endpoints, nullable values, route counts and consumed node/edge IDs across deletion, reopen, CREATE, export/full verification, clean import and subsequent mutation.
Authentication and recovery remain enforced	Existing journal checksum/order/missing-run tests; compaction cancellation, checkpoint retention, cleanup and exact retry; portable import crash windows and pristine-target corruption refusal. Merged #1219, #1224, #1229 and #1231 supply public active-snapshot and returned-error lifecycle coverage.
Resource bounds remain meaningful	streaming_resource_ladder_is_independent_of_base_rows uses 260/516/1,028 nodes, seven-row batches, 2 MiB replay admission and at most 256 KiB logical/allocated decoder spool. It compares exact output with the non-spooling path and limits logical replay-state growth to 128 bytes. Public conformance enforces 2 MiB compaction output and 8 KiB logical replay-state ceilings. These counters exclude process/native memory.
Portable current-format identity correctness	valid_identity_package_keeps_absent_primary_and_runtime_catalog_bytes, invalid_delta_identity_package_preserves_pristine_target_authority, and absent_primary_round_trip_and_topology_replay_refusal_preserve_state distinguish valid full-width identities from unsupported topology replay.

This ledger maps ordinary implementation criteria to their existing tests. It adds no release-gate requirement and does not claim final capacity completion. The integrated #1194 report reconciles the merged repairs, admitted S20/S22 measurements, physical owners, resource tradeoffs and the separate final-capacity outcome.

Bounded manifest allocation evidence (#1204)

The fixed heterogeneous #1196 workload (4,097 nodes, 65,537 edges, four routes) now publishes 352 exact manifest entries as 209 production objects occupying 856,064 native allocated bytes. The immediately preceding current-format baseline used 483 objects and 1,978,368 bytes for the same entries: a 1,122,304-byte (56.7%) reduction. Attributed permanent allocation falls from 9,814,016 to 8,691,712 bytes. The historical #1196 baseline remains separately recorded at 544 entries / 750 objects / 3,072,000 bytes; its additional 192 edge payload references were removed by earlier work, not by bucket encoding. The deterministic acceptance ceiling is 1,536,000 manifest bytes on native 4-KiB allocation storage.

Frozen executable source 36360cb8 preserves the exact semantic fingerprint through real public construction, reopen/query, export, full verification and clean import. The publishing suite additionally exercises immediate mutation, retained streams, subsequent imported mutation, recovery, cancellation and retry. Maximum-field and corruption tests, exact eight-to-nine split/collapse and retained-root checks, and the 128/256/512-entry update ladder cover the manifest boundary directly.

The source-bound resource record is bounded-manifest-1204.json. Its full-fixture CPU observations include the existing codec experiments and portable lifecycle; they are not a query-speed benchmark. Application syscall reads are 1,303,689,747 baseline versus 1,301,114,860 candidate bytes; writes are 267,812,449 versus 267,870,503 bytes. Thus the physical allocation saving does not imply reduced write traffic in this workload. Process RSS, OS block I/O, and separately sampled overlapping filesystem owners are reported with their measurement limits. The candidate point-in-time whole-project census includes 21,979,136 allocated file bytes plus 1,257,472 directory bytes; no equivalent baseline census or whole-project reduction is claimed.

Bounded CSR allocation and cost evidence (#1205)

The fixed eight-route public lifecycle fixture uses 4,097 nodes, 65,537 edges, random full-width identities and nullable properties. Frozen release executables compare the uncompressed current baseline with the bounded compressed writer; both produce the same semantic fingerprint through reopen, export, full verification and clean import.

Measurement	Uncompressed baseline	Bounded Zstd
Actual CSR payload bytes (18 shards)	4,920,564	1,324,020
Actual CSR allocated bytes	4,972,544	1,363,968
CSR metadata logical bytes	10,329	11,431
Attributed permanent allocated bytes	15,151,104	11,542,528
Whole-project file allocation, point census	44,806,144	41,197,568
Whole-lifecycle process peak RSS, KiB	220,400	238,252
Whole-lifecycle syscall read bytes	2,309,281,609	2,098,377,656
Whole-lifecycle syscall write bytes	357,447,042	328,689,815
Sampled overlapping workspace allocated peak	89,038,848	74,625,024
Cold-probe direct CSR read bytes	35,552,246	9,577,462
Cold-probe first-query median, seconds	3.855	3.891

Payload and allocation both meet the deterministic 70% reduction budget. The whole-lifecycle RSS increase is a measured cost, not a decoded-memory improvement. The cold probe executes exact 256-row two-hop public queries in three fresh processes, with four subsequent queries per facade. Its predeclared timing, RSS and syscall-I/O investigation thresholds pass; this is not a latency improvement claim or a noisy CI timing gate. Private-file cache advice does not guarantee OS cache eviction. Syscall traffic is not physical I/O.

The workspace sampler deduplicates overlapping file owners by device/inode and includes project, staging and portable state. It excludes directory blocks and open-unlinked files and can miss short peaks; the largest observed sampling gaps are approximately 62 ms and 59 ms. These are sampled workspace peaks, not hard temporary-disk bounds. The separate point census includes retained generations. See bounded-csr-1205.json for frozen source/executable hashes, exact observations, commands, decoded bounds, CPU/I/O costs and limitations, including the superseded incomplete baseline trace.

Topology file membership

A compact session retains an explicit topology authority from its generation’s manifest. enumerate_topology_files returns those node and edge files together with the session’s own staged replacements and appends. Payload readers receive that file list; directory discovery is confined to the explicit standalone legacy boundary. A route-scoped property inventory cannot authorize a complete topology scan. A newly created generation without graph participants has an explicit empty authority.

The same authority governs mutation preparation, UUID rebuilding, publication, and reopen. Publication captures only selected topology files, so an unrelated well-formed file placed in a node or edge directory cannot become authenticated through a refresh. A staged commit prepares its candidate membership before installing files and swaps the authority after successful installation; rollback restores the session’s prior membership along with its files.

Ordinary Cypher property ownership

Committed edge SET, map updates and REMOVE resolve the authenticated property owner before accumulating effects. Each input batch probes only its logical relation routes and _exploratory, including newest tombstones. Multiple owners are refused; an entity without a property row keeps its logical route. Edges created in the same statement retain the pending writer’s route. Replacement maps read existing keys in batches from the resolved owner, so omitted properties are removed from the same authority that receives the replacement.

Typed scans and fixed expansions carry their catalog-resolved route as an internal constant column. This uses the generation’s storage binding, without reinterpreting ontology IDs as runtime IDs. During staging, current property rows and live counts come from the writable workspace. The pinned generation supplies declared semantic schema metadata for a route’s first property write; its historical counts and generation authority are not substituted for the workspace’s. SET/REMOVE and pending appends therefore share workspace authority. Immediate query success is insufficient: ownership fixtures also reopen after qualified mutations and exercise full portable verification, clean import and later mutation.

The property_writes demand diagnostics sum owner probes and replacement-key reads across every input batch and SET item. Repeated reads are counted each time. Decoder and authenticated-snapshot peaks describe those readers; target counts describe retained identities. These counters neither measure the entire publication nor bound process RSS. The representative multi-batch regression checks exact results and explicit cumulative work ceilings.

Frozen-release measurements cover the entire test process, including fixture construction, mutations, exact queries and portable lifecycle work. The original baseline refuses corruption before completing that lifecycle, so these are corrected-path costs rather than a speedup comparison.

Fixture	CPU seconds (3 runs)	Maximum RSS (KiB)	Successful syscall read / write bytes	Sampled allocated bytes
Mixed construction and ordinary mutation, 33 / 4,097 nodes	4.20–4.21	120,536	340,327,349 / 69,275,673	14,802,944
8,193-edge multi-batch owner and replacement work	1.57–1.60	166,980	114,304,045 / 25,674,019	5,394,432
Qualified mutation, reopen and portable lifecycle	1.07–1.16	90,456	28,610,264 / 2,582,102	2,129,920

Syscall bytes include non-file descriptors and count a file copy once as a read and once as a write; they are not physical disk I/O. The evidence records GNU filesystem counters separately. Disk sampling deduplicates shared device/inode pairs across the private workspace and all coexisting lifecycle artifacts. Its largest observed gap was 15.10 ms despite a requested 5 ms interval; it excludes directories and unlinked open files and is not a hard bound. Native process RSS is separate from the logical reader-buffer ceilings above.

See Cypher ownership evidence for executable/source hashes, all observations, deterministic budgets, reproduction instructions, failed prototypes and the known local #1192 fixture limitation.

Property scan projection reaches the authenticated overlay reader before decoded rows and external merge runs are built. The scan restores requested column order and duplicates afterward; an empty projection retains its row count. UUID and tombstone keys, full-file authentication, schema/resource admission and late failure refusal remain mandatory. This changes query materialization, without changing permanent encoding or replay/compaction policy.

A frozen release comparison on 4,097 random-ID nodes with sixteen nullable 256-byte text columns measured three fresh processes per query, each executing five queries. Exact nullable values and identities passed for every run.

Query	First-query median ms, before → after	Reused-query median ms, before → after	Process CPU seconds, before → after
UUID and score	321 → 37	291 → 28	1.49–1.55 → 0.21–0.22
All properties	327 → 342	294 → 309	2.61–2.72 → 2.71–2.77
Ordered LIMIT 16	323 → 38	291 → 28	1.53–1.57 → 0.21–0.22
Negative text lookup	321 → 57	277 → 46	1.46–1.56 → 0.29–0.31

For UUID/score projection, five-query successful syscall reads fell from 227,678,948 to 90,168,422 bytes and writes from 131,441,481 to 44,041,853 bytes. Avoided writes exceeded the preselected 76,267,520-byte threshold: one raw pass over all unrequested payload values per query. Maximum observed RSS fell from 136,192 to 76,692 KiB. GNU filesystem inputs remained 84,808 blocks; this is not physical-read reduction evidence. Full projection has effectively unchanged I/O and slightly higher measured latency/CPU within its baseline-derived envelope. Process CPU includes facade open and exact result checking, so comparisons use the same query and oracle on both executables.

A separate descriptor-aware sample counted private regular files and open unlinked files together, globally deduplicated by device/inode. Narrow-query sampled overlapping allocation fell from 37,019,648 to 20,127,744 bytes, including the unchanged 17,768,448-byte retained project. The collector inspected only the private roots and descendant descriptors; the executable retained its original user/group identity. Sampling is non-atomic, excludes directory blocks and unlinked mappings without an open descriptor, and is not a hard bound. Earlier directory-only samples remain recorded with their narrower scope.

Physical-plan metrics retain cumulative completed-reader spill/authentication bytes and rows, plus the maximum logical decoder retention. They are neither native RSS nor failed-reader work. An executed scan regression asserts exact reordered/duplicate/empty projections and payload-derived spill/decoder ceilings; public lifecycle tests additionally cover mutation, snapshots, reopen and portable verification/import followed by mutation.

See property projection evidence for source/executable hashes, all observations, selection budgets, instrumentation limits, reproduction commands and validation results.

Integrated query and maintenance assessment (#1207)

The comparison starts after the permanent-storage prerequisites and uses main a3a620c9b2e81108e4a21830c404920c2f948dd4. Rust 1.96.0, Arrow/Parquet 58.4.0 and DataFusion 54.1.0 are pinned by the repository. Measurements used an admitted ext4 process root on a Ryzen 7 3800X (16 logical CPUs), approximately 128 GiB RAM and 4 KiB allocation units. Scalar warm medians are the median of three per-process medians, each using executions 2–5; they are not pooled medians. Each executable was built once, copied, made read-only and SHA-256 checked before and after its subprocess campaign. No native build overlapped resource measurements. The accompanying evidence records source/executable hashes, exact commands, budgets and observations.

Two selected query repairs are already merged: #1241 moves an eligible input predicate ahead of fixed-path expansion, and #1247 projects property values before overlay materialization. Their existing evidence files contain public mutation, reopen and portable-lifecycle proof as well as exact result oracles. On their respective workloads, warm query medians fell from 3.850 s to 0.254 s and from 290.6 ms to 27.8 ms. These are separate workloads; their speedups must not be multiplied. Neither query repair reduced the measured physical filesystem input blocks. Full-width property queries became slightly slower within their preselected envelope; that is not a full-scan speedup.

Candidate	Bounded comparison and decision
Predicate/projection placement	#1241 and #1247 are validated, merged improvements. The fixed-path probe reduces generated candidates from 1,113,889 to 272 and key rows from 8,932,490 to 80,898. The wide narrow query avoids 87,399,628 syscall write bytes across five executions. Remaining key validation is not credited as avoidable payload work.
Optimizer statistics	Conservative cardinality forwarding changes actual join/aggregate execution, but none of seven scalar workloads meets its preselected process-CPU saving. Empty-label warm execution improves from 14.6 to 3.3 ms and saves 472,867 read / 331,707 write bytes over five queries; retain this result without claiming a general improvement. Exact forwarding is invalid because the child statistics ignore per-fragment limits and footer metadata does not prove completed decoding. The experimental patch is reverted. A separate identity-row-marker experiment meets its empty-count CPU budget and selects #1249; it does not forward statistics.
Parquet layout	On five CURRENT wide shards, 128-row groups reduce a one-target-per-shard second pass from 8,094,549 to 1,012,462 compressed column bytes. This excludes the mandatory full validation pass. Repeated-text allocation doubles (143,360 to 286,720 bytes), while dense second-pass bytes increase from 109,509 to 160,364. Reject the global row-group change. The 16 KiB page target produces byte-identical files with the existing 1,024-row write batch. Plain strings save 45,056 allocated bytes for random text but breach the observed RSS selection cap and add 12,288 bytes for repeated text. Reject that global encoding change. Conditional policies remain unvalidated possibilities, not selected repairs.
Property Bloom filters	Random-text filters occupy 8,303 bytes and produce 14 false positives in 20,480 verified-absent checks, with no false negatives for present values. Repeated-text filters occupy 235 bytes. Current readers must validate requested values before negative pruning, so these experiments demonstrate filter effectiveness but zero validated public read savings. Do not publish filters without an admitted consumer or credit skipping mandatory validation.
Fragmentation and maintenance	Public construction, eight fixed CREATE/DELETE events and 1/8/32 repeated property transactions preserve exact UUID/value/null results. Existing compaction folds all runs, and existing cleanup reclaims retained generations. Query-only warm latency remains approximately 14 ms. Full process CPU falls from 0.46–0.50 s to 0.39–0.40 s; this includes open/validation and is not a query-latency gain. Compaction fails the preselected read-I/O ceiling in all three cases. The failed budget stays visible; whole-tree authentication cannot be removed merely by retaining a lease or parsed runs. No additional compactor is selected.

The additional selected repair, #1249, anchors the existing null-preserving COUNT(*) row marker to an available qualified identity instead of an unused property. All seven scalar oracles match. Count-all warm time falls from 14.319 to 10.853 ms; its process-CPU saving misses the preselected 0.02 s threshold, while empty-label count meets that threshold (0.17 to 0.15 s). Count-all, label count and empty count each avoid 923,923 syscall read bytes and approximately 308.8 KB writes per five executions. Explicit nullable count, SUM and selective-property count remain controls. All measured RSS ceilings pass. The prototype is reverted from this assessment; #1249 owns production implementation and its direct correctness/CI close gate.

Qualified row-count marker — #1249

The production marker now selects an existing qualified identity from the aggregate input, retaining the scalar-only fallback and true-for-null behavior. Explicit nullable counts keep their argument. The earlier prototype observations above remain historical; the final paired evidence uses baseline 6907d926 (tree-identical to merged b9765e8e) and candidate e8cb7dcb, frozen before any subprocess measurement.

On the same 4,101-node fixture, count-all warm execution changes from 14.453 to 10.926 ms, label count from 14.482 to 11.314 ms and empty-label count from 14.701 to 11.064 ms. Median process CPU falls by 0.03/0.02/0.03 seconds, meeting those cases’ original 0.02-second thresholds. Each avoids 923,911 syscall read bytes and approximately 308.8 KB writes over five queries. Process CPU includes facade opening and the oracle; warm timing covers execution only. Filesystem input blocks remain 3,328: these are not block-device read savings.

All exact scalar oracles pass, and six of seven original RSS ceilings pass. The explicit identity-count control reaches 70,184 KiB against its unchanged 69,952 KiB ceiling. That observation remains failed. A fixed eight-pair, alternating diagnostic crosses the same ceiling on the unchanged baseline three times and candidate once. Executed plans and work are identical for this control; sampled residence is dominated by executable mappings. This supports retaining the targeted improvement with a disclosed control-budget limitation, not claiming all ceilings pass, proving RSS equivalence, or conclusively attributing the original unsampled peak. No threshold or failed result is replaced by the diagnostic.

The 129-row deterministic public regression writes 16,184 logical spill bytes for row counting versus 50,465 for nullable property counting, avoiding at least the 29,952 unrequested payload bytes. Both authenticate 8,850 bytes and validate 129 physical rows. Logical decoder peaks are 38 versus 624 bytes; these are reader admission estimates, not process RSS. Completed diagnostic counters are reported per physical-plan occurrence and are not additive shared-plan totals.

Exact UUIDs, nullable scores and payload hashes survive public construction, SET with an active old stream, immediate queries, reopen, export/full verification, clean import, subsequent REMOVE and another reopen. A property-file modification after facade admission returns structured GF_PROJECT_CORRUPT for demanded-route counts, including an empty-result predicate. Unknown labels return exact non-null zero without opening an unrelated payload, preserving the existing demand contract. The nullable-count and typed-error repairs in #1251/#1253, mandatory-key and late-decoder refusal tests remain in force.

The permanent and whole-fixture physical inventories are unchanged. Descriptor samples include unlinked open files and globally deduplicate overlapping owners; both variants have a sampled whole-fixture maximum of 5,447,680 bytes. Individual samples and their scheduling gaps are not hard bounds. This repair changes no publishing, replay, compaction or permanent-format policy.

The fragmentation test fixes topology cardinality at 4,101 live nodes across chain lengths; it repeatedly overwrites the same score and removes/restores an actually present nullable property. It asserts the verified run count rather than assuming API calls leave a delta chain. Compaction input is 2/16/64 records and 526/4,376/17,504 bytes. Retained allocation grows from 17,821,696/28,487,680/66,658,304 bytes to 19,333,120/29,999,104/68,169,728 bytes while old generations coexist. Explicit cleanup with zero retained ancestors, after private fixture leases are released, reduces this to 2,330,624/2,359,296/2,457,600 bytes. CURRENT is unchanged by cleanup, and all public query oracles pass afterward. This is an operator-policy experiment, not a change to default retention.

Compaction reads 62,176,116/62,292,189/62,685,028 syscall bytes against ceilings 49,654,012/49,774,517/50,180,680. The comparison includes compaction, facade refresh and three validation queries; its read baseline includes open plus 15 queries. It does not isolate query cost. A repeated run load can reread the 17,504-byte chain and a full graph authentication pass; the latter has a conservative 1,148,656-byte lower bound from the cleaned CURRENT inventory. A lease prevents cleanup, not in-place modification. No unchecked “already verified” flag or weakened corruption check is justified by these observations.

Resource meanings are deliberately separate. Syscall return bytes include cached I/O and copy operations; GNU time filesystem blocks reflect a different boundary. Logical decoder/join/replay counters are not process RSS. Directory and descendant-FD samples deduplicate device/inode owners and include unlinked open files, but exclude unlinked mappings without descriptors and directory blocks. Their observed peaks, including their recorded scheduling gaps, are samples rather than hard bounds. Failed and superseded collectors/builds stay identified in the evidence; no incomplete baseline supports an improvement.

These decisions cover the tested settings and workloads, not every possible layout, conditional encoding policy or planner feature. The final epic ledger must also include the existing admitted S20/S22 ladder on the merged executable. No S24/S26 run or additional certification workflow is authorized here.

Reproduction uses crates/graphforge-api/tests/permanent_storage_budgets.rs: fragmentation_statistics_public_probe, statistics_public_query_probe and property_layout_bloom_assessment. The default tests construct their own public fixtures and assert results; environment variables select retained fixtures for separate subprocess measurements. The layout fixture verifies actual decoded schemas/values, all present Bloom values, 4,096 verified-absent probes per shard and selected-row second-pass results. It never changes the published source project.

Build the selected source with cargo test --release -p graphforge-api --test permanent_storage_budgets --no-run --message-format=json, using an isolated CARGO_TARGET_DIR and admitted TMPDIR. Extract the reported test executable, copy it outside the build tree, make it read-only and record sha256sum before starting any measurement. Run each case in three fresh processes under /usr/bin/time -v, without concurrent native builds:

GF_FRAGMENT_ROOT="$EVIDENCE_ROOT/chain-32" GF_FRAGMENT_ROUNDS=32 \
  GF_FRAGMENT_MODE=prepare "$FROZEN_TEST" --exact \
  fragmentation_statistics_public_probe --nocapture --test-threads=1
# Repeat read before compaction, after compaction, and after explicit cleanup.
GF_FRAGMENT_ROOT="$EVIDENCE_ROOT/chain-32" GF_FRAGMENT_ROUNDS=32 \
  GF_FRAGMENT_MODE=read /usr/bin/time -v "$FROZEN_TEST" --exact \
  fragmentation_statistics_public_probe --nocapture --test-threads=1
GF_STATS_ROOT="$EVIDENCE_ROOT/chain-32" GF_STATS_CASE=count_all \
  /usr/bin/time -v "$FROZEN_TEST" --exact \
  statistics_public_query_probe --nocapture --test-threads=1
GF_LAYOUT_SOURCE="$EVIDENCE_ROOT/wide/source" \
  GF_LAYOUT_OUTPUT="$EVIDENCE_ROOT/layout-fresh" GF_LAYOUT_DATA=random \
  GF_LAYOUT_VARIANT=baseline /usr/bin/time -v "$FROZEN_TEST" --exact \
  property_layout_bloom_assessment --nocapture --test-threads=1

Prepare the wide source with wide_property_public_query_probe, GF_WIDE_PROBE_ROOT and GF_WIDE_PROBE_MODE=prepare. Fragment modes are prepare/read/compact; use chain lengths 1, 8 and 32. Scalar cases are count_all/count_identity/count_label/count_property/sum_property/selective/empty. Layout cases are baseline/row_group_128/page_16k/string_plain/bloom, with GF_LAYOUT_DATA=random/repeated and a fresh output directory each time. Compaction executes once per prepared fixture. A separately frozen CLI runs maintenance cleanup-preview and maintenance cleanup-execute --yes, both with --retained-ancestors 0 --cleanup-batch 0, only after the private fixture’s leases are released. This preserves CURRENT and does not change default policy.

The aggregate evidence contains all process observations, syscall totals, page/row-group counts, selection ceilings, FD samples and failed/superseded evidence limitations. Tracing uses strace -f -qq -yy -s 0 with read/pread/readv, write/pwrite/writev, copy/sendfile and fsync/fdatasync families. Traced and sampled runs are separate from latency observations. Source-only conditional policies, Bloom consumers and stronger immutable authentication handles are not silently counted as implemented improvements.

Research revision 2 adds authenticated CAS placement for immutable Version participants and graph-row projections (ADR 0039). Retention passes the existing exclusive CAS guard through content authentication, then rechecks generation reachability before deletion. Recovery/read-side reachability acquires shared CAS access before the writer lock. No callback reacquires a shared CAS lock while cleanup holds it exclusively. Historical materialization writes only to an empty private target and preserves the Project’s read-only state.

Physical durability owner

graphforge-storage::durable_commit owns physical staging, descriptor seals, atomic visibility, retained-parent acknowledgment and exact owned retirement. Callers retain content checksums/authentication, logical predicates and recovery formats. SealedArtifact consumes the actual producer descriptor or an opaque seal witness; PendingCommit retains uncertain visibility and acknowledgment. A directory batch validates all visible identities before one fence. Construction batches share their parent and close unlocked producer handles after visibility; recovery-owned inputs survive failed publication. No failed acknowledgment may remove visible authority. Append journals bind their accepted offset to the same retained descriptor and namespace. Cache release retains its final dirty window.

The durability inventory method records the reproducible counting unit and input digests.

Only uncheckpointed sorting/merge scratch barriers are retired: rebuilt outputs still receive their durable seal before publication. Durable rewrite inputs, UUID intents and checkpoint recovery inputs remain sealed. Two duplicate delta and compaction tests that ran only the generic crash model are replaced by five actual primitive fault tests. Real family reopen, native replacement and modeled power-loss omission witnesses remain. The standard Cargo CI discovers the new owner tests automatically. No additional product workflow/job is retired here; the gate registry and platform lanes retain their existing coverage.