Architecture

GraphForge is a Rust-core embedded workbench: Cypher and analyst verbs converge on DataFusion execution and Arrow results, with Parquet/JSON project storage and thin language bindings. Detail and status banners live in ../book/architecture/overview.md; this document is the public contributor lifecycle summary.

Context diagram

flowchart LR
    Analyst["Analyst / notebook"] --> Bindings["Python / Node / Rust facade"]
    Agent["Coding agent / skills"] --> Bindings
    Bindings --> Api["graphforge-api"]
    Api --> Cypher["graphforge-cypher → graphforge-ir → graphforge-rel → graphforge-exec"]
    Api --> Verbs["Analyst verbs"]
    Verbs --> Exec["graphforge-exec / DataFusion"]
    Cypher --> Exec
    Exec --> Arrow["Arrow RecordBatches"]
    Api --> Storage["graphforge-storage Parquet + JSON"]
    Knowledge["graphforge-provenance / graphforge-knowledge"] --> Storage
    Knowledge -. "UUID reference only" .-> GraphData["Graph topology + properties"]
    Storage --> GraphData

Components

Component	Responsibility	Depends on
graphforge-api	Public facade: lifecycle, Cypher, analyst verbs	graphforge-cypher, graphforge-exec, graphforge-storage, knowledge crates
graphforge-cypher / graphforge-ast	Parse Cypher to AST	—
graphforge-ir / graphforge-rel	Graph IR and relational lowering	AST / ontology
graphforge-core	Shared identities, values, options, and facade errors	No workspace crate dependencies
graphforge-plan	DataFusion logical extension nodes and mutation specifications	Core, IR, DataFusion
graphforge-exec	DataFusion execution, algorithms, search	IR, logical extension nodes, storage
graphforge-io	Bounded, atomic Parquet and Arrow IPC result sinks	Arrow, Parquet, futures; no workspace crate dependencies
graphforge-storage	Project generations, Arrow schemas, and Parquet storage	Core, discovery, filesystem, IR, ontology
graphforge-ontology	Progressive ontology validation	API / binder
graphforge-provenance / graphforge-knowledge	Provenance and epistemic records	UUID references to graph
Bindings	Thin FFI projections	graphforge-api / core facade

Data model

• Project — durable research universe containing or referencing evidence, graph knowledge, metadata, ontologies, and research lineage. M11 extends the existing project workspace with Branches, Versions, and Proposals.
• Node / edge — graph-native entities with properties; surrogate keys in execution; UUID identity at the API boundary.
• Ontology — optional progressive model (exploratory / advisory / strict).
• Knowledge records — provenance, confidence, epistemic assertions attached by UUID, never as graph-table columns that alter Cypher results.
• Results (data plane) — Arrow tables/batches for Cypher, analyst verbs, and other tabular/data-bearing surfaces. Control, metadata, lifecycle, explanation, and construction may return scalars, collections, unit, or handles; see ../book/architecture/overview.md.
• On disk — Parquet for graph data; JSON for metadata and contracts.

Schemas and frozen inventories also live under ../contracts/ and architecture deep-dives in ../book/architecture/.

Problem model and terminology

Concept	Meaning in this project	Relationships, states, rules, and owner
Project	Durable research universe, not a single graph snapshot	Existing graph/knowledge workspace extended by the M11 research lifecycle; facade / storage
Graph layer	Topology, properties, traversal, algorithms	Never stores knowledge semantics; Cypher reads only this layer
Knowledge layer	Provenance, evidence, epistemic status	Attaches by UUID; append-only interpretation (ADR 0006)
Workbench layer	Analyst verbs, search, workflows, recipes	Consumes lower layers; holds no graph-semantic state
Progressive ontology	Exploration-first typing	Modes exploratory → advisory → strict; ADR 0003
Catalog ID vs ontology ID	Distinct ID spaces	Never substitute one for the other
Analyst verb	Intent API bypassing Cypher	rank/cluster/paths/analyze/similar/find → Arrow

M11 research model

The analyst research experience supersedes the earlier portable-workspace-only Project definition and expands the workflow to Explore → Focus → Branch → Analyze → Compare → Propose → Integrate. The research workspace contract defines Slice selection, independent Branches, immutable Versions, separately governed Forks, and selective Proposals.

Evidence, Knowledge, Research, and Lineage are analyst-facing conceptual layers, not replacements for the three Rust ownership layers. Evidence/Artifact derivation, claims, hypotheses, and acceptance history use domain-owned records with UUID references. Graph visibility and local graph edits belong to the selected workspace’s graph view. Facade orchestration assembles both under storage publication authority; bindings hold no alternative state engine. Branches share one Project container and CURRENT authority; parent integration and its acceptance receipt publish together. A Branch’s original base remains immutable while selected object/field comparison baselines advance explicitly.

Explicit graph suppression changes a Branch’s active graph. Epistemic status and canonicality never silently filter ordinary Cypher or algorithms. Canonical means accepted in a Project/community context and is distinct from confidence or the existing “supported” status. Integration does not automatically promote imported knowledge to canonical; that is a separate recorded decision, optionally in the same publication.

Associated projects including XYG and graphforge-nextjs, other applications, and peer extensions own UX rendering, hosting, authentication, and access enforcement. Core supplies metadata, provenance, comparison, conflict, and proposal state through Rust and thin Python/Node/CLI surfaces. Consumer UI implementation is outside the M11 Core close gate.

Existing checkpoints, append-only knowledge ledgers, ontology composition, and portable selection are foundations. None alone constitutes a Branch lifecycle. Retained research Versions pin selected dependency/baseline closure, not whole ancestors merely for genealogy. Measure retained storage as well as copy cost; external-only evidence and unavailable historical expansion must be disclosed. Compatible-reader reproduction is not pre-v1 migration support. The consumer projections defined in #1346 have native owner tests; #1358 supplies the composed Rust/Python/Node/CLI journey and captured consumer outputs. See acceptance evidence.

Key flows

Cypher query

1. Binding or Rust caller invokes execute on graphforge-api.
2. graphforge-cypher parses; binder applies ontology rules for the active mode.
3. Plan lowers through Graph IR → relational plan → DataFusion.
4. graphforge-exec streams Arrow batches back through the facade.

Analyst verb

1. Caller invokes a verb on graphforge-api (no Cypher string).
2. Facade exports adjacency/index views and dispatches algorithm or search.
3. Execution produces scored Arrow batches via the same result contract.

Project reopen

1. Caller opens a project path.
2. graphforge-storage loads Parquet/JSON generations.
3. Subsequent Cypher/verbs observe the last published generation (recovery rules in storage/checkpoint docs and ADRs).

Cross-cutting concerns

• Error handling: structured GraphForge error codes; fail closed on unsupported containers, capability gaps, and writer-busy.
• Configuration: zero-config in-memory default; path argument selects durable project.
• Security / tenancy: single-user embedded default; not a multi-tenant server.
• Observability: local diagnostics + CI contract gates; see OBSERVABILITY.md.
• Correctness bar: TCK + non-Cypher surface inventories; wrapper/logical-plan tests alone are insufficient (AGENTS.md).
• Build system: Cargo with nextest owns CI Rust compilation and tests (ADR 0048); maturin and napi build the binding packages. See ../development/agent-environment.md. Publish credentials stay in release workflows.

Decisions

ADR bodies and their current active/superseded status live in the canonical ADR index. The table below highlights selected decisions; it is not the complete ADR sequence. The DocSlime index is adrs/README.md (links only; does not renumber bodies).

ADR	Decision
0001	Rust core owns semantics
0002	Recursive descent + Pratt parser for graphforge-cypher
0003	Progressive ontology — exploration first
0004	Graph-native adjacency index
0005	Graph / knowledge / workbench layers
0006	Append-only epistemic interpretation
0007	Runtime temporal values
0008	Heterogeneous list values
0009	Nested heterogeneous lists
0010	Wide date and duration
0011	Dynamic heterogeneous value lists
0012	Knowledge and epistemic domain ownership and schema evolution
0013	Durable project-generation protocol
0014	Complete-workspace checkpoints
0015	Three embedded project-write modes
0016	Repository integration and deployment configuration boundary
0018	Acknowledged durability and isolation contract
0036	The GraphForge release version contract (consolidates and supersedes 0017, 0033 and 0034)

Risks & trade-offs

• Research/notebook scale — not positioned for multi-tenant billion-edge serving; limits documented in ../reference/scale-limits.md.
• Pre-v1 format churn — no compatibility promise for historical containers; operators must use current-generation projects.
• Docs layers — DocSlime lifecycle + Guide + Book; Starlight (docs-site/) syncs an allowlist and must not fork a second product narrative (#2731 polish may still refine nav).
• ADR body home — physical ADR directory is docs/adr/; do not fork a second sequence under engineering/adrs/.