Vocabulary

Nine words carry most of the design. They are worth pinning down before anything else, because the CLI, the API and the engine all use them in the same narrow sense.

Document

One Markdown file stored in the database, addressed by its bundle and its path inside the bundle: sales and tables/orders.md. Its version is the sha256 of its content, and that hash is what a change names when it replaces or deletes the document, so two writers cannot silently overwrite each other. See Transactions.

The database keeps the exact text. Frontmatter, keys the compiler ignores, CRLF line endings and a byte-order mark all come back out of an export byte for byte. Text must be valid UTF-8; invalid byte sequences are refused before anything is published rather than silently replaced.

A path is relative, uses forward slashes, ends in .md and stays within 200 characters. Every segment starts with a letter or a digit and uses only letters, digits, ., _ and -, so nothing can escape its bundle and nothing can hide in a dot directory.

index.md and log.md are navigation documents. They are stored and exported like any other, and never become concepts.

Concept

What the compiler makes of a document: a table, a dataset, a metric, an API, a runbook.

---
type: BigQuery Table
title: Orders
description: One row per completed customer order.
grain: order_id
status: current
---

Only type is required by OKF. The compiler reads these fields and leaves the rest in the stored document:

kind is slugged, so BigQuery Table becomes bigquery_table. Everything else in the frontmatter stays in the document and out of the read model, which is where much of the token saving comes from.

A Markdown file with no frontmatter still compiles: its id comes from the path, its summary from its first sentence, its title from its first heading, and its links from the body. What it earns is a conformance warning, which --strict turns into a failing exit, so OKF conformance is a gate you can enforce rather than the price of admission. A docs folder is knowledge before anyone annotates it.

Bundle

A namespace of documents inside a tenant, and the unit an agent narrows to. On import, every immediate subdirectory of the folder is a bundle, dot directories excepted, and nesting below that is free.

sources/acme/
  sales/            <- a bundle
    tables/orders.md
    metrics/revenue.md
  ops/              <- another bundle
    runbooks/deploy.md

A write that names a new bundle creates it, and deleting a bundle removes every document in it in one commit. Bundle names are 1 to 64 characters of letters, digits, _ and -, starting with a letter or a digit, because the name reaches the filesystem when you export.

Tenant

The unit of isolation. A tenant is a directory in the store with its own HEAD, its own revisions, its own snapshots and its own search index, and a transaction never spans two of them. Bundles compile together into one snapshot per tenant, and the manifest carries a bundle column so an agent can filter without loading anything extra.

Tenant ids follow the same rule as bundle names. The id reaches the filesystem, so it is checked against an allowlist and rejected rather than cleaned up and trusted.

Search indexes are per tenant, never shared. A shared index leaks content across tenants through ranking alone.

Concept ids

An id is the shortest unambiguous form of its path, in three tiers:

1. the bare filename, tables/orders.md becomes orders
2. the parent and filename, if the bare name collides, tables/orders
3. the full relative path, if that still collides

Short ids are worth the effort because the manifest pays for every id on every read.

Across bundles the same rule applies once more. A bare id stays bare while it is unique across the whole tenant, and only the ones that clash pay for a bundle/ prefix. Links are rewritten to match.

Links

The links column is built from ordinary Markdown links in the body. No special syntax, and nothing in the frontmatter.

Joined with [customers](./customers.md) on `customer_id`.
See the [parent dataset](../datasets/orders_db.md).

A link target is resolved relative to the file it appears in, and .md is appended when it is missing, so [customers](./customers) works too. The resulting path is looked up in the same path-to-id map the manifest uses, and the id goes in the column.

These are skipped rather than resolved:

Anchors and query strings are stripped before resolution, so [orders](./orders.md#schema) links to orders. A target that resolves to no concept becomes a diagnostic, stored with the revision and printed on import:

warn: tables/payments.md: unresolved link "./custmers.md"

Link ids are sorted and deduplicated in the row, which is part of what keeps the manifest byte-identical across commits that do not change it.

Manifest

One TSV row per concept, and the artifact the whole read side exists to produce.

# tenant: acme
# bundles: ops sales
id	bundle	kind	status	grain	summary	links
orders	sales	bigquery_table	-	order_id	One row per completed customer order.	customers

Rows are sorted by bundle, then by id. Link lists are sorted. Nothing carries a timestamp, and statistics go to stderr rather than into the file. That is not tidiness, it is a hard requirement: the manifest earns its keep by sitting in the cached prompt prefix, and any byte that changes between commits invalidates the cache.

Summaries are flattened to a single line and truncated at 120 characters by default, since a tab or a newline in a cell would split the row.

Section

A named range inside a concept, taken from its own Markdown headings. get(id, "schema") returns that slice instead of the whole document.

Heading text is lowercased, non-alphanumeric runs become underscores, and a duplicate name gets a numeric suffix. Text before the first heading is body. Headings inside fenced code blocks are excluded, because a # comment line in a SQL example is not a heading and real bundles are full of them.

Revision

One commit: the complete source state of a tenant after a transaction, with a pointer to the revision before it. Its id is the sha256 of its immutable record, which includes the parent and the time of the commit, so two commits always get two revisions even when they store the same documents.

{
  "revision": "0d07a9f035a9416201367d342e146b379021a43cc9234b699c7c559e4ee7b808",
  "parent": "e9c762984ea34398083fa766b787af4d307673612eead03416a3a80dec46112f",
  "snapshot": "0203add32834701b39b60d1b330ad88349a42c93e52bf29d0a660961372098ff",
  "created": "2026-10-01T02:42:30.592Z",
  "concepts": 4,
  "bundles": ["ops", "sales"]
}

Use revision ids for concurrency, history and restore: a transaction can insist the tenant is still at the revision it read, and a restore must. See History and restore.

Snapshot

The compiled read model of a revision: one immutable .tnt file holding the manifest, a directory of concept offsets and section ranges, and zstd-compressed bodies. Its name is the sha256 of its own bytes.

Because identical documents compile to identical bytes, two revisions with the same documents share one snapshot. Restoring an older revision makes a new revision that points at the snapshot the old one had. Use snapshot ids to validate what you cached: the HTTP manifest's ETag is the snapshot digest, and snapshot() returns it.

HEAD

A small record naming the tenant's current revision, its snapshot and its source archive, with their checksums. It is the only file a commit replaces in place. Everything else a commit writes is new and immutable, and is synchronized to disk before HEAD is atomically replaced, so that replacement is the commit point.

A corrupt HEAD fails closed: the engine does not fall back to an older revision on its own. See Durability and failures.

A tenant created before the engine existed has a current file instead of a HEAD. It can still be read, but the database commands refuse it until it is migrated. See Migrating a legacy tenant.