Large tenants

The manifest costs about 41 tokens per concept, so it stops being worth reading whole somewhere past a few thousand of them.

Narrow instead of paginating.

await connection.manifest('sales'); // one bundle
await connection.search('orders', { bundle: 'ops' }); // rank within one bundle

Why a bundle slice is cheap

The snapshot groups rows by bundle, so one bundle is a contiguous run of lines. The reader splits the manifest by bundle once, the first time a filtered manifest is asked for, and serves every later call from that split; the unfiltered read stays a straight slice of the snapshot.

Concepts	500	5,000	20,000
Bundles	1	10	40
Whole manifest	20,549	205,851	835,922
One bundle slice	20,549	20,419	20,486
Snapshot on disk	0.56 MB	5.6 MB	22.6 MB

Each bundle holds 500 concepts, so the slice a reader asks for stays the same size as the tenant grows. At 500 concepts there is only one bundle, which is why the first column matches.

The slice stays flat as the tenant grows. That is the whole trick: a 20,000-concept tenant costs 20,486 tokens to read one bundle of it, against 835,922 for all of it.

Which to reach for

You know	Use
The domain	`manifest(bundle)`
Nothing, but you have a question	`search(query)`
The domain and the question	`search(query, { bundle })`
The exact ids	`get(ids)`, skip the manifest

Manifest or search, decided by arithmetic

Whether to read the manifest whole or rank first depends on the corpus, not on a concept count. The manifest is paid once and amortised over a session; a search is paid per question. So the MCP server measures the manifest at startup, estimates one search result from the manifest's own row lengths, and appends one sentence to the manifest tool's description: prefer search once the manifest costs more than twenty results, read it whole below that. On the 500-concept reference catalogue the benchmarks use, it reads:

This tenant's manifest measures ~21,000 tokens; prefer "search" over reading it whole.

The rule is adviceFor, exported from langonrock/client along with ADVICE_RATIO. It is a string in and a string out, so a client that holds a manifest reaches the same verdict the server states, deterministically: same manifest, same advice.

Search stays narrow on purpose

BM25 returns the top k, then a deterministic one-hop expansion over the link graph adds concepts the hits point at. That is usually where the join partner, the parent dataset or the metric definition lives, and finding it costs no model call.

The expansion is capped at k and ranked by how many hits point at a target.

The cap is not optional

Without it, a single hub drags in everything it links to. On the Stack Overflow sample one dataset row with sixteen outgoing links turned eight hits into twenty-six rows, which is most of the manifest. Search that does not narrow is worse than reading the manifest, because it costs a round trip to learn nothing.

Pass expand: false to turn it off entirely when you want exactly the ranked hits.

Latency is not where the problem is

Median milliseconds of three runs, Bun 1.4.2 on an Apple M1 Pro:

Operation	500	5,000	20,000
Compile and write a snapshot	42	226	725
Open a snapshot, cold	0.86	4.26	18.9
Read the whole manifest	0.03	0.11	0.44
Batched `get` of 3 sections	0.04	0.04	0.05
Build the BM25 index	19	152	578
BM25 query, with `pos`	0.22	0.55	1.65

get is flat. A batched fetch of three sections costs the same 0.04 ms on twenty thousand concepts as on five hundred. Compiling, indexing, search and the whole-manifest read grow with the corpus.

None of this is where the time goes, though. One saved model round trip is worth about a second, three to four orders of magnitude more than any read in this table. Optimize round trips, not milliseconds.

Memory is the real ceiling

The index is rebuilt in memory once per snapshot: 1.1 MB of heap at 500 concepts, 8.1 MB at 5,000 and 32 MB at 20,000. In the engine benchmarks, a fresh process that opens 20,000 concepts, builds the index and searches peaks at 0.36 GB resident. A daemon keeps the 16 most recently used tenants open with their indexes, which is the number to watch before any of the latencies above.

The build streams the bodies off a single read of the snapshot's blob region. After a folder import serve rebuilds the index right away, so the first search after a save to the folder does not pay for it; after a commit through the API, the first search does.