Skip to main content

Module storage

Module storage 

Source
Expand description

Row-per-entry storage, schema v2 (Phase 30, review-01 section 2.1; ADR-0138).

Schema v1 kept the whole vault as one JSON string in one row, so every write parsed and re-serialised the lot, the compare-and-swap token was a hash of the blob (two people editing different entries conflicted, and both branches of the conflict prompt discarded someone’s changeset), version_history grew inside the thing written most often, and nothing could be indexed.

v2 stores one row per entry and per project, a row for the category list and a row for every other top-level key. The public document API is unchanged (load_vault returns the same JSON, save_vault takes it), so the desktop app, unv-server and unv need no change; what changes is what a save does:

  • only rows whose content changed are written;
  • version_history lives in its own table and is attached on load;
  • every save records which rows it changed in vault_changes, and a writer who read an older version is merged, not refused: entries it left untouched keep whatever the other writer did, and only an entry both sides changed (or one side changed and the other deleted) is a conflict, named in the error.

§The version token

"<seq>.<state hash>". The hash is over every row’s (kind, key, position, content hash); the sequence number says when, so a stale token can be turned into “which rows changed since” instead of “something changed”. The token is opaque to callers, as before; a token in the old bare-hex form fails the merge lookup and is a whole-vault conflict, which is what it always was.

§What a merge can and cannot know

A row’s content hash (rev) is the identity of its content. For each row the writer sends back, the changes since its base version tell us what that row looked like when it read it (prev_rev of the first later change). If the writer’s copy still has that hash, the writer did not touch it and the stored row wins; if the writer’s copy matches the stored row, there is nothing to decide; otherwise both sides edited it. History is bounded (the newest KEEP_SAVES saves); a token older than that is a whole-vault conflict.

Structs§

Ent
One storable unit of the document.

Constants§

HARD_KEEP_SAVES
An absolute ceiling, so a runaway loop cannot grow the change log without bound.
KEEP_DAYS
Saves newer than this many days are kept too (Phase 30.2): a script saving thousands of times a day used to burn through 2,000 saves in hours, and an hour-old token then became a whole-vault conflict.
KEEP_SAVES
The newest saves of change history that are always kept for merging stale writers, however old they are.

Functions§

entry_history
One entry’s version_history, read without loading anything else (Phase 30.2). None when no entry has that id; an empty array when it has no history.
init_schema
join
Reassemble the document from units in order, consuming them.
load
load_ent
One stored row by key, with its history attached for entries.
load_ents
Every stored unit in document order, or None when nothing has been saved.
load_ents_opts
As load_ents, optionally leaving out each entry’s version_history, which is up to 50 revisions of secret material per entry and is not wanted by a selective read.
load_lite
The document without any entry’s version_history.
merge
Resolve the writer’s units against what is stored. Returns the units to persist, or the conflicting row labels.
merge_window
How far back a stale writer can still be merged: the oldest kept save’s time and how many saves are kept. None for a vault never saved.
migrate_if_needed
Migrate on its own transaction (the read path). Cheap when nothing is pending.
migrate_in_txn
Convert the v1 blob into rows. Must run inside a write transaction. No audit rows are written: nothing about the data changed, only where it lives.
needs_migration
True when a v1 blob is waiting to be converted.
snapshot
split
Split a vault document into storable units, in document order. Consumes the document so a large vault is moved, not cloned.
verify
Integrity: every row hashes to its rev, and the rows hash to the stored token.
version
The current version token, or None for an empty vault.
write
Persist ents as the new state. Must run inside a write transaction. Only rows whose content, position or history changed are touched. Returns the new token (the current one, unchanged, when nothing differs).

Type Aliases§

Snapshot
Every row’s (rev, pos), read once per save.