Hyperlawhypertext legal database

Info How the data is built

How the data is built

From an official file to a node, an edge and a page.

Hyperlaw is a pipeline, not a hand-built website. Each corpus is fetched from its publisher, parsed into a common model, linked, loaded into Postgres and rendered by the site. This page explains the shape of that pipeline and the choices that affect what you see.

From source to node

  1. Fetch. A corpus adapter downloads the official XML, HTML, JSON or PDF and records a source URL and checksum. Conditional GETs avoid re-downloading unchanged documents.
  2. Parse. A jurisdiction-specific parser maps the publisher's structure onto instruments, nodes and edges. Section numbers become citation paths; headings become structural nodes; definitions and schedules get their own anchors.
  3. Resolve. A citation grammar extracts references from the text and resolves them against the loaded instruments. A reference that names an Act, section and subsection is resolved only when the target exists.
  4. Load. The loader is idempotent: re-loading an instrument replaces its nodes and edges without disturbing other corpora.
  5. Render. Pages are server-rendered and cached by instrument revision, so a content change invalidates only the affected pages.

Precision over recall

The engine would rather leave a reference unresolved than guess. A bare s. 8 is bound only when the surrounding text makes the Act unambiguous; an unknown Act name stays plain. This is why some citations you can resolve by eye are not links. The alternative — a plausible but wrong link — is worse than no link in a legal research tool.

The same rule governs case citations. Full citations, short names with a clear antecedent and pinpointed mentions link; a bare ibid. or an ambiguous surname stays text. When a case is recognised but not held, the span may be marked wanted rather than linked to a guessed decision.

What is stored

The database stores instrument metadata, nodes (with citation paths, labels, text and amendment years), citation edges (with span positions and provenance), case records and their paragraphs, footnotes, citations and edges, plus history snapshots and the Reader's private documents. It does not store publisher annotations, editorial notes or headnotes as law; those are apparatus and are excluded from the text.

Search is PostgreSQL full-text search over instrument titles, provision text, case styles of cause and case paragraphs. It is not semantic search and it does not use an external model. Results are ranked by match quality, with aliases and exact citations first; the search page can be sorted by date, title, docket number or link count.

Provenance and corrections

Each instrument records its source URL and checksum. A link in a judgment records the span and the target, so the site can show you why it made the connection. Corrections can be applied at render time without rewriting the stored engine output; a later re-cite or corpus update can then replace the correction with a real target.