Compound Documents

How live transclusion works in Xudanu: documents assembled from live references in the O-tree — single source of truth, no drift, no copies.

1. The Core Concept 2. Architecture: Inline Transclusion Elements 3. The O-Tree: Single Source of Truth 4. Resolution: Following the Golden Thread 5. Span Migration: Tracking Content Through Edits 6. Attribution Links (Separate from Content) 7. The Placement Flow 8. Live Update Pipeline 9. Self-Transclusions 10. Editor Rendering 11. Use Cases: Simple to Visionary 12. What Was Deprecated

1. The Core Concept

A compound document is a document whose content includes live references to other documents. When a source document changes, every compound document that references it automatically reflects the change. This is Ted Nelson's transclusion: the same content can appear in multiple places simultaneously, always in sync, without duplication.

A Compound Document with Live Transclusion References SOURCE WORK A "The quick brown fox jumps over the lazy dog" SOURCE WORK B "Seseparrendipity happens every day" COMPOUND WORK (O-TREE) Text: "Intro " Transclusion(WorkA, 0:15) Text: " and then " Transclusion(WorkB, 4:20) Text: " the end." Resolved: "Intro The quick brown and then rendipity happens the end."

Figure 1: A compound work's O-tree contains Text elements (editable) and Transclusion elements (live references, zero-width). Resolution fetches source content on demand.

2. Architecture: Inline Transclusion Elements

Xudanu implements compound documents via RangeElement::Transclusion — a first-class element type in the O-tree. This is neither a side-table nor a separate work type. Every work's O-tree can contain a mix of Text elements (editable content) and Transclusion elements (live references). The O-tree is the single source of truth.

The O-Tree Is the Single Source of Truth OLD: SIDE-TABLE (DEPRECATED) O-Tree: "Intro Hello End" (text copy) CompoundEdition side-table: [Text, Span, Text] Problem: two copies can drift apart Migrated to inline elements via CLI tool NEW: INLINE TRANSCLUSION (IMPLEMENTED) O-Tree entries: pos 0: Text("Intro ") pos 1: Transclusion(WorkA, 0, 15) char_len=0 pos 2: Text(" End.") ✓ No side-table — one store only ✓ No drift possible ✓ No special work type needed ✓ CRDT handles it (already supports mixed elements) ✓ Self-transclusions supported (cycle detection) ✓ Recursive resolution (32 levels, golden thread) ✓ Span migration on source edits

Figure 2: The old side-table approach (left, deprecated) vs. the implemented inline approach (right). Transclusion elements live directly in the O-tree.

3. The O-Tree: Single Source of Truth

The O-tree (ordered tree) is the work's primary content store. It already supported mixed element types: Text, Data, Label, PlaceHolder, IDHolder, Work, Blob, Overlay. We added Transclusion as a new variant:

pub enum RangeElement {
    Text { text: String },                              // editable text
    Data { bytes: Vec<u8> },
    Label { label_id: RangeElementId, inner: Box<RangeElement> },
    PlaceHolder { id: RangeElementId },
    IDHolder { id: u64 },
    Work { work_id: RangeElementId },
    Blob { content_hash: u64, mime_type: String, ... },
    Overlay { overlay: ImageOverlay },
    Transclusion { source_work_id: u64, char_start: usize, char_end: usize },  // NEW
}

The Transclusion element has char_len() = 0 — it's a zero-width embed, similar to how rich text editors handle images. It occupies a position in the O-tree but doesn't contribute to the character index. This means:

4. Resolution: Following the Golden Thread

Resolution walks the O-tree, fetching live content from source works for each Transclusion element. If a source work ALSO contains Transclusion elements, resolution follows the chain recursively — Nelson's "golden thread" — up to 32 levels deep. Cycle detection prevents infinite loops.

Recursive Resolution: Following the Golden Thread DOC C (compound) Text: "Top: " Transclusion(DocB, 0:100) Text: " end." DOC B (also compound) Text: "Middle: " Transclusion(DocA, 0:5) Text: " done." DOC A (source) Text: "Hello World" Resolving Doc C: "Top: " + resolve(DocB) + " end." = "Top: Middle: Hello done. end." Max depth: 32 levels. Cycle detection: A→B→A returns to_text(A) instead of empty.

Figure 3: Recursive resolution follows Transclusion chains. Doc C resolves through Doc B to Doc A — the golden thread.

5. Span Migration: Tracking Content Through Edits

When a source work is edited, all Transclusion elements referencing it have their offsets automatically adjusted via migrate_inline_transclusions_for_delta(). This is wired into the text delta dispatch path.

Span Migration: Offsets Track Content Through Source Edits BEFORE: Source = "Hello World" transclusion [0:5] AFTER INSERT "GPS " AT POS 0 transclusion [4:9] Both resolve to "Hello" — the reference followed the content.

Figure 4: Insertions and deletions in source works trigger automatic offset migration.

6. Attribution Links (Separate from Content)

Links are the attribution and provenance layer — separate from content resolution. A transclusion placement creates both a Transclusion element (for content) and a link (for attribution). They answer different questions:

Transclusion elements answer: "What text should I show here?"
Links answer: "Where did this text come from and who vouches for it?"

Links power the backfollow engine (BLAKE3 fingerprint matching), the margin bars in the canvas overlay, and the provenance chain display. They exist alongside Transclusion elements but serve a fundamentally different purpose.

7. The Placement Flow

When a user places a transclusion, only ONE write happens — an ElementInsert that creates a Transclusion element in the O-tree. No text copy is inserted. The attribution link is created separately.

Step-by-step
// 1. Select text in source work
holdSelection(excerpt, sourceWorkId, charStart, charEnd)

// 2. Navigate to target work, click to place
handlePlaceTransclusion(position):
    // A. Insert Transclusion element into O-tree (SINGLE SOURCE OF TRUTH)
    await client.elementInsert(workId, position, {
        type: "transclusion",
        transclusion_source: sourceWorkId,
        transclusion_start: charStart,
        transclusion_end: charEnd,
    })

    // B. Create attribution link (METADATA, separate from content)
    await transclusion.placeTransclusion(client, workId, position)

No text is copied into the O-tree. The Transclusion element IS the reference. Resolution fetches content on demand.

8. Live Update Pipeline

When a source work is edited, the compound system propagates the change through a four-stage pipeline:

Live Update Pipeline 1. TEXT DELTA User edits source work. Delta applied to O-tree. 2. SPAN MIGRATION migrate_inline_transclusions_for_delta() shifts Transclusion offsets. 3. PUSH NOTIFICATION CompoundSourceChanged event pushed to all viewers. 4. RE-RESOLVE Client calls resolveInlineTransclusions() → display updates instantly.

Figure 5: Four-stage pipeline from source edit to compound refresh. Push-based (no polling).

9. Self-Transclusions

A work can transclude from itself. This is the simplest way to create a transclusion — select text in a work, then place it back in the same work. Cycle detection returns the work's raw text (to_text()) instead of infinite-looping.

Example: Self-Transclusion
Work A O-tree:
  pos 0: Text("Hello World")
  pos 1: Text(" - ")
  pos 2: Transclusion(A, 0, 5)   // references itself!

Resolution:
  to_text(A) = "Hello World - "    (skips Transclusion)
  resolve(A):
    1. Text("Hello World") → "Hello World"
    2. Text(" - ") → " - "
    3. Transclusion(A, 0, 5) → cycle detected → return to_text(A) = "Hello World - "
       chars [0:5] of "Hello World - " = "Hello"
  Result: "Hello World - Hello"

10. Editor Rendering

In the editor, Transclusion elements are rendered as read-only inline spans with amber background and dashed underline. The user can read the transclusion content but cannot edit it (it belongs to the source work).

Editor Rendering: Resolved Text with Read-Only Transclusion Spans Intro The quick brown and then rendipity happens the end. Amber background + dashed underline = read-only transclusion Left margin bar (canvas overlay) = per-source indicator getEditableText() extracts only editable text for delta computation. Transclusion text is skipped.

Figure 6: The editor shows resolved text. Transclusion regions are styled and non-editable.

Undo Support

When a user places a transclusion and immediately presses Cmd+Z, the system calls elementRemoveTransclusion() to delete the Transclusion element. This works because the undo handler checks for a pending transclusion insertion when the text undo stack is empty.

11. Use Cases: Simple to Visionary

Level 1 — Quotation Anthology

Collect excerpts from multiple source works. Each excerpt is a live Transclusion element. If a source is corrected, the anthology reflects it automatically.

Level 1 — Shared Boilerplate

Maintain a single "disclaimer" work. Every document referencing it has a Transclusion element. Update once, all update.

Level 2 — Collaborative Study Guide

Assemble passages from a textbook (Transclusion elements) with interleaved commentary (Text elements). When the textbook is revised, passages update automatically.

Level 3 — Golden Thread / Deep Transclusion

A compound work references another compound work, which references another. Resolution follows the chain recursively (32 levels). This is Nelson's golden thread — following content back through layers of derivation.

Vision — Nelson's Docuverse

Everything is transclusion. No document copies content — every document is assembled from live references. Authors receive micropayments when content is transcluded. Xudanu implements the technical foundation; the economic layer is future work.

12. What Was Deprecated

The compound edition side-table (compound_editions: HashMap<BeId, CompoundEdition>) is deprecated. Existing side-table data was migrated to inline Transclusion elements via:

$ xudanu-server migrate-compound data
Migrating compound editions to inline transclusion elements...
  work 03ef: 1 spans migrated to inline
  work 03f1: 1 spans migrated to inline
Done: 2 spans migrated across 2 works

The side-table infrastructure (CompoundGetEdition, CompoundSetEdition, CompoundResolveWork, CompoundResolveRecursive, CompoundInsertElement, CompoundRemoveElement, CompoundMoveElement) remains for backward compatibility but receives no new writes. All new transclusion placements go through ElementInsert with Transclusion elements.

FeatureStatusDetails
RangeElement::Transclusion Done Zero-width O-tree element, fingerprinted, serde-serializable
Recursive resolution (32 levels) Done Golden thread with cycle detection, self-transclusion support
Span migration on source edits Done Offsets track content through insertions/deletions
Push-based live updates Done CompoundSourceChanged event to inline + side-table viewers
Editor rendering (read-only spans) Done Amber background, contenteditable=false, getEditableText for deltas
Undo support Done Cmd+Z removes last transclusion via elementRemoveTransclusion
Side-table migration Done CLI: migrate-compound <data-dir>
Compound side-table Deprecated Backward compatible, no new writes
Flatten action (break transclusion) Future Convert Transclusion element to editable Text on user request
Federated compound resolution Future Transclusion elements referencing works on other servers
Bottom line: The O-tree is the single source of truth. Transclusion elements live directly in it. No side-table, no drift, no copies. Resolution is recursive (32 levels). Self-transclusions work. The compound infrastructure is solid and ready for real-world use.