A security audit of the multi-server federation layer: trust boundaries, authorization gaps, risk areas, and concrete recommendations to bring federation access control in line with the single-server club model.
The federation layer establishes an encrypted, mutually-authenticated channel between servers using sound cryptography (X25519 + Ed25519 + ChaCha20-Poly1305). Once a peer passes the handshake, however, it operates as a privileged super-user with broad read/write access to the local data store. There is no per-work, per-blob, or per-edition authorization on federation requests — the club-based access control that protects content in the single-server model is bypassed entirely on the federation data plane.
read_club), inject fabricated provenance, create works with no quota, and — through governance gaps — potentially rotate other servers' identity keys. The cryptography prevents outsiders from connecting, but a malicious or compromised peer faces almost no restrictions once connected.
This document maps every trust boundary, catalogs the authorization gaps by severity, and provides concrete recommendations for aligning federation access control with the existing single-server club model.
Seven trust boundaries in the federation stack. Green = cryptographically sound. Red = authorization gaps. Purple = consensus layer with gaps.
The /federation WebSocket route (build_federation_router, federation_handler.rs:184) accepts connections with no middleware, no CSRF guard, no rate limiter, and no IP allowlist. The handler handle_federation_socket performs the handshake inline.
Anyone who can reach the port may open the socket; security rests entirely on the cryptographic handshake (Boundary 2).
| Issue | Impact | Severity |
|---|---|---|
| Handshake frames (Hello/Signature) sent in cleartext before encryption begins | Passive observer learns server identities, public keys, and ephemeral keys | MEDIUM |
| No rate limiting on handshake attempts | Resource-exhaustion DoS: each failed attempt holds a WebSocket + 30s timeout | MEDIUM |
Dialer uses ws:// (plaintext), not wss:// | Outbound handshake metadata exposed on the wire | MEDIUM |
| 64 MiB frame size limit with no per-type caps | Peer can send 64 MiB garbage frames; connection stays alive through parse failures | LOW |
/federation behind TLS (enforce wss://). Add an explicit peer-address allowlist. Rate-limit handshake attempts per IP. Reduce frame size limit to something proportional to expected sync volumes (e.g., 4 MiB).
Three-step handshake producing an encrypted channel:
server_id(my_eph, peer_eph) with its Ed25519 identity key, sends verifying key + kex keyThen federation_is_peer_known (server.rs:11116) gates on the operator-configured known_peer_keys set.
| Issue | Impact | Severity |
|---|---|---|
federation_is_peer_known returns true unconditionally when federation is disabled | If operator forgets to enable federation but still exposes /federation, any valid-signature peer is accepted — and is_peer_known is the ONLY membership gate on the inbound path | HIGH |
known_peer_keys has no expiry or rotation | Compromised keys remain valid indefinitely | MEDIUM |
TrustedServerRegistry / verify_server_identity never wired into federation path | The well-designed identity-verification layer (signed manifests, expiry, constant-time comparison) is unused | HIGH |
Membership (MembershipState) never consulted on inbound handler | A peer in known_peer_keys but ejected via governance still gets full data-plane access | HIGH |
TrustedServerRegistry into the handshake so peer keys are validated against signed manifests with expiry. Consult MembershipState::is_member() after the handshake and before any data-plane access. Default to denying peers when federation is disabled.
This is the most critical area. After the handshake, an authenticated peer can issue read frames with no authorization check beyond "you passed the handshake".
| Frame | What it returns | read_club check? | Severity |
|---|---|---|---|
SyncPull (bulk) | All works + all standalone editions + all blobs | Partial — works filtered to public_club, but editions and blobs are not filtered | CRITICAL |
ContentGet { work_id } | Full edition for a work by ID | None — any work_id returns content | CRITICAL |
BlobGet { hash } | Raw blob bytes + MIME type | None — any hash returns the blob | CRITICAL |
ContentFetch { fingerprint } | Edition or blob by content fingerprint | None — resolves fingerprint to work, no club check | CRITICAL |
TranscludeQuery | Entire transclusion index entries | None — leaks full transclusion graph | HIGH |
CrdtSyncPull { work_ids } | O-tree CRDT state for requested works | Checks server_id == peer_server_id only; work_ids not checked against read_club | CRITICAL |
read_club filter on bulk SyncPull works signals the authors intended club-scoped replication, but the filter is missing on every other read path.
work.read_club against the peer's identity:
// In every federation read handler:
let work = self.works.get(&work_id).ok_or(...)?;
let read_club = work.work.current_edition().read_club()
.unwrap_or(self.system_clubs.public_club);
// Option A: Only replicate public content
if read_club != self.system_clubs.public_club {
return Err(FederationError::NotAuthorized);
}
// Option B: Check against a "federation read club" the peer belongs to
if !peer_clubs.contains(&read_club) {
return Err(FederationError::NotAuthorized);
}
Also filter federation_export_editions and federation_export_blobs the same way federation_export_works already filters.
| Frame | Effect | Auth check | Severity |
|---|---|---|---|
SyncPush | Creates new public works from pushed editions | None — any peer mints arbitrary public works, no quota | CRITICAL |
CrdtSyncPush | Modifies collaborative editing buffer for any work_id; imports span_provenance | Checks server_id == peer_server_id only; no edit-rights check; provenance imported verbatim | CRITICAL |
StateSyncPush | Merges foreign ReconcileState; can move "current edition" pointer | None — peer can change which edition is presented as current | HIGH |
SyncPush blobs | Imports blobs into blob store | BLAKE3 hash verified, but any peer can inject blobs with no quota | MEDIUM |
SpanProvenance on SyncWorkEntry and CrdtWorkUpdate is imported verbatim with no signature verification. A peer can attach fabricated provenance — claiming a span originated from any author or server — to locally-stored content. Since provenance is the integrity spine of the system, this undermines the trustworthiness of all federated content.
ReconcileState.current uses last-writer-wins by (timestamp, server_id). The timestamp is clamped to now + 3600s, but a peer that sets a near-future timestamp wins the "current" pointer for any work, effectively censoring other editions from the default view.
// Before importing span_provenance from a peer:
for sp in &update.span_provenance {
let element_fingerprints = collect_fingerprints(&edition, sp.start, sp.end);
if !verify_span_provenance(&sp.provenance, &element_fingerprints) {
tracing::warn!("peer {} sent invalid provenance, rejecting", peer_server_id);
continue; // skip this span's provenance
}
// Optionally: verify that provenance.author_public_key corresponds
// to the origin server's known signing key
}
Also: add edit-rights checks on CrdtSyncPush (peer must be in the work's edit_club), add quotas on SyncPush work creation, and clamp LWW timestamps to now (not now + 1h).
MembershipState is an OR-set CRDT. Join requires ≥ min_endorsements valid endorsements from existing members. The default min_endorsements = 2.
| Issue | Impact | Severity |
|---|---|---|
Membership OR-set merges are unauthenticated (MembershipSyncPush) | A peer can inject membership entries for arbitrary server_ids | CRITICAL |
| Endorser verification keys read from the merge result | Privilege insertion loop: inject member → forge their endorsement → satisfy min_endorsements for further joins | CRITICAL |
| Bootstrap mode accepts joins with ZERO endorsements | If left on, any peer that passes the key check joins with no endorsement | MEDIUM |
| Membership not checked on inbound data plane | Ejected peers retain full data access | HIGH |
TrustedServerRegistry (signed manifests), not from the CRDT merge result. Membership sync should authenticate each entry against the registry before merging. This breaks the privilege-insertion loop because the attacker cannot forge registry signatures.
Lightweight PBFT: Proposal → Prepare → Commit → Sealed. Transactions: Admit, Expel, KeyRegister, RoyaltyRecord.
| Issue | Impact | Severity |
|---|---|---|
| Sealed batches not re-validated on receipt | Receiver checks only proposer_id + sequence; does not verify quorum or vote signatures | CRITICAL |
| PBFT votes carry no signatures | A Byzantine leader can fabricate a SealedBatch with fake vote list | CRITICAL |
KeyRegister replaces any member's keys with no old-key proof | Combined with above, a leader can rotate all victim keys to attacker-controlled keys | CRITICAL |
Default cluster_size = 1 → quorum of 1 | Single-node "consensus" — leader alone reaches both prepare and commit quorum | HIGH |
Non-leader can push GovernancePrePrepare | Receiver prepare-votes for it without checking proposer is the current leader | MEDIUM |
RoyaltyRecord has no upper bound | Malicious peer can pollute the royalty ledger with arbitrary amounts | MEDIUM |
PbftVote should include an Ed25519 signature over (view, seq, voter_id, phase). The receiver verifies each vote's signature against the voter's registry-pinned key.GovernanceSealed, count distinct valid prepare + commit votes; reject if either count is below quorum_size.KeyRegister: The transaction must include a signature from the old key authorizing the rotation.cluster_size = 3 (minimum for meaningful PBFT). Refuse to start with quorum-of-1 in production mode.
The authenticated principal is the Ed25519 verifying key (from the handshake), but authorization decisions throughout the frame handlers key off the self-declared server_id string from the Hello frame. There is no binding between the two.
server_id — including one belonging to another serverorigin_server_id from the push payload (attacker-controlled)MembershipLeave trusts the self-declared server_id to evictCrdtSyncPush/Pull "authorization" only checks the frame's server_id matches the declared hello ID — both are unverifiedTrustedServerRegistry in server_identity.rs already implements exactly the right mechanism: signed attestations binding server_id ↔ verifying_key with expiry and constant-time comparison. verify_server_identity() should be called during the handshake, after Hello, to verify that the declared server_id matches the authenticated verifying key. If the registry has no entry for this binding, the connection is refused.
The single-server model has a coherent, layered access control system. The federation data plane bypasses every layer:
| Access control mechanism | Single-server (end-user) | Federation (peer server) |
|---|---|---|
| Authentication | Password (Argon2id) or Ed25519 key + rate limiting (10 attempts / 5 min) | X25519 + Ed25519 handshake (sound crypto) |
| read_club enforcement | Every read path checks session.has_authority(read_club) | BYPASSED on ContentGet, BlobGet, ContentFetch, CrdtSyncPull |
| edit_club enforcement | Every write path checks session.has_authority(edit_club) | BYPASSED on SyncPush, CrdtSyncPush, StateSyncPush |
| Provenance verification | verify_span_provenance() checked on display; signatures verified against author keys | Imported verbatim, no verification |
| Rate limiting | Per-club login limit + per-session/per-IP sliding windows | None on data plane |
| Quota | Personal club limit (max_personal_clubs) | None — unlimited work creation via SyncPush |
| Audit trail | SecurityMonitor + tamper-evident security.log + attribution.log | Federation operations not audited |
| CSRF protection | Token-based on WS upgrade | N/A (different protocol) |
The end-user path enforces 6 layers of access control. The federation path enforces 1 (the handshake) and skips the rest.
| # | Gap | Location | Impact | Severity |
|---|---|---|---|---|
| C1 | ContentGet/ContentFetch/BlobGet/TranscludeQuery ignore read_club | server.rs:11493, 11504, 11570, 11514 | Private content read by any peer | CRITICAL |
| C2 | federation_export_editions / federation_export_blobs unfiltered | server.rs:11216, 11341 | Bulk exfiltration of all editions + blobs | CRITICAL |
| C3 | CrdtSyncPull work_ids not club-checked | server.rs:11414 | Private work CRDT state exposed | CRITICAL |
| W1 | SyncPush/CrdtSyncPush import content + provenance with no auth | server.rs:11231, 11437 | Content & provenance injection | CRITICAL |
| W2 | StateSyncPush LWW hijack of "current" edition | server.rs:11683 | Edition censorship/swap | HIGH |
| M1 | Membership OR-set merge unauthenticated; endorser keys from merge result | federation.rs:1698, server.rs:11987 | Privilege insertion loop | CRITICAL |
| G1 | Sealed PBFT batches not re-validated; votes unsigned | federation_handler.rs:778 | Forged governance execution | CRITICAL |
| G2 | KeyRegister needs no old-key proof | server.rs:12138 | Key theft via governance | CRITICAL |
| G3 | Default cluster_size=1 → quorum of 1 | federation.rs:234 | Single-node "consensus" | HIGH |
| A1 | server_id not bound to verifying key on federation path | federation_handler.rs:439 | Identity spoofing | HIGH |
| A2 | federation_is_peer_known returns true when federation disabled | server.rs:11117 | Open peer acceptance if misconfigured | HIGH |
| A3 | TrustedServerRegistry / verify_server_identity never wired in | server_identity.rs:290 | Unused strong identity layer | HIGH |
| T1 | Handshake frames in cleartext; ws:// not wss:// | federation_handler.rs:225, federation_active.rs:145 | Metadata leakage to passive observer | MEDIUM |
| T2 | No rate limiting on handshake; 64 MiB frames | federation_handler.rs:184, 463 | DoS via resource exhaustion | MEDIUM |
The goal is to make federation access control at least as restrictive as end-user access control. Here's the roadmap:
federation_is_peer_known to return false when federation mode is disabled. Currently it returns true, allowing any key-valid peer in.
read_club == public_club filter that federation_export_works already uses to federation_export_editions and federation_export_blobs. This is a few-line fix that closes the bulk exfiltration path.
GovernanceState::new(1) to GovernanceState::new(3) and add a runtime warning if cluster_size < 3 in production mode.
Introduce a FederationAuthorization struct that wraps the peer's identity and resolved clubs:
struct FederationAuthorization {
peer_server_id: String,
peer_verifying_key: [u8; 32],
resolved_clubs: HashSet<BeId>, // clubs this peer belongs to
}
impl FederationAuthorization {
fn can_read(&self, work: &WorkState, server: &Server) -> bool {
let read_club = work.read_club()
.unwrap_or(server.system_clubs.public_club);
read_club == server.system_clubs.public_club
|| self.resolved_clubs.contains(&read_club)
}
fn can_edit(&self, work: &WorkState, server: &Server) -> bool {
let edit_club = work.edit_club()
.unwrap_or(server.system_clubs.empty_club);
self.resolved_clubs.contains(&edit_club)
}
}
Check auth.can_read() before every read handler and auth.can_edit() before every write handler. This mirrors the single-server pattern exactly.
Before accepting span_provenance from a peer, call verify_span_provenance() to confirm the Ed25519 signature is valid for the claimed content. Reject invalid provenance rather than importing it verbatim. Optionally verify that the author_public_key corresponds to the origin server's registered signing key.
After Hello and before Signature, verify the declared server_id against the peer's authenticated verifying key using verify_server_identity(). Refuse the connection if no valid registry entry exists. This binds the self-declared identity to the cryptographic identity.
signature: [u8; 64] to PbftVote, signed over (view, seq, voter_id, phase) with the voter's keyGovernanceSealed, verify every embedded vote's signature and count distinct valid votes per phase; reject if below quorum_sizeKeyRegister transactions to include a signature from the old keyEndorser verification keys must come from the TrustedServerRegistry, not from the CRDT merge result. This breaks the privilege-insertion loop because the attacker cannot forge registry signatures.
security.log chain (currently un-audited)SyncPush work creation rate per peer (prevents storage exhaustion)applied_sequences across restarts to prevent governance replay after reconnectionwss:// for outbound dialing; refuse ws:// in productionSecurityMonitorReconcileState timestamps to now (not now + 3600s)Until the gaps in Phase 1 and 2 are fixed, federation should be deployed with these constraints:
| Pattern | Guidance |
|---|---|
| Network isolation | Bind /federation to a private network interface or VPN. Do not expose it to the public internet. |
| Shared content only | If you want to keep private works private, don't enable federation yet. The SyncPull works filter is partial; direct-fetch paths have no filter. |
| Disable federation by default | Only enable --peer / --federation-mode when you intentionally want to federate. Ensure federation_is_peer_known returns false when disabled (Fix A2). |
| Minimize peer count | Fewer peers = smaller attack surface. Each peer is a potential exfiltration and injection point. |
Audit known_peer_keys | Review the key list regularly. Remove keys for decommissioned peers immediately. |
| Don't rely on governance | The PBFT layer has gaps (G1-G3). Don't rely on it for security-critical decisions until Phase 3 fixes are in. |