Sync

WebSocket Sync (Real-Time)

Bidirectional real-time sync over WebSockets. Changes pushed immediately to all clients.

// TypeScript/Browser
import { SwirlDB, WebSocketSync } from '@swirldb/js';

const db = await SwirlDB.withIndexedDB('my-app');
const sync = new WebSocketSync('wss://api.example.com/ws');

// Connect to namespace (room/document)
await sync.connect('room-123', db);

// All changes now auto-sync
db.data.messages.push({ from: 'Alice', text: 'Hello!' });

// Changes broadcast to all clients in room-123

// Disconnect when done
await sync.disconnect();

// Rust/Server
use swirldb_server::sync::WebSocketSync;

let sync = WebSocketSync::new("0.0.0.0:3030").await?;

// Server handles:
// - Client connections
// - Change merging
// - Broadcast to all clients
// - Persistent storage via redb

sync.run().await?;

Features:

• Immediate propagation (< 50ms latency)
• Automatic reconnection with exponential backoff
• Binary protocol for minimal bandwidth
• Heartbeat/ping for connection health

Use case: Real-time collaboration, chat, live dashboards, multiplayer games

HTTP Sync (Long-Polling Fallback)

HTTP-based sync with long-polling for environments where WebSockets aren't available. Graceful fallback with similar API.

// TypeScript/Browser
import { SwirlDB, HttpSync } from '@swirldb/js';

const db = await SwirlDB.withLocalStorage('my-app');
const sync = new HttpSync('https://api.example.com/sync');

// Connect to namespace
await sync.connect('room-123', db);

// Push changes
db.data.user.name = 'Alice';
await sync.push();

// Long-poll for new changes (25s timeout)
const changes = await sync.poll();
db.applyChanges(changes);

// Automatic polling loop
sync.startPolling(5000); // Poll every 5s
sync.stopPolling();

HTTP Endpoints:

• POST /sync/connect - Initial connection, returns full state
• POST /sync/push - Push local changes to server
• GET /sync/poll - Long-poll for new changes (25s timeout)

Use case: Corporate firewalls, restrictive networks, serverless deployments

WebRTC Sync (Peer-to-Peer) - Planned

Peer-to-peer sync using WebRTC data channels. Clients discover peers via signaling server.

// TypeScript/Browser (future)
import { SwirlDB, WebRTCSync } from '@swirldb/js';

const db = await SwirlDB.withIndexedDB('my-app');
const sync = new WebRTCSync({
    signaling: 'wss://signal.example.com',
    ice: [{ urls: 'stun:stun.l.google.com:19302' }]
});

// Join P2P room
await sync.join('room-123', db);

// Clients sync directly (no server storage)
db.data.doc.title = 'Collaborative Document';

// Server never sees plaintext data

Status: 🔜 Planned

Use case: Privacy-first apps, local-first collaboration, serverless architectures

Incremental Sync

Only transmit changes since last sync using CRDT heads:

// Client sends current heads with MSG_CONNECT
const heads = db.getHeads();
// heads = [hash1, hash2, ...] - vector clock of last seen changes

// Server computes diff
let changes = namespace.getChangesSince(heads);

// Server sends only new changes
socket.send(MSG_SYNC, changes);

// Client applies incrementally
db.applyChanges(changes);

// Result: Minimal bandwidth, fast sync

Conflict Resolution

CRDTs merge changes automatically using Automerge's built-in rules:

Conflict Type      | Resolution Strategy
───────────────────┼─────────────────────────────────
Text edits         | Operational transform (OT)
Concurrent sets    | Last-write-wins (LWW)
Counter increments | Commutative addition
List inserts       | Position-based merge
Map updates        | Per-key LWW
Deletes            | Tombstone tracking

Example:
  Alice: db.data.count = 5
  Bob:   db.data.count = 10

  After sync:
  - LWW: Highest timestamp wins
  - Both clients converge to same value
  - No "conflict markers" or manual resolution

Real-Time Auto-Sync

Push every change immediately.

const db = await SwirlDB.withIndexedDB('my-app');
const sync = new WebSocketSync('wss://api.example.com/ws');
await sync.connect('room-123', db);

// Enable auto-sync (default)
sync.setAutoSync(true);

// Every mutation triggers sync
db.data.messages.push(newMessage);  // Synced immediately
db.data.user.status = 'online';     // Synced immediately

// Batching with debounce (reduce network traffic)
sync.setDebounce(200); // Wait 200ms before syncing batch

Periodic Sync

Sync on a fixed interval. Reduces network traffic, tolerates brief disconnections.

const sync = new HttpSync('https://api.example.com/sync');
await sync.connect('room-123', db);

// Sync every 10 seconds
setInterval(async () => {
    await sync.push();           // Push local changes
    const changes = await sync.poll(); // Get remote changes
    db.applyChanges(changes);
}, 10000);

// Or use built-in polling
sync.startPolling(10000);

Manual Sync

Explicit sync on user action.

const sync = new HttpSync('https://api.example.com/sync');

// Work offline
db.data.drafts.post1 = { title: 'New Post', content: '...' };
db.data.drafts.post2 = { title: 'Another Post', content: '...' };

// User clicks "Sync Now"
document.querySelector('#sync-btn').addEventListener('click', async () => {
    try {
        await sync.push();
        const changes = await sync.poll();
        db.applyChanges(changes);

        showNotification('✅ Synced successfully');
    } catch (e) {
        showNotification('❌ Sync failed - will retry');
    }
});

Docker Deployment

# Dockerfile
FROM rust:1.70 as builder
WORKDIR /app
COPY . .
RUN cargo build --release --bin swirldb-server

FROM debian:bookworm-slim
COPY --from=builder /app/target/release/swirldb-server /usr/local/bin/
EXPOSE 3030
VOLUME /data
CMD ["swirldb-server"]

# docker-compose.yml
version: '3.8'
services:
  swirldb:
    image: swirldb-server:latest
    ports:
      - "3030:3030"
    volumes:
      - ./data:/data
    environment:
      - RUST_LOG=info
      - SERVER_ID=prod-01

Load Balancing

For high-scale deployments, use sticky sessions (session affinity):

# nginx.conf
upstream swirldb {
    ip_hash;  # Sticky sessions based on client IP
    server sync1.example.com:3030;
    server sync2.example.com:3030;
    server sync3.example.com:3030;
}

server {
    location /ws {
        proxy_pass http://swirldb;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

Alternatively, use shared storage (Redis, Postgres) so any server can handle any client.

Namespace Access Control

Validate namespace access using JWT claims:

// Server-side authorization
async fn handle_connect(
    socket: WebSocket,
    namespace_id: String,
    token: String,
) -> Result<()> {
    // Validate JWT
    let claims = validate_jwt(&token)?;
    let actor = Actor::from_jwt_claims(claims)?;

    // Check namespace access
    if !can_access_namespace(&actor, &namespace_id) {
        return Err("Unauthorized".into());
    }

    // Allow connection
    let namespace = get_or_create_namespace(&namespace_id).await;
    namespace.add_client(socket, actor).await;

    Ok(())
}

fn can_access_namespace(actor: &Actor, namespace_id: &str) -> bool {
    // Example: namespace format "org-{org_id}-{room_id}"
    if let Some(org_id) = &actor.org_id {
        namespace_id.starts_with(&format!("org-{}", org_id))
    } else {
        false
    }
}

End-to-End Encryption

Encrypt CRDT changes before transmission (server never sees plaintext):

import { AesGcmProvider } from '@swirldb/js';

// Shared key (use key exchange in production)
const encryption = await AesGcmProvider.fromPassword('room-password', 'room-123');

// Encrypt changes before sync
const changes = db.getChanges();
const encrypted = await Promise.all(
    changes.map(c => encryption.encrypt(c))
);

// Send encrypted changes
await sync.push(encrypted);

// Server stores encrypted blobs (no access to content)

// Other clients decrypt
const remoteChanges = await sync.poll();
const decrypted = await Promise.all(
    remoteChanges.map(c => encryption.decrypt(c))
);
db.applyChanges(decrypted);

Bandwidth Reduction

• Delta sync: Only transmit changes since last sync (automatic)
• Compression: Use LZ4 or gzip for CRDT changes (future)
• Batching: Debounce rapid changes into single sync operation
• Selective sync: Only sync paths marked with Synced hint

// Reduce bandwidth by 90% with batching
sync.setDebounce(1000); // Batch changes for 1 second

// 100 rapid edits...
for (let i = 0; i < 100; i++) {
    db.data.counter = i;
}

// Result: Single sync with final state, not 100 syncs

Connection Management

• Heartbeat: Ping every 30s to detect dead connections
• Reconnect: Exponential backoff (1s, 2s, 4s, 8s, max 60s)
• Resume sync: Use heads to resume from last successful sync
• Offline queue: Queue changes while offline, sync when reconnected

Scalability

• Namespace sharding: Different namespaces on different servers
• Idle cleanup: Remove namespaces from memory after inactivity
• Lazy loading: Load namespaces on-demand from persistent storage
• Metrics: Track active connections, namespace count, message throughput