PhoenixKitSync

Peer-to-peer data sync module for PhoenixKit.

Provides bidirectional data synchronization between PhoenixKit instances — sync between dev and prod, dev and dev, or different websites entirely.

Installation

Add to your parent app's mix.exs:

{:phoenix_kit_sync, "~> 0.1"}

The module is auto-discovered via PhoenixKit's beam scanning — no additional configuration needed. Enable it from the admin dashboard under Modules.

Architecture

Automatic Cross-Site Registration

The module uses permanent connections with automatic cross-site registration:

1. Sender creates a connection pointing to a remote site's URL
2. System automatically notifies the remote site via API
3. Remote site registers the connection based on their incoming settings:
   • Auto Accept: Connection activates immediately
   • Require Approval: Connection appears as pending
   • Require Password: Only accepts with correct password
   • Deny All: Rejects the connection request
4. Both sites have matching connection records for data sync
5. All transfers are tracked in the history with full audit trail

Connection Types

• Sender: "I allow this remote site to pull data from me"
• Receiver: "I can pull data from this remote site" (auto-created via API)

When you create a sender connection, the remote site automatically receives a corresponding receiver connection.

Features

• Ephemeral Code-Based Transfers: One-time manual sync with 8-character secure codes
• Permanent Token-Based Connections: Recurring sync with full access controls
• Token-Based Authentication: Secure tokens, hashed in database, shown only once
• Approval Modes: Auto-approve, require approval, or per-table approval
• Access Controls: Allowed/excluded tables, download limits, record limits
• Security Features: IP whitelist, time-of-day restrictions, expiration dates
• Conflict Resolution: skip, overwrite, merge, append strategies
• Transfer Tracking: Full history with statistics and approval workflow
• Audit Trail: Track who created, approved, suspended, or revoked connections
• Real-Time Progress: Live tracking of sync operations
• Background Import: Async processing via Oban workers
• Cross-Site Protocol: HTTP API + WebSocket for data transfer
• Activity Audit: Every connection mutation (create, approve, suspend, revoke, reactivate, delete, token rotate) and every transfer mutation (create, complete, fail, cancel, approve, deny) writes a sync.<resource>.<verb> row to phoenix_kit_activities with the actor's UUID — visible in the core admin Activity feed
• Centralised error translation: Context functions return {:error, :atom} tuples; UI / API layers translate via PhoenixKitSync.Errors.message/1, which dispatches to gettext-wrapped strings (translations live in core phoenix_kit)

Connection Settings

Sender-Side Controls

Connection Statuses

Incoming Connection Settings

Control how your site handles connection requests from other sites:

Workflow

Setting Up a Sender Connection

1. Navigate to the Sync connections page in the admin dashboard
2. Click "New Connection"
3. Enter a name and the remote site's URL
4. Configure access controls (approval mode, tables, limits)
5. Save — the connection is created and token generated
6. The remote site is notified automatically!
   • If successful, the connection appears in their list
   • Based on their settings, it may be auto-approved or pending
7. If notification fails, share the token manually as a fallback

What Happens on the Remote Site

When you create a sender connection:

• Your site calls POST {remote_url}/{prefix}/sync/api/register-connection
• The remote site creates a matching receiver connection
• Based on their incoming mode:
  • Auto Accept: Ready to use immediately
  • Require Approval: Admin must approve in their connections list
  • Require Password: You need to provide their password
  • Deny All: Connection is rejected

Programmatic API

Connection Management

alias PhoenixKitSync.Connections

# Create a sender connection. The raw token is returned in the third
# tuple element — it's only available at creation time; afterwards only
# the SHA-256 hash is stored.
{:ok, connection, token} =
  Connections.create_connection(%{
    "name" => "Production Backup",
    "direction" => "sender",
    "site_url" => "https://backup.example.com",
    "approval_mode" => "auto_approve",
    "allowed_tables" => ["users", "posts"],
    "max_downloads" => 100,
    "created_by_uuid" => current_user.uuid
  })

# Approve a pending connection (admin_user_uuid is recorded as the actor
# on the resulting `sync.connection.approved` activity-log row).
{:ok, connection} = Connections.approve_connection(connection, admin_user_uuid)

# Suspend a connection
{:ok, connection} = Connections.suspend_connection(connection, admin_user_uuid, "Security audit")

# Reactivate a suspended connection. Pass `actor_uuid:` in opts so the
# activity log records who reactivated it.
{:ok, connection} = Connections.reactivate_connection(connection, actor_uuid: admin_user_uuid)

# Revoke permanently
{:ok, connection} = Connections.revoke_connection(connection, admin_user_uuid, "No longer needed")

# Rotate the auth token (returns the new raw token, hash stored in DB).
# Logged as `sync.connection.token_regenerated`.
{:ok, connection, new_token} = Connections.regenerate_token(connection, actor_uuid: admin_user_uuid)

# Validate a token (used by receiver when connecting)
case Connections.validate_connection(token, client_ip) do
  {:ok, connection} -> # Token is valid, connection is active
  {:error, :invalid_token} -> # Token doesn't exist
  {:error, :connection_expired} -> # Expired or revoked
  {:error, :download_limit_reached} -> # Max downloads reached
  {:error, :ip_not_allowed} -> # IP not in whitelist
  {:error, :outside_allowed_hours} -> # Outside time window
end

Transfer Tracking

alias PhoenixKitSync.Transfers

# Record a transfer
{:ok, transfer} = Transfers.create_transfer(%{
  direction: "send",
  connection_uuid: connection.uuid,
  table_name: "users",
  records_transferred: 150,
  bytes_transferred: 45000,
  status: "completed"
})

# Get transfer history
transfers = Transfers.list_transfers(
  connection_uuid: connection.uuid,
  direction: "send",
  status: "completed"
)

# Get statistics for a connection
stats = Transfers.connection_stats(connection.uuid)
# => %{total_transfers: 25, total_records: 5000, total_bytes: 1500000}

System Control

# Enable/disable
PhoenixKitSync.enabled?()
PhoenixKitSync.enable_system()
PhoenixKitSync.disable_system()
PhoenixKitSync.get_config()

# Local database inspection
{:ok, tables} = PhoenixKitSync.list_tables()
{:ok, schema} = PhoenixKitSync.get_schema("users")
{:ok, records} = PhoenixKitSync.export_records("users", limit: 100)

# Import with conflict strategy
{:ok, result} = PhoenixKitSync.import_records("users", records, :skip)

Remote Client

alias PhoenixKitSync.Client

{:ok, client} = Client.connect("https://sender.com", "ABC12345")
{:ok, tables} = Client.list_tables(client)
{:ok, result} = Client.transfer(client, "users", strategy: :skip)
Client.disconnect(client)

API Endpoints

Cross-site communication endpoints (under the configured URL prefix):

• POST /sync/api/register-connection — Register incoming connection
• POST /sync/api/delete-connection — Delete a connection
• POST /sync/api/verify-connection — Verify connection token
• POST /sync/api/update-status — Update connection status
• POST /sync/api/get-connection-status — Query connection status
• POST /sync/api/list-tables — List available tables
• POST /sync/api/pull-data — Pull table data
• POST /sync/api/table-schema — Get table schema
• POST /sync/api/table-records — Get table records
• GET /sync/api/status — Check module status

Database

Table migrations are currently managed by PhoenixKit's core migration system. If the tables don't already exist, this package can create them automatically via PhoenixKitSync.Migration.up/0.

See docs/table_structure.md in the source repository for full schema documentation.

Future Plans

Auto-Sync Scheduling (Planned)

Connections have fields for auto-sync but the scheduler isn't implemented yet:

• auto_sync_enabled: Enable automatic periodic sync
• auto_sync_tables: Tables to sync automatically
• auto_sync_interval_minutes: How often to sync

Security Considerations

• Token Security: Tokens are SHA-256 hashed in the database and only shown once on creation; comparisons use Plug.Crypto.secure_compare/2 to defeat timing attacks
• Per-connection table authorization: Even with a valid token, every API endpoint that returns or accepts table data (list_tables, pull_data, table_schema, table_records) filters through the connection's excluded_tables blocklist and allowed_tables allowlist
• Parameterised SQL with identifier validation: Every dynamic table / column name interpolated into raw SQL passes through SchemaInspector.valid_identifier?/1 first; values always go through $N parameterised binds
• Optional Password: Additional password can be required per-transfer; refuses registration outright when incoming_mode = "require_password" but no password is configured (no silent bypass)
• IP Whitelisting: Restrict connections to specific IP addresses
• Time Restrictions: Allow connections only during specific hours
• Rate Limiting: Per-connection download / record limits enforced by Connections.validate_connection/2
• Audit Trail: Every state-changing operation lands a row in phoenix_kit_activities with actor_uuid, resource_type, resource_uuid, and a PII-safe metadata subset (no site_url, no auth_token_hash)

Troubleshooting

Connection Issues

1. Verify the token is correct and hasn't been regenerated
2. Check connection status is "active"
3. Verify IP is in whitelist (if configured)
4. Check time-of-day restrictions
5. Verify download/record limits haven't been exceeded

Transfer Failures

1. Check transfer history for error messages
2. Verify table is in allowed tables (if configured)
3. Check approval status if approval mode is enabled
4. Review server logs for detailed errors