Snapshots Architecture
Phony Snapshots provide point-in-time captures of synchronized databases, enabling instant rollback, environment provisioning, and data versioning.
Architecture Overview
┌─────────────────────────────────────────────────────────────────────────┐
│ SNAPSHOTS ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐│
│ │ SNAPSHOT MANAGER (Go) ││
│ │ ││
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ││
│ │ │ Capture │ │ Storage │ │ Restore │ ││
│ │ │ Engine │ │ Manager │ │ Engine │ ││
│ │ │ │ │ │ │ │ ││
│ │ │ • Stream │ │ • S3/GCS │ │ • Parallel │ ││
│ │ │ • Compress │ │ • Encrypt │ │ • Verify │ ││
│ │ │ • Chunk │ │ • Version │ │ • Rollback │ ││
│ │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ ││
│ │ │ │ │ ││
│ │ └────────────────┼────────────────┘ ││
│ │ │ ││
│ │ ▼ ││
│ │ ┌─────────────┐ ││
│ │ │ Metadata │ ││
│ │ │ Store (PG) │ ││
│ │ └─────────────┘ ││
│ │ ││
│ └─────────────────────────────────────────────────────────────────────┘│
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────┐│
│ │ OBJECT STORAGE ││
│ │ ││
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ││
│ │ │ Primary │ │ Replica │ │ Archive │ ││
│ │ │ (S3) │──│ (S3 Cross- │──│ (Glacier) │ ││
│ │ │ │ │ Region) │ │ │ ││
│ │ │ Hot │ │ Warm │ │ Cold │ ││
│ │ │ < 30 days │ │ 30-90 days │ │ > 90 days │ ││
│ │ └─────────────┘ └─────────────┘ └─────────────┘ ││
│ │ ││
│ └─────────────────────────────────────────────────────────────────────┘│
│ │
└─────────────────────────────────────────────────────────────────────────┘Snapshot Types
Full Snapshot
Complete database capture:
┌─────────────────────────────────────────────────────────────────────────┐
│ FULL SNAPSHOT │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ Database Snapshot │
│ ════════ ════════ │
│ │
│ ┌─────────────────┐ ┌─────────────────────────────────────┐│
│ │ users (500MB) │──────────►│ snapshot_20260131_100000.tar.gz ││
│ │ orders (2GB) │ Stream │ ││
│ │ products (100MB)│ + Gzip │ Contents: ││
│ │ logs (5GB) │ │ ├── manifest.json (metadata) ││
│ └─────────────────┘ │ ├── schema.sql (DDL) ││
│ │ ├── users.csv.gz (500MB → 80MB) ││
│ Total: 7.6 GB │ ├── orders.csv.gz (2GB → 300MB) ││
│ │ ├── products.csv.gz (100MB → 15MB) ││
│ │ └── logs.csv.gz (5GB → 600MB) ││
│ │ ││
│ │ Compressed: ~1 GB ││
│ └─────────────────────────────────────┘│
│ │
└─────────────────────────────────────────────────────────────────────────┘Incremental Snapshot
Only changes since last snapshot:
┌─────────────────────────────────────────────────────────────────────────┐
│ INCREMENTAL SNAPSHOT │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ Base Snapshot Incremental │
│ (Day 1) (Day 2) │
│ ═══════════ ═══════════ │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Full DB dump │ │ Only changes: │ │
│ │ 7.6 GB │ │ • 50 new users │ │
│ │ Compressed: 1GB │ │ • 200 new orders│ │
│ └─────────────────┘ │ • 10 updates │ │
│ │ │ │
│ │ Size: 5 MB │ │
│ └─────────────────┘ │
│ │
│ RESTORE PROCESS: │
│ ════════════════ │
│ 1. Restore base snapshot │
│ 2. Apply incremental changes in order │
│ 3. Verify checksums │
│ │
└─────────────────────────────────────────────────────────────────────────┘Storage Architecture
Chunked Storage
Large snapshots are split into chunks for parallel upload/download:
json
{
"snapshot_id": "snap_abc123",
"chunks": [
{
"index": 0,
"table": "users",
"offset": 0,
"size": 104857600,
"checksum": "sha256:abc123...",
"path": "s3://phony-snapshots/org_xyz/snap_abc123/chunk_000.gz"
},
{
"index": 1,
"table": "users",
"offset": 104857600,
"size": 104857600,
"checksum": "sha256:def456...",
"path": "s3://phony-snapshots/org_xyz/snap_abc123/chunk_001.gz"
}
]
}Encryption
All snapshots are encrypted at rest:
┌─────────────────────────────────────────────────────────────────────────┐
│ ENCRYPTION MODEL │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ 1. ENVELOPE ENCRYPTION │
│ ═══════════════════════ │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Master Key │ │ Data Key │ │ Snapshot │ │
│ │ (AWS KMS) │───►│ (per snapshot) │───►│ Chunks │ │
│ │ │ │ │ │ │ │
│ │ Never leaves │ │ Encrypted with │ │ Encrypted with │ │
│ │ KMS │ │ master key │ │ data key │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
│ │
│ 2. KEY HIERARCHY │
│ ═════════════════ │
│ │
│ Organization Master Key (KMS) │
│ │ │
│ ├── Snapshot Key (snap_001) ─► Encrypted chunks │
│ ├── Snapshot Key (snap_002) ─► Encrypted chunks │
│ └── Snapshot Key (snap_003) ─► Encrypted chunks │
│ │
│ BENEFIT: Rotate org key without re-encrypting all snapshots │
│ │
└─────────────────────────────────────────────────────────────────────────┘Lifecycle Management
Retention Policies
json
{
"retention": {
"policy": "tiered",
"rules": [
{
"age": "< 7 days",
"keep": "all",
"storage_class": "STANDARD"
},
{
"age": "7-30 days",
"keep": "daily",
"storage_class": "STANDARD_IA"
},
{
"age": "30-90 days",
"keep": "weekly",
"storage_class": "GLACIER_IR"
},
{
"age": "> 90 days",
"keep": "monthly",
"storage_class": "DEEP_ARCHIVE"
}
],
"minimum_retention": "30 days",
"legal_hold": false
}
}Storage Tiers
| Tier | Access Time | Cost | Use Case |
|---|---|---|---|
| Standard | Instant | $$$ | Recent snapshots (< 7 days) |
| Standard-IA | Instant | $$ | Infrequent access (7-30 days) |
| Glacier IR | Minutes | $ | Compliance (30-90 days) |
| Deep Archive | Hours | ¢ | Long-term (> 90 days) |
Restore Operations
Full Restore
┌─────────────────────────────────────────────────────────────────────────┐
│ FULL RESTORE PROCESS │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ 1. PREPARE │
│ ══════════ │
│ • Validate snapshot integrity (checksums) │
│ • Verify target database connection │
│ • Check available disk space │
│ • Acquire exclusive lock on target │
│ │
│ 2. RESTORE SCHEMA │
│ ═════════════════ │
│ • Drop existing tables (if overwrite=true) │
│ • Execute DDL from schema.sql │
│ • Disable foreign key constraints │
│ • Disable triggers │
│ │
│ 3. RESTORE DATA │
│ ════════════════ │
│ • Download chunks in parallel (8 concurrent) │
│ • Decompress and decrypt │
│ • Bulk load into tables (COPY command) │
│ • Order: FK dependencies respected │
│ │
│ 4. FINALIZE │
│ ═══════════ │
│ • Enable foreign key constraints │
│ • Enable triggers │
│ • Rebuild indexes │
│ • Update statistics (ANALYZE) │
│ • Verify row counts │
│ │
└─────────────────────────────────────────────────────────────────────────┘Table-Level Restore
Restore specific tables only:
bash
phony snapshot restore snap_abc123 \
--target staging \
--tables users,orders \
--mode merge # or 'replace'Point-in-Time Restore
Using incremental snapshots:
bash
phony snapshot restore \
--target staging \
--point-in-time "2026-01-30T15:30:00Z"
# System finds:
# 1. Base snapshot before timestamp
# 2. Incrementals to apply
# 3. Restores to exact pointComparison Modes
Diff Between Snapshots
bash
phony snapshot diff snap_001 snap_002
# Output:
# Table: users
# Added: 50 rows
# Modified: 12 rows
# Deleted: 3 rows
#
# Table: orders
# Added: 200 rows
# Modified: 0 rows
# Deleted: 0 rowsSchema Drift Detection
bash
phony snapshot schema-diff snap_001 current
# Output:
# Table: users
# + Column: phone_verified (boolean)
# ~ Column: email (varchar(100) → varchar(255))
#
# Table: audit_logs
# + New tableAutomation
Scheduled Snapshots
json
{
"schedule": {
"full": {
"cron": "0 2 * * 0",
"description": "Weekly full snapshot on Sunday 2 AM"
},
"incremental": {
"cron": "0 2 * * 1-6",
"description": "Daily incremental Mon-Sat 2 AM"
}
},
"triggers": {
"before_sync": true,
"before_migration": true
}
}Event-Driven Snapshots
json
{
"webhooks": {
"on_create": "https://yourapp.com/webhook/snapshot-created",
"on_restore_start": "https://yourapp.com/webhook/restore-started",
"on_restore_complete": "https://yourapp.com/webhook/restore-completed"
}
}Performance
| Operation | 1 GB | 10 GB | 100 GB | 1 TB |
|---|---|---|---|---|
| Create (full) | ~1 min | ~5 min | ~30 min | ~4 hours |
| Create (incremental) | ~10 sec | ~30 sec | ~2 min | ~15 min |
| Restore (Standard) | ~2 min | ~10 min | ~1 hour | ~8 hours |
| Restore (Glacier) | +5 min | +5 min | +10 min | +30 min |
Optimization factors:
- Parallel chunk processing (up to 16 workers)
- Compression ratio (typically 6-10x for text data)
- Network bandwidth to/from storage
- Target database write speed
Tier Limits
| Feature | FREE | STARTER | TEAM | BUSINESS |
|---|---|---|---|---|
| Snapshots | 3 | 10 | 50 | Unlimited |
| Retention | 7 days | 30 days | 90 days | Custom |
| Max size | 1 GB | 10 GB | 100 GB | Unlimited |
| Cross-region | - | - | 1 region | 3 regions |
| Incremental | - | - | ✓ | ✓ |
| Point-in-time | - | - | - | ✓ |