Execution Model
Phony packages can be executed on multiple runtimes with different performance characteristics. This document explains the execution architecture.
Overview
┌─────────────────────────────────────────────────────────────────────────┐
│ UNIFIED EXECUTION: Same Package, Different Engines │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ Phony Package │
│ (git repo · phony.json · versioned) │
│ │ │
│ ┌───────────────┼───────────────┐ │
│ ▼ ▼ ▼ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ CLI │ │ OSS │ │ Cloud │ │
│ │ (Rust) │ │Libraries │ │ (Rust) │ │
│ │ │ │ │ │ │ │
│ │ Training │ │ PHP/Py/ │ │ Nuxt+Go+ │ │
│ │ + local │ │ JS │ │ Rust │ │
│ │ generate │ │ │ │ │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ │ │ │ │
│ │ │ │ │
│ Built today Planned Target design │
│ Train + generate 10-50K/sec 5M/sec │
│ Free (MIT) Free (MIT) Paid │
│ │ │ │
│ └───────┬───────┘ │
│ ▼ │
│ Same Output! │
│ (Deterministic with seed) │
│ │
└─────────────────────────────────────────────────────────────────────────┘Runtime Comparison
| Feature | CLI (Rust) — built | OSS Libraries — planned | Cloud (Rust) — target |
|---|---|---|---|
| Purpose | Training + local generation | Generation only | Training + Generation + DB sync |
| Training Speed | Parallel (rayon) + streaming builder | N/A | DB-orchestrated streaming |
| Generation Speed | local (Rust) | 10-50K/sec | 5M/sec |
| Languages | Binary | PHP, Python, JS | API + CLI trigger |
| Cost | Free | Free | Paid |
| License | MIT | MIT | Proprietary |
| Offline | Yes | Yes | No |
| Advanced Optimizations | No | No | Yes (not shared) |
| DB Sync | No | No | Yes |
| Mock API | No | No | Yes |
| Snapshots | No | No | Yes |
Key Architecture Decision: Generation is free and unlimited at every tier — the paid Cloud value is orchestration, PII visibility, and compliance reporting around production-data sync, not generation itself. The Cloud runtime's parallel throughput (5M rec/s) and training optimizations remain proprietary capabilities of the platform, but they are never a meter or paywall on generation.
CLI Runtime (Rust)
The CLI handles training and local generation — it creates .ngram models (phony train) and generates values from any generator or PGDL definition (phony generate), plus the git-based package-manager commands.
Architecture
┌─────────────────────────────────────────────────────────────────────────┐
│ CLI ARCHITECTURE (as built) │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐│
│ │ phony CLI ││
│ ├─────────────────────────────────────────────────────────────────────┤│
│ │ Commands: ││
│ │ ├── phony train Train an N-gram model from input data ││
│ │ ├── phony info Display model metadata ││
│ │ ├── phony validate Validate a .ngram file (exit code 4 on fail) ││
│ │ ├── phony stats Show model statistics ││
│ │ ├── phony generate Generate values from a generator/PGDL def ││
│ │ ├── phony install Resolve + fetch deps, write phony.lock ││
│ │ ├── phony add/remove Manage dependencies in phony.json ││
│ │ ├── phony update Re-resolve dependencies (all or one) ││
│ │ └── phony list List the resolved dependency closure ││
│ ├─────────────────────────────────────────────────────────────────────┤│
│ │ ┌───────────────┐ ┌────────────────┐ ┌───────────────┐ ││
│ │ │ ngram-core │ │ phony-pgdl │ │ pkg (store) │ ││
│ │ │ train/save/ │ │ registry + │ │ git fetch, │ ││
│ │ │ load/generate│ │ compile + │ │ lockfile, │ ││
│ │ │ (binary v3) │ │ PEL + assets │ │ blob cache │ ││
│ │ └───────────────┘ └────────────────┘ └───────────────┘ ││
│ └─────────────────────────────────────────────────────────────────────┘│
│ │
│ Output: .ngram model files, generated JSON/JSONL │
└─────────────────────────────────────────────────────────────────────────┘Commands (built)
# Training (one item per line; whole file for --token-type text)
phony train names.txt -o models/names.ngram \
-n 3 # n-gram order 2-5 (default 3)
phony train prose.txt -t text -o models/prose.ngram --locale tr_TR \
--min-count 2 # prune rare transitions (privacy + size)
# tokenizer overrides: --min-word-length, --lowercase, --word-filter REGEX
# Model inspection / validation
phony info models/names.ngram
phony stats models/names.ngram
phony validate models/names.ngram # invalid → exit code 4
# Generation (deterministic from --seed)
phony generate def.json --seed 42 --count 100 -f jsonl
phony generate --use "@phony/core:logic.uuid_v7" --seed 42 -n 10
phony generate --use "@acme/demo:employee_email" \
--package ./demo --locale "tr_TR,*" \
--reference-time 1735689600 # freeze now()/today()/relative dates
# --manifest DIR auto-loads a project's phony.json closure; --frozen = no fetch
# Package manager (git-based)
phony install # resolve + fetch closure, write phony.lock
phony add github.com/acme/phony-demo@1.2.0
phony remove "@acme/demo"
phony update # re-resolve, ignoring the lock
phony list --jsonTarget design (not built): phony login and the cloud control-plane command family (phony sync, phony snapshot, phony mock deploy). There is no publish — packages are plain git repos (see the Package Manager).
OSS Libraries Runtime (planned)
Native-language runtimes so applications can generate data in-process.
Implementation status
Nothing has shipped yet. What follows is the current plan. The portable contract the runtimes implement against — SplitMix64, derive_seed, the binary .ngram format, sorted vocab + ascending-id edge ordering — is built and specified by the Rust engine (phony-core/FORMAT.md and the crate docs).
Determinism strategy (the portable contract)
Every runtime implements SplitMix64 and derive_seed exactly as specified — a few dozen lines of integer arithmetic per language. Native RNGs (array_rand, random.choice, Math.random) are ruled out: they can't reproduce the engine's streams, and Math.random cannot even be seeded. A shared conformance vector suite (seed → expected outputs, per generator and per model fixture) gates every runtime release.
Distribution strategy (one port, native bindings elsewhere)
| Runtime | Strategy | Status |
|---|---|---|
| PHP | One hand-written pure-PHP port of the generation engine (no extension required — Composer-installable everywhere), validated against the conformance vectors | Planned |
| Python | Native binding: PyO3 wheel wrapping the Rust core | Planned |
| JS/TS | WASM build of the Rust core (Node + browser) | Planned |
| Ruby | Deferred | Deferred |
PHP is the one full reimplementation because it is the weakest FFI/WASM host of the target languages; everywhere else the Rust core itself is the runtime, so determinism is inherited rather than re-proven.
API sketches (target design)
The intended developer experience (entity-level APIs assume the cloud-side schema layer):
use Phonyland\Phony;
$phony = new Phony('@phony/ecommerce-tr');
$phony->seed(12345);
$user = $phony->generate('User'); // entity layer — target design
$users = $phony->generateMany('User', 1000);
$data = $phony->scenario('development');from phony import Phony
phony = Phony('@phony/ecommerce-tr')
phony.seed(12345)
user = phony.generate('User') # entity layer — target design
df = phony.to_dataframe('User', 1000)import { Phony } from '@phonycloud/phony';
const phony = new Phony('@phony/ecommerce-tr');
phony.seed(12345);
const user = await phony.generate('User'); // entity layer — target design
for await (const u of phony.stream('User', 100000)) {
await db.insert(u);
}Performance (estimates)
| Language | Speed (rec/sec) | Notes |
|---|---|---|
| PHP | 10-30K | Pure-PHP port; good for Laravel seeding |
| Python | 20-50K | PyO3 wheel runs the Rust core |
| TypeScript | 15-40K | WASM build of the Rust core |
Cloud Runtime (target design)
Maximum performance with enterprise features. The cloud platform is target design — its generation core is the same built Rust engine (phony-core), but the surrounding product (dashboard, API, DB sync, mock API, snapshots) is not built yet.
Architecture
┌─────────────────────────────────────────────────────────────────────────┐
│ CLOUD ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐│
│ │ Nuxt Dashboard ││
│ │ (TypeScript) ││
│ │ ├── Auth, Billing, UI ││
│ │ ├── Package Management ││
│ │ ├── Visual Schema Editor ││
│ │ └── Training Studio ││
│ └─────────────────────────────────────────────────────────────────────┘│
│ │ │
│ ▼ HTTP/REST │
│ ┌─────────────────────────────────────────────────────────────────────┐│
│ │ Go Engine ││
│ │ ├── API Server (net/http) ││
│ │ ├── DB Connectors (pgx, go-mysql) ││
│ │ ├── Sync Workers (asynq) ││
│ │ ├── Mock API Router ││
│ │ └── Export Manager ││
│ └─────────────────────────────────────────────────────────────────────┘│
│ │ │
│ ▼ CGO/FFI │
│ ┌─────────────────────────────────────────────────────────────────────┐│
│ │ Rust Core ││
│ │ (phony-core) ││
│ │ ├── N-gram Engine (~5M rec/sec) ││
│ │ ├── PGDL Compiler ││
│ │ ├── Template Engine ││
│ │ └── Parallel Generation ││
│ └─────────────────────────────────────────────────────────────────────┘│
│ │
│ Features: │
│ • High-speed generation (5M rec/sec) │
│ • Database sync & anonymization │
│ • Mock API hosting │
│ • Data snapshots & rollback │
│ • Team collaboration │
│ • Scheduled jobs │
│ │
└─────────────────────────────────────────────────────────────────────────┘Local-first: the DB connectors and sync workers above run in the customer-deployed agent (the data plane inside your network) — the hosted control plane orchestrates jobs and stores metadata only. Production row data never reaches Phony's servers. See Cloud Platform Architecture.
API Usage
# CLI against the control plane (target design)
phony login
phony sync start --source prod --target staging # executes on YOUR agentThere is no cloud-executed phony generate — generation is always local and always free. The control-plane API orchestrates sync/snapshot jobs and receives metadata only; the hosted /generate surface exists solely to feed synthetic-only features like hosted mock endpoints (see the REST API reference).
Cloud SDK (PHP)
use Phonyland\PhonyCloud;
$cloud = new PhonyCloud('api_key_here');
// Database sync — orchestrated here, executed by the agent in your infra
$cloud->sync([
'source' => 'prod', // agent-side connection names, not DSNs —
'target' => 'staging', // credentials never leave your network
'package' => '@phony/ecommerce-tr',
'tables' => ['users', 'orders'],
]);
// Create snapshot (stored in YOUR storage; cloud keeps the catalog entry)
$snapshot = $cloud->snapshot('staging-2024-03-15');
// Restore snapshot (the agent materializes it)
$cloud->restore($snapshot->id, 'staging');Performance
| Operation | Speed | Notes |
|---|---|---|
| Generation | 5M rec/sec | Parallel Rust engine |
| DB Sync | 100K rows/sec | With anonymization |
| Export (JSON) | 2M rec/sec | Streaming |
| Export (SQL) | 500K rows/sec | With transactions |
Execution Flow
1. Package Loading (git-based — as built)
┌─────────────────────────────────────────────────────────────────────────┐
│ PACKAGE LOADING (git-based) │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ 1. Resolve │
│ ├── Read the project's phony.json (git URL + version per dep) │
│ ├── MVS over the repo's vX.Y.Z tags (or a pinned ref) │
│ └── Pin the result in phony.lock (commit + checksum) │
│ │
│ 2. Fetch │
│ ├── Shallow git fetch into the global store (~/.phony/store) │
│ └── Remote assets (large .ngram models) by URL + sha256 │
│ │
│ 3. Load the closure │
│ ├── load_package_dir() per package: phony.json │
│ ├── Register compositions into the registry │
│ └── Merge list/model assets into the locale-keyed asset index │
│ │
│ 4. Generate │
│ └── Any generator or asset from any dependency is addressable │
│ │
└─────────────────────────────────────────────────────────────────────────┘Cloud-platform target design
The flows below describe entity-level orchestration (package manifests with schemas, entity dependency graphs, FK resolution, scenario distributions). The built engine has no entity layer — its real flow is: load packages (compositions + assets) → resolve a generator by name → derive per-cell seeds → produce values. Entity/relationship orchestration is the cloud-product design.
2. Generation Flow
┌─────────────────────────────────────────────────────────────────────────┐
│ GENERATION FLOW │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ Request: Generate User entity │
│ │
│ 1. Initialize RNG │
│ ├── Use provided seed OR │
│ └── Generate random seed │
│ │
│ 2. Resolve entity schema │
│ ├── Get field definitions │
│ └── Determine generation order (dependencies first) │
│ │
│ 3. For each field (in order): │
│ │ │
│ ├── Logic generator? │
│ │ └── Execute algorithm with params │
│ │ │
│ ├── List generator? │
│ │ ├── Load list (cached) │
│ │ └── Weighted random selection │
│ │ │
│ ├── Model generator? │
│ │ ├── Load model (cached) │
│ │ └── N-gram weighted random walk │
│ │ │
│ ├── Template generator? │
│ │ ├── Select variant (weighted) │
│ │ ├── Resolve placeholders (recursive) │
│ │ └── Apply operations │
│ │ │
│ ├── Reference (FK)? │
│ │ └── Get value from referenced entity │
│ │ │
│ └── Computed? │
│ └── Evaluate expression with current values │
│ │
│ 4. Apply entity constraints │
│ ├── Unique checks │
│ └── Validation rules │
│ │
│ 5. Return generated entity │
│ │
└─────────────────────────────────────────────────────────────────────────┘3. Relationship Resolution
┌─────────────────────────────────────────────────────────────────────────┐
│ RELATIONSHIP RESOLUTION │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ Scenario: Generate Orders with User references │
│ │
│ 1. Build dependency graph │
│ Order → User (Order.user_id references User.id) │
│ OrderItem → Order (OrderItem.order_id references Order.id) │
│ OrderItem → Product (OrderItem.product_id references Product.id) │
│ │
│ 2. Topological sort │
│ Generation order: User → Product → Order → OrderItem │
│ │
│ 3. Generate in order │
│ ├── Generate all Users, store IDs │
│ ├── Generate all Products, store IDs │
│ ├── Generate Orders, pick random User IDs │
│ └── Generate OrderItems, pick random Order/Product IDs │
│ │
│ 4. Distribution (if configured) │
│ ├── Gaussian: Most users have ~5 orders (mean) │
│ ├── Uniform: Each user has equal probability │
│ └── Custom: User-defined distribution │
│ │
└─────────────────────────────────────────────────────────────────────────┘Determinism
The determinism contract is built and testable today with the Rust CLI:
phony generate --use "@phony/core:logic.uuid_v7" --seed 12345 -n 1
# the same seed always reproduces the same valueEvery value is a pure function of a seed derived as derive_seed(root_seed, field_path, row_index) (FNV-1a + a mix function), drawn through the portable SplitMix64 PRNG. Because the .ngram format fixes vocabulary order and edge order, and every distribution samples by cumulative weights, any conforming runtime reproduces identical output.
The cross-language illustration below is target design (no OSS runtime has shipped yet — see the OSS section above):
// PHP (planned)
$phony = new Phony('@phony/ecommerce-tr');
$phony->seed(12345);
$user = $phony->generate('User');
// { id: '...abc123', first_name: 'Mehmet', ... }# Python (planned)
phony = Phony('@phony/ecommerce-tr')
phony.seed(12345)
user = phony.generate('User')
# { id: '...abc123', first_name: 'Mehmet', ... }All runtimes must produce the exact same output — that is the conformance bar, enforced by the shared test-vector suite.
Use Case Selection
Target-state guidance; today the Rust CLI covers the local rows.
| Use Case | Recommended Runtime | Why |
|---|---|---|
| Local development | OSS Library (planned; CLI today) | Fast enough, offline |
| Unit tests | OSS Library (planned; CLI today) | Deterministic, fast |
| CI/CD seeding | OSS Library or Cloud | Depends on data volume |
| Load testing | Cloud | High volume needed |
| Staging refresh | Cloud | DB sync + anonymization |
| Production mirroring | Cloud | Enterprise features |
| Model training | CLI | Training is CLI-only |