Execution Model
Phony packages can be executed on multiple runtimes with different performance characteristics. This document explains the execution architecture.
Overview
┌─────────────────────────────────────────────────────────────────────────┐
│ UNIFIED EXECUTION: Same Package, Different Engines │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ .phony Package │
│ (Portable, versioned) │
│ │ │
│ ┌───────────────┼───────────────┐ │
│ ▼ ▼ ▼ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ CLI │ │ OSS │ │ Cloud │ │
│ │ (Rust) │ │Libraries │ │ (Rust) │ │
│ │ │ │ │ │ │ │
│ │ Training │ │ PHP/Py/ │ │ Nuxt+Go+ │ │
│ │ Only │ │ JS/etc │ │ Rust │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ │ │ │ │
│ │ │ │ │
│ No generation 10-50K/sec 5M/sec │
│ Model training Good enough Maximum │
│ Free (MIT) Free (MIT) Paid │
│ │ │ │
│ └───────┬───────┘ │
│ ▼ │
│ Same Output! │
│ (Deterministic with seed) │
│ │
└─────────────────────────────────────────────────────────────────────────┘Runtime Comparison
| Feature | CLI (Rust) | OSS Libraries | Cloud (Rust) |
|---|---|---|---|
| Purpose | Training only | Generation only | Training + Generation |
| Training Speed | Basic | N/A | 10x faster (parallel) |
| Generation Speed | N/A | 10-50K/sec | 5M/sec |
| Languages | Binary | PHP, Python, JS | API + CLI trigger |
| Cost | Free | Free | Paid |
| License | MIT | MIT | Proprietary |
| Offline | Yes | Yes | No |
| Advanced Optimizations | No | No | Yes (not shared) |
| DB Sync | No | No | Yes |
| Mock API | No | No | Yes |
| Snapshots | No | No | Yes |
Key Architecture Decision: High-speed generation (5M/sec) and advanced training optimizations are Cloud-exclusive. This creates a clear value proposition: OSS for basic usage, Cloud for performance-critical workloads.
CLI Runtime (Rust)
The CLI is for training only - it creates models but does not generate data.
Architecture
┌─────────────────────────────────────────────────────────────────────────┐
│ CLI ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐│
│ │ phony CLI ││
│ ├─────────────────────────────────────────────────────────────────────┤│
│ │ ││
│ │ Commands: ││
│ │ ├── phony train Train N-gram models ││
│ │ ├── phony validate Validate PDL schemas ││
│ │ ├── phony packages Manage packages ││
│ │ └── phony cloud * Cloud features (after login) ││
│ │ ││
│ ├─────────────────────────────────────────────────────────────────────┤│
│ │ ││
│ │ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ ││
│ │ │ phony-core │ │ PDL │ │ Package │ ││
│ │ │ (Rust) │ │ Compiler │ │ Manager │ ││
│ │ │ │ │ │ │ │ ││
│ │ │ • N-gram algo │ │ • Parse JSON │ │ • Create │ ││
│ │ │ • Tokenize │ │ • Validate │ │ • Build │ ││
│ │ │ • Serialize │ │ • Resolve │ │ • Publish │ ││
│ │ └───────────────┘ └───────────────┘ └───────────────┘ ││
│ │ ││
│ └─────────────────────────────────────────────────────────────────────┘│
│ │
│ Output: .phony packages, .ngram model files │
│ │
└─────────────────────────────────────────────────────────────────────────┘Commands
bash
# Training
phony train names.txt -o models/names.ngram
phony train data.csv --column email -o models/emails.ngram
# Validation
phony validate schema.pdl.json
phony validate my-package.phony
# Package Management
phony packages create my-package
phony packages build ./my-package
phony packages install @phony/ecommerce-trOSS Libraries Runtime
Native implementations for each programming language.
Architecture
┌─────────────────────────────────────────────────────────────────────────┐
│ OSS LIBRARY ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ phonycloud/ │ │ phony │ │ @phonycloud/ │ │
│ │ phony-php │ │ (Python) │ │ phony │ │
│ │ (PHP) │ │ │ │ (TypeScript) │ │
│ ├─────────────────┤ ├─────────────────┤ ├─────────────────┤ │
│ │ │ │ │ │ │ │
│ │ Native PHP │ │ Native Python │ │ Native JS/TS │ │
│ │ implementation │ │ implementation │ │ implementation │ │
│ │ │ │ │ │ │ │
│ │ • gzdecode() │ │ • gzip.open() │ │ • zlib.gunzip() │ │
│ │ • json_decode() │ │ • json.load() │ │ • JSON.parse() │ │
│ │ • array_rand() │ │ • random.choice │ │ • Math.random() │ │
│ │ │ │ │ │ │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
│ │ │ │ │
│ └───────────────────┼───────────────────┘ │
│ ▼ │
│ Same .phony package │
│ Same output (with seed) │
│ │
└─────────────────────────────────────────────────────────────────────────┘PHP Example
php
use Phonyland\Phony;
// Load package
$phony = new Phony('@phony/ecommerce-tr');
// Generate single entity
$user = $phony->generate('User');
// {
// id: '018d6e5c-5c3a-7f1e-8b5a...',
// first_name: 'Mehmet',
// email: 'mehmet.yilmaz@gmail.com',
// ...
// }
// Generate multiple
$users = $phony->generateMany('User', 1000);
// With seed (deterministic)
$phony->seed(12345);
$user1 = $phony->generate('User');
$phony->seed(12345);
$user2 = $phony->generate('User');
// $user1 === $user2
// Scenario-based
$data = $phony->scenario('development');
// Generates all entities according to scenario counts
// Export
$phony->export('User', 1000, 'users.json');
$phony->export('Order', 5000, 'orders.csv');Python Example
python
from phony import Phony
# Load package
phony = Phony('@phony/ecommerce-tr')
# Generate single entity
user = phony.generate('User')
# Generate multiple
users = phony.generate_many('User', 1000)
# With seed
phony.seed(12345)
user1 = phony.generate('User')
phony.seed(12345)
user2 = phony.generate('User')
assert user1 == user2
# Scenario-based
data = phony.scenario('development')
# Export to pandas DataFrame
df = phony.to_dataframe('User', 1000)
# Export to file
phony.export('User', 1000, 'users.json')TypeScript Example
typescript
import { Phony } from '@phonycloud/phony';
// Load package
const phony = new Phony('@phony/ecommerce-tr');
// Generate single entity
const user = await phony.generate('User');
// Generate multiple
const users = await phony.generateMany('User', 1000);
// With seed
phony.seed(12345);
const user1 = await phony.generate('User');
phony.seed(12345);
const user2 = await phony.generate('User');
// user1 deep equals user2
// Scenario-based
const data = await phony.scenario('development');
// Streaming (for large datasets)
for await (const user of phony.stream('User', 100000)) {
await db.insert(user);
}Performance
| Language | Speed (rec/sec) | Notes |
|---|---|---|
| PHP | 10-30K | Native arrays, good for Laravel |
| Python | 20-50K | NumPy acceleration possible |
| TypeScript | 15-40K | V8 optimizations |
Cloud Runtime
Maximum performance with enterprise features.
Architecture
┌─────────────────────────────────────────────────────────────────────────┐
│ CLOUD ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐│
│ │ Nuxt Dashboard ││
│ │ (TypeScript) ││
│ │ ├── Auth, Billing, UI ││
│ │ ├── Package Management ││
│ │ ├── Visual Schema Editor ││
│ │ └── Training Studio ││
│ └─────────────────────────────────────────────────────────────────────┘│
│ │ │
│ ▼ HTTP/REST │
│ ┌─────────────────────────────────────────────────────────────────────┐│
│ │ Go Engine ││
│ │ ├── API Server (net/http) ││
│ │ ├── DB Connectors (pgx, go-mysql) ││
│ │ ├── Sync Workers (asynq) ││
│ │ ├── Mock API Router ││
│ │ └── Export Manager ││
│ └─────────────────────────────────────────────────────────────────────┘│
│ │ │
│ ▼ CGO/FFI │
│ ┌─────────────────────────────────────────────────────────────────────┐│
│ │ Rust Core ││
│ │ (phony-core) ││
│ │ ├── N-gram Engine (~5M rec/sec) ││
│ │ ├── PDL Compiler ││
│ │ ├── Template Engine ││
│ │ └── Parallel Generation ││
│ └─────────────────────────────────────────────────────────────────────┘│
│ │
│ Features: │
│ • High-speed generation (5M rec/sec) │
│ • Database sync & anonymization │
│ • Mock API hosting │
│ • Data snapshots & rollback │
│ • Team collaboration │
│ • Scheduled jobs │
│ │
└─────────────────────────────────────────────────────────────────────────┘API Usage
bash
# CLI with Cloud backend
phony login
phony cloud generate User --count 1000000 --package @phony/ecommerce-tr
# Direct API
curl -X POST https://api.phony.cloud/v1/generate \
-H "Authorization: Bearer $TOKEN" \
-d '{
"package": "@phony/ecommerce-tr",
"entity": "User",
"count": 1000000,
"format": "json",
"seed": 12345
}'Cloud SDK (PHP)
php
use Phonyland\PhonyCloud;
$cloud = new PhonyCloud('api_key_here');
// High-speed generation
$users = $cloud->generate('User', 1000000, [
'package' => '@phony/ecommerce-tr',
'format' => 'json',
]);
// Database sync
$cloud->sync([
'source' => 'mysql://prod-server/app_db',
'target' => 'mysql://staging-server/app_db',
'package' => '@phony/ecommerce-tr',
'tables' => ['users', 'orders'],
]);
// Create snapshot
$snapshot = $cloud->snapshot('staging-2024-03-15');
// Restore snapshot
$cloud->restore($snapshot->id, 'mysql://staging-server/app_db');Performance
| Operation | Speed | Notes |
|---|---|---|
| Generation | 5M rec/sec | Parallel Rust engine |
| DB Sync | 100K rows/sec | With anonymization |
| Export (JSON) | 2M rec/sec | Streaming |
| Export (SQL) | 500K rows/sec | With transactions |
Execution Flow
1. Package Loading
┌─────────────────────────────────────────────────────────────────────────┐
│ PACKAGE LOADING │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ 1. Locate package │
│ ├── Local: ~/.phony/packages/@phony/ecommerce-tr/ │
│ ├── Installed: vendor/phonycloud/phony-ecommerce-tr/ │
│ └── Registry: https://registry.phony.cloud/@phony/ecommerce-tr │
│ │
│ 2. Load and decompress │
│ ├── Read .phony file │
│ ├── Detect format (new package vs legacy model) │
│ └── Extract contents to memory │
│ │
│ 3. Parse manifest │
│ ├── Read manifest.json │
│ ├── Check dependencies │
│ └── Resolve inheritance chain │
│ │
│ 4. Compile schema │
│ ├── Parse schema.pdl.json │
│ ├── Validate all references │
│ ├── Build generator graph │
│ └── Cache compiled schema │
│ │
│ 5. Lazy-load resources │
│ ├── Models loaded on first use │
│ ├── Lists loaded on first use │
│ └── Memory-efficient for large packages │
│ │
└─────────────────────────────────────────────────────────────────────────┘2. Generation Flow
┌─────────────────────────────────────────────────────────────────────────┐
│ GENERATION FLOW │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ Request: Generate User entity │
│ │
│ 1. Initialize RNG │
│ ├── Use provided seed OR │
│ └── Generate random seed │
│ │
│ 2. Resolve entity schema │
│ ├── Get field definitions │
│ └── Determine generation order (dependencies first) │
│ │
│ 3. For each field (in order): │
│ │ │
│ ├── Logic generator? │
│ │ └── Execute algorithm with params │
│ │ │
│ ├── List generator? │
│ │ ├── Load list (cached) │
│ │ └── Weighted random selection │
│ │ │
│ ├── Model generator? │
│ │ ├── Load model (cached) │
│ │ └── N-gram weighted random walk │
│ │ │
│ ├── Template generator? │
│ │ ├── Select variant (weighted) │
│ │ ├── Resolve placeholders (recursive) │
│ │ └── Apply operations │
│ │ │
│ ├── Reference (FK)? │
│ │ └── Get value from referenced entity │
│ │ │
│ └── Computed? │
│ └── Evaluate expression with current values │
│ │
│ 4. Apply entity constraints │
│ ├── Unique checks │
│ └── Validation rules │
│ │
│ 5. Return generated entity │
│ │
└─────────────────────────────────────────────────────────────────────────┘3. Relationship Resolution
┌─────────────────────────────────────────────────────────────────────────┐
│ RELATIONSHIP RESOLUTION │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ Scenario: Generate Orders with User references │
│ │
│ 1. Build dependency graph │
│ Order → User (Order.user_id references User.id) │
│ OrderItem → Order (OrderItem.order_id references Order.id) │
│ OrderItem → Product (OrderItem.product_id references Product.id) │
│ │
│ 2. Topological sort │
│ Generation order: User → Product → Order → OrderItem │
│ │
│ 3. Generate in order │
│ ├── Generate all Users, store IDs │
│ ├── Generate all Products, store IDs │
│ ├── Generate Orders, pick random User IDs │
│ └── Generate OrderItems, pick random Order/Product IDs │
│ │
│ 4. Distribution (if configured) │
│ ├── Gaussian: Most users have ~5 orders (mean) │
│ ├── Uniform: Each user has equal probability │
│ └── Custom: User-defined distribution │
│ │
└─────────────────────────────────────────────────────────────────────────┘Determinism
With the same seed, all runtimes produce identical output:
php
// PHP
$phony = new Phony('@phony/ecommerce-tr');
$phony->seed(12345);
$user = $phony->generate('User');
// { id: '...abc123', first_name: 'Mehmet', ... }python
# Python
phony = Phony('@phony/ecommerce-tr')
phony.seed(12345)
user = phony.generate('User')
# { id: '...abc123', first_name: 'Mehmet', ... }bash
# Cloud
phony cloud generate User --count 1 --seed 12345 --package @phony/ecommerce-tr
# { id: '...abc123', first_name: 'Mehmet', ... }All three produce the exact same output.
Use Case Selection
| Use Case | Recommended Runtime | Why |
|---|---|---|
| Local development | OSS Library | Fast enough, offline |
| Unit tests | OSS Library | Deterministic, fast |
| CI/CD seeding | OSS Library or Cloud | Depends on data volume |
| Load testing | Cloud | High volume needed |
| Staging refresh | Cloud | DB sync + anonymization |
| Production mirroring | Cloud | Enterprise features |
| Model training | CLI | Training is CLI-only |