Tech Stack
📚 PGDL Architecture: This page covers the technical implementation stack. For the data generation architecture (PGDL, generators, packages), see Architecture Overview.
System Architecture
┌─────────────────────────────────────────────────────────────────────────┐
│ PHONY - ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ╔═══════════════════════════════════════════════════════════════════╗ │
│ ║ UNIFIED CLI: phony (Foundation) ║ │
│ ╠═══════════════════════════════════════════════════════════════════╣ │
│ ║ ║ │
│ ║ PURPOSE: Single tool for entire workflow (like gh, vercel) ║ │
│ ║ LICENSE: MIT (fully open source) ║ │
│ ║ DISTRIBUTION: Single binary, no dependencies ║ │
│ ║ ║ │
│ ║ OFFLINE (No Auth): ║ │
│ ║ $ phony train input.txt --locale tr_TR -o names.ngram ║ │
│ ║ $ phony train data.csv --column name -o model.ngram ║ │
│ ║ $ phony info model.ngram # Show model metadata ║ │
│ ║ $ phony generate gen.json -n 1000 --seed 42 # Local generation ║ │
│ ║ ║ │
│ ║ CLOUD (After: phony login): ║ │
│ ║ $ phony models push/pull # Cloud model library ║ │
│ ║ $ phony sync # Database sync ║ │
│ ║ $ phony mock start/deploy # Mock API ║ │
│ ║ $ phony generate --count 1M # Bulk/scale generation (5M/s) ║ │
│ ║ ║ │
│ ║ WHY UNIFIED CLI: ║ │
│ ║ • Offline-first (train + generate locally, free, no auth) ║ │
│ ║ • Progressive disclosure (cloud features unlock with login) ║ │
│ ║ • Language-agnostic (PHP dev, Python dev, anyone can use) ║ │
│ ║ • Single binary distribution (curl | sh install) ║ │
│ ║ • Same core reused in Cloud backend (Rust) ║ │
│ ║ ║ │
│ ╚═══════════════════════════════════════════════════════════════════╝ │
│ │
│ ╔═══════════════════════════════════════════════════════════════════╗ │
│ ║ OSS GENERATION LIBRARIES ║ │
│ ╠═══════════════════════════════════════════════════════════════════╣ │
│ ║ ║ │
│ ║ PURPOSE: Generate data from .ngram models (GENERATION ONLY) ║ │
│ ║ APPROACH: Pure native implementation per language ║ │
│ ║ DEVELOPMENT: AI-assisted (AI writes, human reviews) ║ │
│ ║ FORMAT: Reads .ngram model format (created by Rust CLI) ║ │
│ ║ ║ │
│ ║ TIMELINE: ║ │
│ ║ ├── Year 1 Q1-Q2: PHP/Laravel (phonycloud/phony-php) ║ │
│ ║ ├── Year 2: Python (phonycloud/phony-python) ★ Revenue ║ │
│ ║ ├── Year 3: TypeScript (@phonycloud/phony) - Optional ║ │
│ ║ └── Future: Community contributions for other languages ║ │
│ ║ ║ │
│ ║ WHAT OSS PACKAGES DO: ║ │
│ ║ ├── Load .ngram models (bundled or custom) ║ │
│ ║ ├── Generate data (N-gram weighted random walk) ║ │
│ ║ ├── Deterministic output (seed support) ║ │
│ ║ └── Framework integrations (Laravel, FastAPI, etc.) ║ │
│ ║ ║ │
│ ║ WHAT OSS PACKAGES DON'T DO: ║ │
│ ║ └── Training (use Rust CLI or Phony Cloud instead) ║ │
│ ║ ║ │
│ ║ PERFORMANCE: ~10-50K records/sec (good enough for typical use) ║ │
│ ║ MAINTENANCE: Community can maintain non-PHP implementations ║ │
│ ║ ║ │
│ ╚═══════════════════════════════════════════════════════════════════╝ │
│ │
│ ╔═══════════════════════════════════════════════════════════════════╗ │
│ ║ CLOUD PLATFORM ║ │
│ ╠═══════════════════════════════════════════════════════════════════╣ │
│ ║ ║ │
│ ║ STACK: Nuxt + Go + Rust (3 languages) ║ │
│ ║ ║ │
│ ║ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ║ │
│ ║ │ NUXT │ │ GO │ │ RUST │ ║ │
│ ║ │ Dashboard │ │ Engine │ │ Core │ ║ │
│ ║ │ │ │ │ │ │ ║ │
│ ║ │ • Vue 3 │ │ • DB Sync │ │ • N-gram │ ║ │
│ ║ │ • TypeScript│ → │ • Mock API │ → │ • Train │ ║ │
│ ║ │ • Auth │ │ • Training │ │ • Generate │ ║ │
│ ║ │ • Billing │ │ • pgx (DB) │ │ • Model I/O │ ║ │
│ ║ │ │ │ │ │ │ ║ │
│ ║ └─────────────┘ └─────────────┘ └─────────────┘ ║ │
│ ║ ║ │
│ ║ RUST CORE: phony-core (via crates.io) + cloud FFI, compiled via CGO ║ │
│ ║ NO LARAVEL: Go handles all backend (sync, API, processing) ║ │
│ ║ PERFORMANCE: ~5M records/sec (Rust-powered) ║ │
│ ║ SCALE: TB-level database sync capable ║ │
│ ║ ║ │
│ ╚═══════════════════════════════════════════════════════════════════╝ │
│ │
│ ╔═══════════════════════════════════════════════════════════════════╗ │
│ ║ MONETIZATION STRATEGY ║ │
│ ╠═══════════════════════════════════════════════════════════════════╣ │
│ ║ ║ │
│ ║ FREE EVERYWHERE (OSS, MIT): ║ │
│ ║ ├── Training: CLI + files, full speed, unlimited ║ │
│ ║ └── Generation: CLI + libraries, unlimited — never metered ║ │
│ ║ (generation is the adoption funnel, not the moat) ║ │
│ ║ ║ │
│ ║ MONETIZATION (Cloud, priced per connected data source): ║ │
│ ║ ├── Tiers: $0 / $49 / $199 / $599 / Enterprise ║ │
│ ║ ├── Database sync & anonymization (local-first data plane) ║ │
│ ║ ├── Mock API hosting, snapshots, scheduled jobs ║ │
│ ║ └── Team collaboration, compliance ║ │
│ ║ ║ │
│ ║ KEY INSIGHT: ║ │
│ ║ The moat is the prod-data platform (connect DB → detect PII → ║ │
│ ║ safe, referentially-intact copy) — not generation speed. ║ │
│ ║ ║ │
│ ╚═══════════════════════════════════════════════════════════════════╝ │
│ │
└─────────────────────────────────────────────────────────────────────────┘Generation is free and unlimited everywhere; Cloud pricing is per connected data source — see Pricing.
Why This Architecture?
| Decision | Rationale |
|---|---|
| No Laravel | Go already handles sync/API; Laravel would be overhead |
| Nuxt for Dashboard | Vue familiarity (from Inertia), TypeScript, easy deploy |
| Go for Engine | Best DB libraries (pgx), great concurrency, fast HTTP |
| Rust for Core | Maximum N-gram performance, memory efficiency |
| Pure native OSS | Easy install, no binary deps, community maintainable |
| AI-assisted dev | Makes unfamiliar languages feasible |
Binary .ngram format | One documented byte layout; compact, deterministic across runtimes |
The .ngram Model Format
Models are saved as .ngram files — a portable binary format (magic PHNYNG03 + gzip-compressed body) defined normatively in FORMAT.md in the phony-core repository.
┌─────────────────────────────────────────────────────────────────────────┐
│ .NGRAM FILE FORMAT │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ FILE STRUCTURE: │
│ ═══════════════ │
│ .ngram file = 8-byte magic ("PHNYNG03") + gzip-compressed binary body │
│ │
│ Body (single n-gram order): │
│ ├── order, token_type, tokenizer config, metadata │
│ ├── interned vocab (sorted; index = u32 id) │
│ ├── opener + transition distributions (LEB128 varint id/weight) │
│ ├── word/sentence/paragraph length distributions │
│ └── deduped originals (unique item + count) │
│ │
│ INSPECT: │
│ $ phony info model.ngram # metadata │
│ $ phony validate model.ngram # format + integrity │
│ $ phony stats model.ngram # statistics │
│ │
│ WHY THIS FORMAT: │
│ ════════════════ │
│ ✓ One documented byte layout — readable with plain integer reads │
│ (LEB128 varints + UTF-8 strings); no schema library, no reflection │
│ ✓ Compact: interned vocab + varint weights + gzip │
│ ✓ Deterministic: normative edge ordering → identical sampling │
│ across runtimes │
│ ✓ Fast load: the on-disk layout mirrors the in-memory CSR graph │
│ │
└─────────────────────────────────────────────────────────────────────────┘Detailed specification: the normative byte layout lives in
FORMAT.mdin thephony-corerepository. For the modelling concepts, see N-gram Models Architecture.
Format Versioning
The version is carried in the magic string (PHNYNG03). The project is pre-release: readers reject anything not starting with the current magic, and there is no legacy format to support.
Reading Models
.ngram files are not ad-hoc JSON — they are read through the ngram-core reader, which parses the binary layout (per FORMAT.md) directly into the in-memory graph:
rust
use ngram_core::NgramModel;
let model = NgramModel::load("tr_TR/names.ngram")?; // validates magic + layout
let names = model.generate_many(/* seed */ 42, /* count */ 10, /* max_len */ 32);Planned language libraries (PHP first, then JS/Python) will ship the same reader: the fixed byte layout is what makes a dependency-free port possible — plain integer reads, no external dependencies.
Timeline
┌─────────────────────────────────────────────────────────────────────────┐
│ IMPLEMENTATION TIMELINE │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ YEAR 1 Q1-Q2: FOUNDATION │
│ ═════════════════════════ │
│ Week 1-4: Rust (phony-core + phony-cli) │
│ ├── phony-core: N-gram algorithms, model I/O │
│ ├── phony-cli: train, generate, info, validate, stats │
│ ├── .ngram format (portable binary, zero dependencies) │
│ └── Pre-train bundled models (tr_TR, en_US, etc.) │
│ │
│ Week 5-10: PHP Generation Library (phonycloud/phony-php) │
│ ├── Load .ngram models (from CLI) │
│ ├── N-gram generation (native PHP) │
│ ├── Deterministic seed support │
│ └── Pre-trained models bundled │
│ │
│ Week 11-14: Laravel Integration (phonycloud/phony-laravel) │
│ ├── Service provider, facades │
│ ├── Factory integration │
│ └── Artisan commands │
│ │
│ YEAR 1 Q3-Q4: CLOUD MVP │
│ ════════════════════════ │
│ Week 1-4: Rust (phony-core + cloud FFI + optimizations) │
│ Week 5-10: Go Engine (sync, mock API, training orchestration) │
│ Week 11-16: Nuxt Dashboard (UI, auth, billing) │
│ Week 17-20: PHP Cloud SDK (phonycloud/phony-cloud) │
│ │
│ YEAR 2: PYTHON ECOSYSTEM │
│ ═════════════════════════ │
│ Q1-Q2: Python Generation Library (pip install phony) │
│ ├── Load .ngram models │
│ ├── N-gram generation (native Python) │
│ └── FastAPI/Django integrations │
│ │
│ Q3-Q4: Python Cloud SDK + Advanced Cloud Features │
│ ├── pip install phony-cloud │
│ ├── DB Sync advanced features │
│ └── Mock API stateful mode │
│ │
│ YEAR 3: TYPESCRIPT + SCALE │
│ ═════════════════════════ │
│ Q1-Q2: TypeScript Generation (npm install @phonycloud/phony) │
│ ├── Works in Node.js and browser │
│ └── TypeScript Cloud SDK │
│ │
│ Q3-Q4: Enterprise Features │
│ └── SSO, audit logs, on-premise option │
│ │
└─────────────────────────────────────────────────────────────────────────┘
VISUAL TIMELINE:
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3
├─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┤
YEAR 1 │Rust │ PHP │ Cloud MVP │
│ CLI │ Gen │████████████████████│
│█████│█████│ │
│ │ │ │
YEAR 2 │ │ Python Gen │ Cloud Adv │
│ │█████████████│█████████████│
│ │ │ │
YEAR 3 │ │ TypeScript │ Enterprise │
│ │█████████████│████████████│
│
💰 │ ←─── Revenue Starts ───→│←── Scale ───→Layer Details
Layer 1: Nuxt (TypeScript + Vue)
| Aspect | Detail |
|---|---|
| Purpose | Web dashboard, auth, billing UI |
| Why | Vue familiarity, TypeScript, easy deploy to Vercel |
| Tech | Nuxt 3, Vue 3, Auth.js, Stripe SDK |
Layer 2: Go Engine
| Aspect | Detail |
|---|---|
| Purpose | DB sync, Mock API server, training orchestration |
| Why | Goroutines for concurrency, memory efficiency, fast HTTP |
| Tech | Go 1.22+, pgx, go-mysql, asynq |
Layer 3: Rust Core
| Aspect | Detail |
|---|---|
| Purpose | N-gram train/generate/save/load |
| Why | Maximum performance for hot path (5M/sec) |
| Tech | Rust stable, binary .ngram model I/O (gzip via flate2), FFI to Go |
Communication Flow
Nuxt ──HTTP──► Go Engine ──FFI (CGO)──► Rust CoreAI-Assisted Development
Development leverages AI agents for code generation with human review.
What AI Does Well
- Writing isolated modules with clear interfaces
- Test generation
- Boilerplate and CRUD operations
- Documentation
- Code translation between languages
What Needs Human Expertise
- Architectural decisions
- Performance optimization strategies
- Debugging complex issues
- Security review
- Production incident response
Language-Specific AI Effectiveness
| Language | AI Code Quality | Review Difficulty |
|---|---|---|
| TypeScript | ★★★★★ | Easy |
| Go | ★★★★ | Medium (readable) |
| Rust | ★★★★ | Hard (lifetimes, borrowing) |
Key Insight: AI can write Rust well, but reviewing Rust for correctness requires understanding ownership/borrowing. Go is more forgiving for reviewers.