Sync latest TrueGrowth updates

openspec/changes/unify-local-data-persistence/design.md

@@ -0,0 +1,25 @@
+## Context
+The desktop shell already pins Electron `userData`, but development Local API runs can still default to `.truegrowth-runtime`, while packaged or Electron-launched runs use `Application Support/TrueGrowth/runtime`. Browser `localStorage` and `IndexedDB` are origin scoped, so `127.0.0.1:7200` and preview ports cannot share data directly.
+
+## Goals / Non-Goals
+- Goals: make desktop business data survive restarts and port changes, expose storage health, migrate legacy runtime records, and avoid duplicating large media files.
+- Non-Goals: rewrite every browser IndexedDB cache in one step, delete legacy runtime folders, or bundle a new native database dependency.
+
+## Decisions
+- Use `~/Library/Application Support/TrueGrowth/data/truegrowth.sqlite` (platform equivalent on Windows/Linux) as the canonical ledger.
+- Continue using the existing `sqlite3` CLI integration pattern to avoid adding native npm dependencies.
+- Store structured business records as JSON payloads in typed ledger tables, with a `kv` table for aggregate state and migration markers.
+- Keep JSON shadows during migration for compatibility, but prefer SQLite reads whenever available.
+- Index legacy large files by absolute path and local media URL; do not copy the 35GB `.truegrowth-runtime` tree by default.
+
+## Risks / Trade-offs
+- SQLite CLI may be missing on a fresh machine. Mitigation: diagnostics report unavailable status and Local API falls back to existing JSON behavior until the runtime readiness flow installs or bundles SQLite.
+- Browser-only IndexedDB data cannot be read by Node without Chromium internals. Mitigation: make the desktop Local API the authoritative source going forward and keep browser storage as a transitional source.
+- Migration may discover stale or deleted files. Mitigation: diagnostics report missing targets and preserve legacy source metadata.
+
+## Migration Plan
+1. Create canonical data/runtime directories under desktop user data.
+2. Initialize ledger schema and record current storage roots.
+3. Import legacy runtime tasks, cleared task IDs, business workflow JSON/SQLite state, AIGCPanel state, social publishing state, and media file indexes.
+4. Keep legacy files untouched and record migration versions in `storage_migrations`.
+5. Expose diagnostics so support can see which roots are active and which legacy sources were imported.

openspec/changes/unify-local-data-persistence/proposal.md

@@ -0,0 +1,15 @@
+# Change: Unify TrueGrowth local data persistence
+
+## Why
+TrueGrowth desktop currently stores business data across Electron browser storage, project-local runtime files, desktop runtime files, JSON shadows, and small SQLite stores. Repeated local runs and different ports can make data appear lost because each browser origin and runtime root sees a different persistence estate.
+
+## What Changes
+- Add one desktop-local persistence contract centered on a canonical per-user data directory and SQLite ledger.
+- Route Local API business data through the canonical storage layer for tasks, assets, workflow state, social publishing state, runtime settings, and migration records.
+- Add diagnostics, migration, export, and import endpoints so data roots and legacy sources are visible and recoverable.
+- Keep browser storage for UI preferences, caches, and temporary offline queues rather than treating it as the final business-data source.
+- Preserve legacy project-local runtime data by indexing or importing metadata without deleting or moving large output files automatically.
+
+## Impact
+- Affected specs: `desktop-local-persistence`
+- Affected code: `scripts/truegrowth-local-api.mjs`, Electron startup environment, local runtime client helpers, storage diagnostics and migration flows

openspec/changes/unify-local-data-persistence/specs/desktop-local-persistence/spec.md

@@ -0,0 +1,49 @@
+## ADDED Requirements
+
+### Requirement: Canonical Desktop Data Root
+TrueGrowth SHALL use a single writable per-user desktop data root as the canonical location for business persistence.
+
+#### Scenario: Local API starts outside Electron
+- **WHEN** the Local API starts without explicit runtime path environment variables
+- **THEN** it resolves the same platform user data root that Electron uses
+- **AND** it stores canonical data under a `data` directory and runtime outputs under a `runtime` directory.
+
+#### Scenario: Electron starts the Local API
+- **WHEN** Electron launches or imports the Local API
+- **THEN** it provides the canonical `TRUEGROWTH_ELECTRON_USER_DATA` and `TRUEGROWTH_RUNTIME_STATE_DIR`
+- **AND** repeated desktop launches use the same persisted data root.
+
+### Requirement: SQLite Business Ledger
+TrueGrowth SHALL store desktop business records in a canonical SQLite ledger.
+
+#### Scenario: Runtime task is created
+- **WHEN** a local runtime task is queued, updated, completed, failed, or cleared
+- **THEN** the task and clear marker are written to the ledger
+- **AND** restarting the Local API restores the latest task state from the ledger.
+
+#### Scenario: Runtime asset is imported or generated
+- **WHEN** a media asset is imported or generated
+- **THEN** its metadata is written to the ledger with local file path and URL identity
+- **AND** the media file remains in the managed runtime output directory or its indexed legacy path.
+
+#### Scenario: Workflow or publishing state changes
+- **WHEN** workflows, schedules, runs, events, accounts, campaigns, or jobs change
+- **THEN** the aggregate state is saved to the ledger
+- **AND** JSON shadow files remain compatible during migration.
+
+### Requirement: Storage Diagnostics And Migration
+TrueGrowth SHALL expose local storage diagnostics and safe migration controls.
+
+#### Scenario: User checks storage health
+- **WHEN** a client requests storage diagnostics
+- **THEN** the response includes canonical paths, active runtime path, SQLite availability, ledger path, legacy source status, and warnings for split roots.
+
+#### Scenario: Legacy project runtime exists
+- **WHEN** migration runs and `.truegrowth-runtime` contains previous records or outputs
+- **THEN** TrueGrowth imports metadata and indexes file paths without deleting or copying large files by default
+- **AND** records the migration result in the ledger.
+
+#### Scenario: User exports or imports local data
+- **WHEN** export or import is requested
+- **THEN** TrueGrowth serializes or restores ledger-managed business data through the Local API
+- **AND** invalid import payloads are rejected without clearing existing data.

openspec/changes/unify-local-data-persistence/tasks.md

@@ -0,0 +1,26 @@
+# Tasks: Unified local persistence
+
+## 1. Specification
+- [x] 1.1 Create OpenSpec proposal, design, task list, and delta spec.
+- [ ] 1.2 Validate with `openspec validate unify-local-data-persistence --strict` once the OpenSpec CLI is available in this checkout.
+
+## 2. Local API Storage
+- [x] 2.1 Resolve canonical desktop data and runtime paths for Electron and standalone Local API runs.
+- [x] 2.2 Initialize `truegrowth.sqlite` with tables for tasks, assets, project records, settings, workflow state, social publishing, and migration records.
+- [x] 2.3 Persist runtime tasks, cleared task IDs, runtime assets, workflow state, social publishing state, AIGCPanel state, and runtime settings to the ledger.
+- [x] 2.4 Preserve JSON shadow files for compatibility while preferring ledger reads.
+- [x] 2.5 Persist Drawnix workspace folders, boards, and workspace state through the Local API ledger while keeping IndexedDB as a shadow/offline fallback.
+
+## 3. Migration And Diagnostics
+- [x] 3.1 Add storage diagnostics, migrate, export, and import endpoints.
+- [x] 3.2 Import legacy `.truegrowth-runtime` metadata without moving large files.
+- [x] 3.3 Report split roots, SQLite availability, and canonical path status.
+
+## 4. Client And Verification
+- [x] 4.1 Add frontend Local API client helpers and types for the storage endpoints.
+- [x] 4.2 Run Node syntax checks, TypeScript checks, and targeted Local API verifier coverage.
+- [x] 4.3 Document remaining browser IndexedDB transition limits.
+
+## Notes
+- OpenSpec strict validation is still pending because the local `openspec` CLI is not available in this checkout (`pnpm exec openspec ...` reports command not found).
+- Browser IndexedDB remains as an offline/shadow cache for Drawnix workspace and UI-only preferences; Local API/SQLite is now authoritative when the desktop Local API is reachable.