Local Stack and Migrations

Add a Configuration Field

1. Add the field to the schema in scripts/stack.ts.
2. Add a complete safe placeholder to config/local-stack.example.json.
3. Update config/local-stack.schema.json when the generated schema is tracked separately.
4. Map it to every required host and container environment file.
5. Update the consuming app’s environment validator.
6. Represent optional values explicitly only where the schema maps empty to undefined.
7. Add stack-generator tests.
8. Update Local Stack.

Generated environment files are outputs. Do not add another local environment contract or ask operators to edit generated files.

scripts/stack.ts owns the source-checkout experience. It always selects compose.yaml plus compose.build.yaml, then adds the development and local-auth overlays when configured. The base Compose file must remain build-free because the same file ships in the portable release bundle.

scripts/portable-stack.ts is the container-executed deployment boundary. It may create a minimal production configuration only when none exists, validates the same typed schema, generates only Compose/container environment files, and requires one exact semantic version for every first-party image. Keep portable generation free of host Bun, Git, jq, and repository-path assumptions.

Public local-stack URLs must be HTTPS and structurally exact origins. Derive Google and generic managed-MCP callbacks from the public backend origin; allow a provider-specific callback to override only its runtime, and retain loopback fallback when no public backend exists. Keep these public browser/provider URLs separate from internal BACKEND_URL values. A Vite public origin must produce one explicit allowed hostname; never use an unrestricted host allowlist or derive callbacks from request headers.

The accepted Cloudflare overlay is optional deployment infrastructure, not part of the base topology. Select it only from typed production configuration after validating both public origins, the named tunnel UUID, and an existing non-empty absolute credential JSON path. Generate ordered ingress under .runtime, mount credentials read-only from outside the repository, and pin the Cloudflared image. Keep remotely managed token tunnels external, keep development unchanged, and do not provision accounts, DNS, credentials, or access policy. Document stable-hostname requirements, ordered callback path routing, WebSocket forwarding, and the lack of an end-user authentication gate whenever public local testing is supported.

Add a Container Service

• Keep the Dockerfile with the owning app.
• Add a health check.
• Wait for healthy infrastructure and one-shot migrations.
• Use Compose DNS names inside containers.
• Make persistent/disposable lifecycle explicit.
• Add bounded idempotent shutdown.
• Log startup, shutdown, failures, and recovery—not every healthy tick.

Agent images use the Debian/glibc Bun base with CA certificates and required shell tooling. Do not move Codex execution to Alpine without proving binary and TLS compatibility.

PostgreSQL readiness uses pg_isready -h 127.0.0.1. During first-time initialization, its temporary Unix-socket server can answer before the final TCP server starts. Waiting for TCP keeps migrations and application startup behind the actual connection they require. CI uses the same check.

Garage mounts the docker/ directory read-only and uses GARAGE_CONFIG_FILE for both its server and healthcheck. A directory mount keeps the file visible after an editor or checkout atomically replaces garage.toml. An older container reporting a missing /etc/garage.toml needs recreation with the current Compose configuration; retain its named data and metadata volumes.

When changing topology, test both paths:

bun run stack config
.github/scripts/tests/deployment-bundle.test.sh

The portable rendering must have no build directives, bind published ports to loopback, preserve the pinned PostgreSQL/Redis/Garage/Cloudflared tags, and ensure only codex-auth-sync mounts host auth.json. The Cloudflare credential bind must exist only in its production-gated overlay.

Codex Credential Synchronization

Local authentication has one deployment-scoped host-credential writer. codex-auth-sync alone mounts the host auth.json and the shared codex_state volume. It validates last_refresh, installs only a strictly newer generation with an atomic mode-0600 rename, and publishes a readiness marker before chat or worker startup. Invalid source or active files fail closed.

The service watches the host file and also scans at codex.authSyncIntervalSeconds, which defaults to 60 seconds and accepts 10–86,400. Keep the interval in the canonical JSON config, generated Compose environment, runtime validator, tests, and user reference together.

Agent executions do not poll credentials. A Codex-auth failure writes a synchronization request marker and waits briefly for a new generation. The adapter retries once only before any item.started, item.updated, or item.completed event. Never replay after an item event, on cancellation, or without proof that the generation advanced.

Add a Migration

1. Update shared and app-local typed contracts.
2. Change schema enums, tables, and relationships in their owning modules.
3. Generate the migration and snapshot.
4. Inspect generated SQL.
5. Update repositories and lifecycle services.
6. Update endpoint validators and OpenAPI.
7. Add migration and repository tests.
8. Verify upgrade behavior against existing state.

Do not edit an already-applied migration to represent a new schema change.

The shared-insights cutover retires the old metric-points/entity-events segment index, numeric point projections, unversioned calculation definitions, and entity/FX caches. Their rows cannot be converted into trustworthy analytical facts because they lack source identity and calculation provenance. Back up an existing database before this cutover. Original object-store files remain; canonical facts/buckets, hashed definitions, organizations, chats, tasks and integration settings are preserved. The new collectors establish supported history independently. This is a hard cutover, not a compatibility reader for old analytics.

The following data migration removes retired operating_context.role values before strict Context validation at catalog reconciliation. It increments the affected Context revision and invalidates old chat sessions, preserving resources, credentials, access settings and current purpose ordering.

Exercise migrations with retained old rows and already-written canonical analytics, as well as an empty database. Run the opt-in analytical-cutover-migration.postgres.test.ts against disposable PostgreSQL before releasing changes to this boundary.

Add a Pinned Vendor Binary

Install it during the owning image build with an exact version and per-architecture checksum. Keep integration binaries below their provider-owned directory and off the global path. Rebuild and verify every affected image.

See Database performance for the table audit, query-plan regressions, and index migration rollout considerations.

Production product journeys

Journey scripts run from the root package, which declares @oblive/types and @oblive/objectstore as development dependencies. Keep every root script’s package imports declared there: Bun’s isolated linker does not make all workspace members available to the root. Reproduce CI with a fresh frozen-lockfile installation when changing dependencies; old node_modules links can hide missing declarations. Root lint does not cache file results because imported type changes can invalidate an unchanged file’s typed checks.

The frontend build command explicitly uses production mode, even when the generated local .env uses development mode. Host backend processes use the configured backend port; the Compose service continues to listen on port 3000.

Use a disposable local stack with PostgreSQL, Redis, Garage, backend and frontend running. Keep chat and worker agents stopped for these checks; the scheduler must run to process bulk operations. Build and serve the frontend with bun run --cwd apps/frontend build and bun run --cwd apps/frontend serve, using the generated environment.

bunx playwright install chromium
JOURNEY_BACKEND_URL=http://127.0.0.1:3000 \
JOURNEY_FRONTEND_URL=http://127.0.0.1:3001 \
JOURNEY_DATABASE_URL=postgresql://oblive:local-password@127.0.0.1:5432/oblive \
bun run test:journeys

The journey creates and removes its own organization, seeds a missing optional health artifact, and checks optional-website onboarding, the shared integration sheet, setup chat without uploads, fresh Astra profiles, repeated Context refresh, one-shot asset recovery, frozen bulk selection, Trash restore and permanent deletion. It requires loopback endpoints and makes no provider/model calls. A failure saves .runtime/product-journey-failure.png; JOURNEY_SCREENSHOT_PATH optionally records the successful final screen.

Run the opt-in infrastructure regressions against the same disposable services:

TEST_REDIS_URL=redis://127.0.0.1:6379 bun test packages/redis/tests/streams.integration.test.ts
TEST_DATABASE_URL=postgresql://oblive:local-password@127.0.0.1:5432/oblive \
TEST_OBJECT_STORE_ENABLED=true \
bun --env-file=apps/backend/.env test apps/backend/tests/database

These supplement the regular suite with real PostgreSQL transitions and query plans, abandoned Redis delivery recovery after reconnect, and Garage cleanup that resumes after an injected connection failure. The object-store credentials come from the generated backend environment.