Upgrading

Choose a reviewed release or revision supported by your deployment. Verify the target exists; these docs do not designate an example tag as a published release.

Before upgrading

1. Review the target's migrations, configuration changes and deployment notes.
2. Back up the database and rehearse restoration. Preserve operator-managed secrets and configuration outside source control.
3. Rehearse migrations on representative data, including large usage tables and billing state.
4. Schedule the service window and deploy compatible gateway, core, auth, billing and frontend versions.

Build and verify

For an existing local Compose checkout already updated to the intended revision:

make tokamak-reload
make tokamak-health

The reload target rebuilds images and recreates containers. Production environments should use their reviewed image/deployment pipeline rather than treating an unreviewed pull from main as a release process.

The customer budget/invitation/key changes require the corresponding auth/core/gateway versions. Purchase review requires the billing quote endpoint and customer app together. Keep per-organization billing activation under the operator's explicit rollout procedure.

Database migrations

Core auto-migrates when AUTO_MIGRATE=true. Auth and billing migrate their own schemas on boot independently of that setting. Files live under tokamak-services/services/{core,auth,billing}/migrations/.

Core92 widens usage-limit costs to twelve decimals. It refuses inline rewriting above100,000 live rows or64MiB total relation size per table. Larger deployments need a rehearsed maintenance window and explicitly reviewed tokamak.budget_widen_inline_limit / tokamak.budget_widen_inline_bytes connection settings. It fails rather than silently running with rounded enforcement. After a refusal, verify rollback before repairing the migration version marker and retrying; never simply mark the migration applied.

Auth25 adds its API-key check without scanning historical rows under the DDL lock. New/updated rows are checked immediately. Optional historical validation should run separately from the column/trigger migration.

Core123 adds budget periods, time zones and model sets, with constant defaults and no table rewrite. Its down migration deletes budgets of the new period kinds. Core124 adds budget alerts (budget_events, notification_outbox, notification_deliveries). Alerts are recorded without any configuration; to email them, set TOKAMAK_NOTIFICATION_SMTP_HOST, _PORT, _FROM and, for an authenticated relay, _USERNAME and _PASSWORD (which require TLS), plus TOKAMAK_NOTIFICATION_APP_URL for links. Deploy gateway with core so browsers can read the new X-Tokamak-Budget-* headers.

Core migrations from 000080 on attach their tokamak.* function settings best-effort: a migration role without SET ON PARAMETER (typical on managed PostgreSQL) completes with a WARNING … could not attach instead of failing. Core125 splits one-hour cache writes into cache_creation_1h_tokens; add per_1k_cache_creation_1h_tokens catalog prices only after every core replica runs a build with Core125, because an older replica takes a model carrying that unit out of service. Core126 adds token_usage.tags as a catalog-only column with no index (Core129 drops the GIN index an earlier revision built on dev); Core127 and Core128 add the data capture tables. Core130 adds capture_exports.skipped (a constant-default column, no rewrite); from the same build, capture objects are stored without Content-Encoding and core reads back objects a proxy has already decoded, so an organization bucket behind Cloudflare exports correctly.

Do not downgrade by replacing an image and assuming the schema follows. Some down migrations deliberately refuse destructive loss of live bindings or governance. Follow the release-specific recovery procedure and preserve billing evidence.

Troubleshooting upgrades

Use make tokamak-logs and Troubleshooting.