Skip to the content.

CI/CD topology

Four environments: Local (SQLite on the host, DynamoDB Local in Docker) → Development (Docker + DynamoDB Local) → Test (SAM) → Production (SAM, manual).

Catalog: docs/README.md. Env vars: docs/web/setup/environment.md. Docker: docker.md. SAM: infra.md.

Agent skills (this repo):

Portable templates for other projects / AI agents:

Flow

PR → development          → CI: Local parity only (skip version-bump-only PRs)
merge → development       → Regression: Full regression (e2e)           ← the only e2e
                            CD AWS Test (dry-run): one job, SYS_ENV=test ← the only test dry-run
                            Version bump (development): patch PR (skipped while a release PR is open)
npm run release:pr        → release/v<version>: ONE commit = development's tree on top of main
PR release/v* → main      → CI: Local parity + Regression gate (regression-gate.yml; reads the post-merge e2e; runs no tests)
                            Version bump (release): minor|major PR into the release branch, lands before the merge
squash-merge → main       → CD AWS Test (SAM deploy → test); on success → production dry-run
                            (no full CI re-run; prod dry-run does not deploy)
                            Version sync: bump PR into development carrying main's release version
release published         → CD AWS Production (tagged commit + Environment approval)
workflow_dispatch         → CD AWS Production (confirm=deploy + Environment approval)

Pull requests never run e2e or a SAM dry-run. Each dry-run runs once per target: test after a merge to development, production after a successful test deploy on main. A legacy release PR straight from development has the development tip as its head, so GitHub also lists that commit’s push checks on it, labelled (push); a release PR from npm run release:pr does not (its head is a new commit). scripts/cicd-workflow-policy.test.mjs fails if any pull_request-triggered workflow runs e2e:regression or sam-dry-run.

Merge policy

Branch How to land changes Merge method
development Pull request only (admin bypass allowed) Merge or squash
main Pull request only (admin bypass allowed) Squash only (workflows assume it)

Configure via Rulesets (npm run gh:setup-rulesets). Squash commit text must be PR title only (npm run gh:setup-merge-settings) — see Skip-CI trap below. Do not treat unprotected direct push as the default path.

Matrix

Environment Runtime Data Trigger GitHub Environment
Local Nitro node-server SQLite data/local.sqlite Developer machine —
Development Docker web + api + DynamoDB Local + MinIO (local Compose; Postgres opt-in) DynamoDB Local (default), PostgreSQL via DATA_BACKEND=postgres Local npm run docker:dev only —
Test SAM: Lambda + HTTP API + S3 + DynamoDB DynamoDB Squash-merge to main (after PR CI) test
Production Same SAM topology DynamoDB (prod stack) GitHub Release published (non-prerelease) or manual workflow_dispatch production

Docker Compose under docker/ maps to this matrix as local stand-ins: local ≈ docker-compose.local.yml (web + DynamoDB Local + MinIO); development ≈ docker-compose.dev.yml (web/api + DynamoDB Local + MinIO; Postgres opt-in); test ≈ docker-compose.test.yml (DynamoDB Local + MinIO + Lambda RIE). All four stacks share docker/compose.aws-emulators.yml. None is identical to AWS (no CloudFront in Compose; split web/api only in Docker dev). DATA_BACKEND overrides the SYS_ENV data-store default. docker:local and docker:dev both bind host 4200 — run only one. Crosswalk table and # aws: service comments: docker/README.md.

Nitro output (SYS_ENV)

Every Nuxt build writes to workspace .output/<sysEnv> (scripts/nitro-output-dir.mjs). Do not mix trees.

SYS_ENV Preset Directory Consumers
local node-server .output/local npm run build / CI Local parity
development node-server .output/development Docker Dockerfile.app (COPY flattened to /app/.output)
test aws_lambda (e2e: node-server) .output/test sam:build test, docker:test, CI e2e
production aws_lambda .output/production sam:build production / CD prod

E2E keeps SYS_ENV=test identity and still lands in .output/test/ when NITRO_PRESET=node-server. The message and blog API store mappers give E2E_STORE_SYS_ENV=local precedence over the public test identity so the no-Docker regression job uses SQLite; scripts/e2e-regression.mjs additionally pins DATA_BACKEND=sqlite because SYS_ENV=test would otherwise select DynamoDB. It also drops NUXT_APP_CDN_URL / NUXT_PUBLIC_CDN_URL when they come from a local .env.test: that CDN doesn’t hold the fresh build’s JS chunks, so the app would never hydrate. Set them explicitly to keep them. @tgmc/web build inputs include SYS_ENV and NITRO_PRESET so Nx caches do not collide.

Workflows

Workflow Purpose
.github/workflows/ci.yml PR-only local parity (Nx affected + Nx Cloud, Node 24, SYS_ENV=local → .output/local). Skips version-bump-only PRs. On PRs to development, when web/docker are affected, also runs SYS_ENV=development npx nx run @tgmc/web:build (.output/development). No e2e, no dry-run. workflow_dispatch (no inputs) exists for the version-bump workflows: on a ci/version-bump-* branch it short-circuits green; on any other ref (a release head) it diffs against main. Exposes deployable via workflow_call.
.github/workflows/regression-gate.yml PRs to main (and dispatch on a release head) only: job Regression gate (scripts/regression-gate.js, see Regression gate) — runs no tests, fails when the release version bump has not landed, then waits for the post-merge Full regression (e2e) run on development and passes only when it succeeded (≤ 45 min wait). Its own workflow, so PRs into development list no skipped gate check.
.github/workflows/regression.yml On push to development (and manual dispatch) only: Full regression (e2e) via npm run e2e:regression with SYS_ENV=test identity on node-server (serves .output/test; no Docker for the app in GHA; SQLite via E2E_STORE_SYS_ENV=local and DATA_BACKEND=sqlite, which scripts/e2e-regression.mjs sets); timeout-minutes: 35. Version-bump merges start no push runs (GITHUB_TOKEN). Never runs in PRs.
.github/workflows/cd-aws-test-dry-run.yml On push to development only (nowhere else): one job — deployable gate, then AWS Test SAM dry-run (sam-dry-run.mjs test) when deployable. No full CI, no deploy, no GitHub Environment. Replaces the old cd-development.yml name.
.github/workflows/version-bump.yml Version bump (development) — push → development: patch-only bump PR, landed by .github/actions/land-version-bump. Skipped while a release PR (development → main) is open, so a release gets one bump.
.github/workflows/version-bump-release.yml Version bump (release) — PR → main: minor|major bump PR into the release PR head (the release/v* branch), landed by .github/actions/land-version-bump before the main merge. Never downgrades; no-op when the head already carries the release version.
.github/workflows/version-sync.yml Version sync — push → main: when main’s version is above development’s (a release landed its bump on the release branch), opens chore: bump version to <main version> (release sync) into development through the same land action. No-op otherwise; never downgrades.
.github/workflows/cd-aws.yml On main (or manual): deployable gate → SAM deploy to test + S3 sync + smoke → on success production SAM dry-run (sam-dry-run.mjs production, no deploy). Does not re-run full CI. Manual workflow_dispatch always deploys test then dry-runs prod.
.github/workflows/cd-aws-prod.yml On Release published (non-prerelease): SAM deploy production from the release tag + Environment reviewers. Manual workflow_dispatch still requires confirm=deploy.
.github/workflows/sync-tamaramack-github-io.yml Exports the tracked main source commit and opens or updates a destination PR from automation/portfolio-sync into development, replacing only apps/portfolio/. Automatic pushes require TAMARAMACK_GITHUB_IO_SYNC_ENABLED=1; manual dispatch remains available.
.github/workflows/update-agents.yml Path-filtered AGENTS.md regen on development/main (Node 24, contents: write)
.github/workflows/pages.yml Publishes docs/ to GitHub Pages (Jekyll Cayman) on push to development/main. The github-pages environment must allow both branches (npm run gh:setup-environments); otherwise the deploy job fails immediately with empty logs.

Portfolio synchronization to tamaramack.github.io

.github/workflows/sync-tamaramack-github-io.yml publishes this repository’s tracked portfolio snapshot to tamaramack/tamaramack.github.io through a destination pull request. It uses git archive at the triggering source commit, records its UTC timestamp for deterministic destination commits, recreates automation/portfolio-sync from destination development, and opens or updates the matching pull request. Run-specific URLs appear in the pull-request body, not the snapshot provenance, so same-commit retries remain unchanged. It never deploys the destination directly.

Ownership boundary:

Owner Path or responsibility
Source repo Exports all tracked content at the triggering commit. Untracked files and working-tree state are excluded.
Source repo Replaces the complete destination apps/portfolio/ snapshot; that directory is disposable sync output.
Destination Owns apps/profile/, the GitHub Pages root. The source workflow and sync helper must never change it.
Destination Owns repository validation, promotion, and deployment workflows after the generated pull request is opened.

The sync additionally fails closed unless the destination is the expected repository and branch, apps/profile/ exists, protected ancestors are not symbolic links, and every path in the entire staged diff is beneath apps/portfolio/. After opening or updating the PR, it retries for up to 45 seconds to confirm destination PR checks were created; it does not wait for them to pass.

Source repository setup:

  1. Create a fine-grained personal access token limited to the destination repository tamaramack/tamaramack.github.io only. Grant Contents: Read and write, Pull requests: Read and write, and Metadata: Read; grant no broader repository access.
  2. Store it in this source repository as the Actions secret TAMARAMACK_GITHUB_IO_SYNC_TOKEN.
  3. Use the source repository Actions variable TAMARAMACK_GITHUB_IO_SYNC_ENABLED as the automatic-run switch. Only the exact value 1 enables runs caused by pushes to source main; workflow_dispatch is allowed regardless of the variable and still requires the token.

Rollout:

  1. Merge the workflow while automatic push runs remain gated by leaving TAMARAMACK_GITHUB_IO_SYNC_ENABLED unset or set to a value other than 1.
  2. Configure TAMARAMACK_GITHUB_IO_SYNC_TOKEN and the TAMARAMACK_GITHUB_IO_SYNC_ENABLED repository variable, keeping the variable non-1 during the first validation if it is newly introduced.
  3. Run Sync portfolio to tamaramack.github.io manually from Actions, selecting the desired source ref in GitHub’s workflow dispatch control (there is no separate ref input).
  4. Inspect the generated destination pull request, confirm that only apps/portfolio/ changed, allow destination checks to pass, and merge it according to destination policy.
  5. Set or retain TAMARAMACK_GITHUB_IO_SYNC_ENABLED=1 for future source main pushes.

Recovery:

Site media in the GitHub.io copy

The site’s own media stays tracked in core/web/public/{i,v,d} (owner decision 2026-10-17), so the git archive snapshot copies it like any other tracked file and the copy serves it at the same /i, /v, /d paths; the default og:image is the tracked /i/portfolio/manhattan-2150-keyframe.png, absolute on the copy’s site URL (site-media.md). The snapshot carries no catalog media (m/ blobs never go in git), and the copy has no media server to ask for a URL, so catalog (new / admin) media would load from the production CDN:

Retired (outside four-env topology; files removed):

CI skip rules (version-bump PRs)

ci.yml short-circuits (job still succeeds — required status checks stay green) when any of:

Automated bump PRs use the ci/version-bump-* branch + skip-ci label. Do not put [skip ci] in the commit message: GitHub skips the whole workflow for that HEAD commit, so required checks never report and auto-merge stays blocked.

Why the bump workflows start CI themselves. A PR opened or updated with GITHUB_TOKEN gets pull_request runs that wait for approval (action_required, “Approve workflows to run”). While one is pending, GitHub ignores every other check on that commit — even a green dispatched Local parity — so the PR shows zero checks and stays BLOCKED (bump PRs used to sit like that until someone approved the run). Pushes and merges made with GITHUB_TOKEN start no push workflows. .github/actions/land-version-bump therefore:

  1. opens the bump PR, or reuses an open same-repository one for that branch (a closed PR from an earlier run, or a fork PR with the same branch name, does not count). The patch path (yield-to-release) instead deletes its branch and stops when a release PR opened while it ran: the release bump supersedes it;
  2. keeps one bump PR per base — the highest version wins (sort -V): lower open ci/version-bump-* PRs are closed as superseded, and the run closes its own PR when a higher one is open;
  3. gets CI onto the bump head (ensure_ci; Local parity short-circuits green there): waits up to 2 min for the head’s pull_request CI run and approves it when it is action_required. If approval is refused, it deletes that pending run. It runs gh workflow run ci.yml --ref <bump branch> only when no pull_request run will report. Then it enables gh pr merge --auto --squash --match-head-commit <pushed sha>;
  4. waits up to 15 min for the merge. Behind the base → updates the branch, gets CI onto the new head the same way and re-arms auto-merge. Conflict (another bump landed first) → disables auto-merge and fails: re-run the workflow run, which bumps from the new base head and updates the same PR. Closed (superseded) → success. Timeout → disables auto-merge and fails;
  5. if a release PR (<base> → main) is open, cancels its stale pull_request CI and Regression gate runs, then gets ci.yml and regression-gate.yml onto the new release head the same way (approve a pending run, else dispatch on <base>), so Local parity and Regression gate report there.

The bot merge itself starts no push workflows, so a version-only commit never re-runs regression, the dry-run or another bump.

Regression gate

Job Regression gate (regression-gate.yml → scripts/regression-gate.js) is the main ruleset’s second required check. It runs on PRs into main and on dispatches outside ci/version-bump-*, installs nothing and runs no tests:

  1. Release bump landed. It resolves the release level for origin/main → head (same range as version-bump-release.yml) and fails unless the head version is at least main’s version bumped by that level — development patch bumps alone (1.6.2 for a 1.7.0 release) do not count. A release with no releasable commits needs no bump.
  2. Regression green. It waits (polling every 30 s, ≤ 45 min) for the newest Full regression (e2e) run on development for the head’s newest commit that is not a version bump, or for any verified bump commit above it (subject chore: bump version to and only package.json / package-lock.json / core/web/package.json changed — a feature PR titled like a bump still needs its own run). When that commit is a release commit (Release-Source: <sha> trailer, see Release PR), the gate reads the development commit it names instead, and fails if the two trees are no longer identical (someone pushed to the release branch). It passes only when that run succeeded. Transient GitHub API errors are retried (5 in a row fail the check).
Gate says Do
Release bump has not landed Wait for Version bump (release); if it failed, fix it and re-run it. It re-runs this check after landing the bump.
Full regression (e2e) … is failure Fix development (new PR → merge → regression re-runs), or re-run the failed regression run if it was flaky; then re-run CI.
No Full regression (e2e) run gh workflow run regression.yml --ref development, then re-run the gate once it is green.
Release commit … no longer matches The release branch got a commit of its own. Land the change in development, then re-run npm run release:pr.
still running after 45 min Re-run the gate after the regression run finishes.

Waiting up to 45 minutes is expected right after a merge into development: the gate holds the release PR until that merge’s regression run finishes.

Automatic semver bumps

scripts/resolve-version-bump.mjs scans conventional commits and applies a channel policy:

Channel When Policy
development Push/merge to development Any releasable change → patch only; chore/docs/… → none
main PR targeting main major if breaking; otherwise minor (patch elevated to minor)
full Default / tests Raw conventional level (patch/minor/major/none)

Raw conventional mapping (before channel policy):

  1. none — only chore/docs/ci/test/style
  2. patch — fix, perf, refactor, build, deps
  3. minor — feat
  4. major — BREAKING CHANGE footer or ! in the conventional header

Commit range: git log --first-parent <base>..<head> — each merged PR counts once, and histories merged in from elsewhere (the vendored Foundation history carries upstream BREAKING CHANGE: footers) cannot drive the bump. A GitHub merge commit (Merge pull request #N …) is classified by its PR title (first body line). Without first-parent, the 2026-10 release range read 18,148 commits instead of 313 and resolved a false major (2.0.0 instead of 1.7.0).

Release range (VERSION_BUMP_USE_BASE_VERSION=1, resolveReleaseBump): main is squash-only, so main..head reaches back past every earlier release. The current version comes from main, and the range starts at the head’s newest first-parent commit chore: bump version to <main version> — the bump that main last shipped (on development, the Version sync commit). Without that commit the whole main..head range counts. On a release branch the range is just the release commit, whose subject npm run release:pr derives from this same range on development.

development path (version-bump.yml): push ci/version-bump-{version} → PR into development → land action (above). Skipped while a legacy release PR (development → main) is open; a release/v* PR does not pause it, because its bump lands on the release branch.

main path (version-bump-release.yml): push ci/version-bump-{version} → PR into the release PR head (the release/v* branch) → land action, which then re-starts CI on the release head. Same-repo PRs only; rulesets block direct pushes to development/main.

Back to development (version-sync.yml): after the squash into main, the released version goes back to development as chore: bump version to <version> (release sync), so later patch bumps continue from it.

Not bumped: theme packages (@tgmc/theme*), packages/*, and web-e2e — they keep independent versions.

Loop prevention: GITHUB_TOKEN merges start no push runs; github-actions[bot] guards cover any other path (and version-bump.yml also skips chore: bump version to subjects); ci/version-bump-* + skip-ci label for CI short-circuit. Commit messages omit GitHub’s [skip ci] token so required workflows still run (needed for auto-merge).

How test CD works (no CI re-run)

cd-aws.yml on push to main:

  1. Job deployable-gate — scripts/ci-is-deployable.mjs (Nx affected + path allowlist); workflow_dispatch forces deployable=true
  2. Job deploy-test — needs: deployable-gate, environment: test, runs when deployable=true

Full lint/test/build already ran on the PR, and Full regression (e2e) ran after the merge to development. Require status checks on the main ruleset (Local parity, Regression gate) so a squash merge cannot land without green PR CI and a green post-merge regression. Squash creates a new SHA on main, so CD intentionally does not look up check-runs by commit.

Skip-CI trap on main squash

GitHub skips every workflow on a push when any commit message in that push contains [skip ci] / [ci skip] / [no ci] / [skip actions] / [actions skip].

main is squash-only. If the repository squash setting is Commit messages (COMMIT_MESSAGES), GitHub pastes every commit subject from the PR into the squash body. Old version-bump commits still say chore: bump version to … [skip ci], so a development → main release squash silently suppresses CD AWS Test (and Pages / Update AGENTS). That is why merges after mid-August 2026 produced no Actions runs on main.

Required repo setting (apply with npm run gh:setup-merge-settings):

Field Value
squash_merge_commit_title PR_TITLE
squash_merge_commit_message BLANK

Never put GitHub’s [skip ci] token in commit subjects (version-bump already uses the skip-ci label + branch short-circuit instead). Keep release PR titles free of those tokens.

Manual recovery when CD was skipped: gh workflow run "CD AWS Test" --ref main.

Nx Cloud and affected CI

The private ubuntu-latest CI runner has 8 GB RAM. The development Nuxt build exceeded Node’s previous 4 GB and 6 GB heap caps in PR #386. The web app now keeps Nitro route chunks separate for the local/development node-server preset, reducing the size of the single Rollup bundle; the AWS Lambda preset retains its single bundle. CI keeps a 6 GB heap limit to leave memory for DynamoDB and the operating system.

CI uses Nx Cloud remote cache (nxCloudId in nx.json) and:

npx nx reset
npm run build:libs
npx nx affected -t test,build --parallel=3 --exclude=@tgmc/web
# when @tgmc/web is affected:
npm run build --workspace=@tgmc/theme
npx nx run @tgmc/web:test
npx nx run @tgmc/web:build
npx nx affected -t test:build-matrix --parallel=1

after nrwl/nx-set-shas (full git history via fetch-depth: 0) and, on PRs to development when web/docker are affected, the separate SYS_ENV=development Nuxt build. The nx reset + build:libs pair before Affected drops a poisoned in-job local cache left by that Development step; web is excluded from the parallel graph so a theme remote-cache restore cannot clear theme/core/dist mid-Nitro (see docs/solutions/build-errors/nx-local-cache-utilities-dist-enoent.md). The next step, Affected build matrix, runs test:build-matrix (today only @tgmc/nuxt-layer-ui): real nuxt builds of an all-features-on and an all-features-off fixture app, one at a time, about 45 s (nuxt-layer-ui.md § Development). It is not part of npm test or the pre-push hook. CI also runs node --test on scripts/ci-is-deployable.test.mjs, scripts/resolve-version-bump.test.mjs, scripts/cicd-workflow-policy.test.mjs, scripts/nitro-output-dir.test.mjs, scripts/e2e-regression.test.mjs, scripts/regression-gate.test.js, scripts/release-pr.test.js, scripts/sam-build.test.mjs, scripts/sam-deploy-flow.test.mjs, and scripts/sam-aws-helpers.test.mjs, and scripts/sam-template.test.js (parses infra/sam/template.yaml: Retain/PITR/versioning/CORS/throttle/domain), scripts/media-smoke.test.js (the media smoke script against a fake fetch), scripts/media-usage-rebuild.test.js (the usage rebuild script against a fake fetch), scripts/media-transfer.test.js (media:transfer argument parsing and exit codes), and scripts/lib/is-main.test.js / scripts/lib/aws-config.test.js (the shared script helpers). Lint excludes @tgmc/utilities, @tgmc/web-e2e, and @tgmc/media-player (pre-existing lint debt / vendor wrappers); their test/build still run when affected.

Repo secret (required): NX_CLOUD_ACCESS_TOKEN — create in Nx Cloud for this workspace and add under GitHub → Settings → Secrets and variables → Actions. CI fails fast if unset. Do not set NX_NO_CLOUD in workflows.

Deployable gate

scripts/ci-is-deployable.mjs sets deployable=true|false from:

Docs/AGENTS/prd-only changes → deployable=false → AWS Test dry-run and cd-aws.yml skip. workflow_dispatch always deploys test (then production dry-run). Production CD: GitHub Release published (non-prerelease) or manual cd-aws-prod.yml with confirm=deploy.

Dry-run matrix

When Workflow Dry-run target Deploy?
Push / merge to development only cd-aws-test-dry-run.yml (one job) test (sam-dry-run.mjs test) No
After successful CD AWS Test on main cd-aws.yml job sam-dry-run-production production (sam-dry-run.mjs production) No

No dry-run in pull requests. Test dry-run must not run on PRs, main, or prod CD. Production dry-run must not run on development.

development branch — AWS Test dry-run only

E2E-only affection (@tgmc/web-e2e) does not force a deploy.

How production approval is enforced

  1. Release path — cd-aws-prod.yml runs on release: published (skips prereleases). Deploys the tagged commit (github.event.release.tag_name). Still requires GitHub Environment production reviewers.
  2. Manual path — workflow_dispatch requires input confirm=deploy
  3. GitHub Environment production — required reviewers (PROD_REVIEWERS), prevent_self_review when supported

Release PR

npm run release:pr (scripts/release-pr.js) opens the PR into main. Because main is squash-only, it never receives development’s commits, so a PR straight from development lists all of them (10,000+, including the vendored Foundation history) even when it changes a few dozen files. The release PR has one commit:

  1. Fetches origin/main and origin/development.
  2. Refuses when main changed files after the last release that development does not match (a fix landed straight on main): the release would revert them. Land the same change in development, then re-run. The last released development commit comes from the newest release title on main, else --source <sha>, else the head of the PR that made main’s tip.
  3. Resolves the release level like version-bump-release.yml and builds a commit with git commit-tree: tree = development, parent = main, subject feat(release): v<version> (development <sha>) (feat(release)!: for major, chore(release): when nothing is releasable), trailer Release-Source: <full sha>. Signed when commit.gpgSign is on. The working tree is never touched, so it runs from any checkout, uncommitted work included.
  4. Force-pushes release/v<version>, then opens the PR (or updates the open one); an open PR from another release/v* branch is closed as superseded.

Then the usual checks run: Version bump (release) lands the bump on the release branch, and the Regression gate follows Release-Source to the development e2e run. Squash-merge with the PR title (it names the source, which the next release reads). Version sync brings the version back to development.

npm run release:pr -- --dry-run builds the commit and prints the plan without pushing. Re-run the script after more work lands in development; it rewrites the release branch, and the bump is applied again. Never push commits to a release/v* branch by hand.

feature → PR → development (Local parity) → merge
     → Full regression (e2e) + CD AWS Test (dry-run) + patch bump PR (auto-merges)
     → npm run release:pr → PR release/v<version> → main (one commit; Local parity + Regression gate;
       minor|major bump PR auto-merges into the release branch first) → squash merge
     → CD AWS Test (auto deploy) → on success production dry-run; Version sync PR into development
     → tag v{version} on main (match package.json) → GitHub Release (Publish)
     → CD AWS Production (Environment approval) → SAM portfolio-prod

Create the release from a commit on main that has already passed test CD. Tag name convention: v1.0.10 (root package.json version). Prereleases do not auto-deploy; use workflow_dispatch with confirm=deploy if needed.

Production checklist (first deploy and every release)

One-time setup in GitHub Settings → Environments → production (script: PROD_REVIEWERS=<user> npm run gh:setup-environments):

Name Kind Required Notes
AWS_ROLE_ARN secret yes OIDC role trusted by repo:OWNER/REPO:environment:production (OIDC bootstrap)
NUXT_ADMIN_TOKEN secret yes Bearer secret for admin + /api/media/*; empty fails writes closed
MEDIA_PRESANITIZE_SECRET secret recommended Verifies the admin’s signed PDF / MP4 uploads; a different value from test; no spaces or quotes (Admin preSanitized secret). Required once the enforce step ships
SITE_URL var yes Canonical origin; must be public https:// (never localhost — sam-cd-deploy rejects it for prod and test)
CORS_ALLOW_ORIGIN var yes Exact front-door origin; not empty and not * (also the S3 CORS origin for uploads)
DOMAIN_NAME var optional Custom domain for the HTTP API (e.g. www.example.com). Needs CERTIFICATE_ARN; the two are validated as a pair
CERTIFICATE_ARN var with domain Regional ACM certificate in us-west-2 covering DOMAIN_NAME (must be ISSUED before deploy)
HOSTED_ZONE_ID var optional Route 53 zone → stack creates the alias A record. Omit to add the CNAME yourself at an external registrar (target = DomainTarget stack output)
API_THROTTLE_BURST var optional Positive integer, default 50 (HTTP API default-route burst)
API_THROTTLE_RATE var optional Positive integer, default 25 (requests/second)
MEDIA_UPLOAD_ORIGINS var optional Comma-separated exact origins (https://admin.example.com) allowed to PUT presigned uploads to the assets bucket, in addition to CORS_ALLOW_ORIGIN
MEDIA_SMOKE_ENABLED var optional 1 runs the post-deploy media smoke (see below) after the greet check; unset or anything else skips it
MEDIA_USAGE_REBUILD_ENABLED var optional 1 runs the post-deploy media usage rebuild (see below) right after the media smoke; unset or anything else skips it

Set DOMAIN_NAME, CERTIFICATE_ARN and HOSTED_ZONE_ID only as GitHub Environment variables on production, never as repository-level variables. Repository vars are visible to the test Environment too, so portfolio-test and portfolio-prod would both try to claim the same custom domain (the second AWS::ApiGatewayV2::DomainName create fails, and a shared Route 53 alias would flip-flop). Give test its own distinct values if it should get a domain.

Blank optional vars are omitted from the deploy, so the template defaults apply and (with no domain vars) the stack creates no domain resources. Test (cd-aws.yml) receives the same optional vars from the test Environment.

Before the first deploy (the full ordered steps are in the production first-deploy runbook):

  1. Required reviewers on the production Environment (prevent_self_review); on plans without that feature, rely on the confirm=deploy manual path and the Actions approval UI.
  2. Request/validate the ACM certificate in us-west-2 (API Gateway regional domains use a certificate in the API’s own region, unlike CloudFront’s us-east-1).
  3. Confirm the test stack deployed from the same commit (the release flow tags a commit that already passed test CD).
  4. After deploy: create the DNS record (alias is automatic with HOSTED_ZONE_ID; otherwise CNAME → DomainTarget), then curl https://<domain>/api/greet.
  5. Optional: set MEDIA_SMOKE_ENABLED=1 on the Environment. cd-aws.yml and cd-aws-prod.yml then run scripts/media-smoke.js right after the greet check, against the stack’s HttpApiUrl with the NUXT_ADMIN_TOKEN secret: it presigns a 1x1 PNG upload, PUTs it, completes the ingest, lists it, waits for its on-ingest processing (the detail’s processing steps; on AWS this proves the S3 upload trigger and the worker end to end, and a failed, stalled or not-started step fails the run), materializes the t thumbnail (polling the job), and deletes the asset. Every asset it creates is tagged smoke and removed on failure too. Each request times out after 30 s (MEDIA_SMOKE_REQUEST_TIMEOUT_MS overrides) so the step cannot hang the job; if an upload succeeded but complete failed, the failure names the orphaned object key and the next presign of that name reclaims it. Locally the same script runs against docker:test with npm run docker:test:smoke; against any stack: BASE_URL=https://… ADMIN_TOKEN=… npm run media:smoke. It is a verification probe of the deployed stack, not media processing, and the only media step a workflow may contain (No media work in GitHub Actions).
  6. Optional: set MEDIA_USAGE_REBUILD_ENABLED=1 on the Environment. cd-aws.yml and cd-aws-prod.yml then run scripts/media-usage-rebuild.js right after the media smoke step (it runs whether or not the smoke is enabled), against the stack’s HttpApiUrl with the NUXT_ADMIN_TOKEN secret: POST /api/media-usage/rebuild rebuilds the usage list (which pages use which assets) from every published post and every slot with an asset, and prints media usage rebuild: OK (added N, removed N, countersRepaired N). It exits 1 on a non-2xx answer (for example 503 Media library unavailable) or a network failure, without printing the token; each request times out after 60 s. It is idempotent: recovery is to re-run the failed job or step, or press Rebuild usage list on the admin Media page (admin § Used by, refusal and Delete anyway). Locally: BASE_URL=https://… ADMIN_TOKEN=… node scripts/media-usage-rebuild.js. It reads the stored posts and slots and processes no media; with the smoke it is one of the two media-related steps a workflow may contain (No media work in GitHub Actions). Enable it on test and production. On Lambda the web awaits each write’s live usage rows for at most 5 s before it answers (core/web/server/plugins/media-usage-wait.ts, site media § Usage list writes); rows whose write the bound cut stay pending (deletes of those assets are refused) until this rebuild or the admin button settles them.

Release flow: the recommended release flow above — squash to main → test CD → tag v{version} → publish GitHub Release → approve the production Environment → portfolio-prod.

Data protection (what a bad deploy cannot destroy):

Rollback: runbook step 8 (CloudFormation, a previous release, table PITR, object versions).

Production first-deploy runbook

The owner runs these steps in order from a workstation, for the first portfolio-prod deploy and again for every release. Each step has its commands, the output to expect, and what to do when it fails; do not move to the next step until the current one passes. Commands are bash, run from any up-to-date checkout of this repo (they read origin/* and never touch the working tree). Region is us-west-2. <ver> is the release version (for example 1.7.0); <n> is a pull-request number.

Site media stays in core/web/public (/i, /v, /d) and ships with every deploy, so production needs no media import. Seeding the catalog (m/*) with npm run media:transfer is optional.

0. Preconditions

  1. Tools and sign-in. aws --version (v2), sam --version, gh --version, node --version (24). Then:

    aws sts get-caller-identity --query Account --output text   # the production account id
    gh auth status                                              # Logged in to github.com
    npm ci && npm run build:libs                                # only needed for media:transfer
    

    If it fails: aws sso login / set AWS_PROFILE; gh auth login.

  2. OIDC role. The account has the GitHub OIDC provider and a deploy role trusted by repo:OWNER/REPO:environment:production (OIDC bootstrap). The preflight sees only that the AWS_ROLE_ARN secret exists; it cannot read the role’s trust policy, so check the trust condition in IAM once.

  3. production Environment. Create it with reviewers, then set the secrets and variables from the checklist table:

    PROD_REVIEWERS=<login>[,<second-login>] npm run gh:setup-environments
    gh secret set AWS_ROLE_ARN --env production            # prompts for the value
    gh secret set NUXT_ADMIN_TOKEN --env production
    gh variable set SITE_URL --env production --body "https://<front-door host>"
    gh variable set CORS_ALLOW_ORIGIN --env production --body "https://<front-door host>"
    gh variable set DOMAIN_NAME --env production --body "<host>"            # optional, with CERTIFICATE_ARN
    gh variable set CERTIFICATE_ARN --env production --body "arn:aws:acm:us-west-2:…"
    gh variable set HOSTED_ZONE_ID --env production --body "<zone id>"      # optional
    gh secret list --env production && gh variable list --env production
    

    Expected: the lists show AWS_ROLE_ARN, NUXT_ADMIN_TOKEN (names only), SITE_URL, CORS_ALLOW_ORIGIN, and the domain variables you chose. Set the domain variables on the Environment, never at repository level.

    • Self-review deadlock: the script turns on prevent_self_review, so the person who publishes the Release cannot approve its deployment. With one reviewer, either add a second reviewer, have that reviewer publish the Release, or untick Prevent self-review on the Environment in the GitHub UI. The preflight fails when the only reviewer is the signed-in gh user with self-review blocked; if the Environment lets admins bypass its rules (can_admins_bypass) it warns instead, because the only way through is a bypass that deploys with no review at all.
    • CORS: production uses the exact SITE_URL origin (scheme + host, no path, never *). CD fails closed on an empty or * value.
    • SITE_URL without a custom domain: CD needs a public https:// SITE_URL before the stack exists, and the API URL is only known after the first deploy. For the first deploy set both variables to the placeholder origin https://portfolio-prod.invalid (.invalid is reserved by RFC 6761 and never resolves, so nobody can own it):

      gh variable set SITE_URL --env production --body "https://portfolio-prod.invalid"
      gh variable set CORS_ALLOW_ORIGIN --env production --body "https://portfolio-prod.invalid"
      

      Until the second deploy in step 6.2 points both at the stack’s HttpApiUrl, the site’s canonical and og:url tags name the placeholder, the HTTP API CORS and the bucket’s S3 CORS (presigned admin uploads) allow only the placeholder origin, so browser uploads from the real origin fail. Do not announce the site before step 6.2.

    • No userinfo: SITE_URL and CORS_ALLOW_ORIGIN never contain user:pass@; CD and the preflight reject it, and the preflight redacts it from its output.
  4. Media smoke on test first. gh variable set MEDIA_SMOKE_ENABLED --env test --body 1. Turn it on for production (same command with --env production) only after it passes on test in step 3.

  5. Test stack health before promoting. The media pipeline (#322) adds an S3 notification to the test bucket; CloudFormation replaces the bucket’s whole notification config, so any hand-made notification would be lost silently:

    npm run prod:preflight -- --stack portfolio-test --skip-sync
    

    Expected: OK: … with bucket-notification either empty or the managed m/ → media-queue rule. CORS_ALLOW_ORIGIN on test may be * (a warning, not a failure). If it fails: fix the named check; for an unexpected notification, record it and decide with the owner before promoting.

1. Confirm development is green

gh run list --workflow regression.yml --branch development --limit 1 --json conclusion,headSha,url
gh run list --workflow cd-aws-test-dry-run.yml --branch development --limit 1 --json conclusion,headSha,url
git fetch origin && git rev-parse origin/development

Expected: both runs "conclusion": "success" on the headSha that equals origin/development. If it fails: fix on a feature branch → PR into development; never push to development or main.

2. Promote development → main

npm run release:pr -- --dry-run     # Release release/v<ver>: minor (main <old> → <ver>) … Dry run: nothing pushed.
npm run release:pr                  # prints the release PR URL
gh pr checks <n> --watch            # Local parity, Regression gate, version bump: all pass
gh pr merge <n> --squash            # PR title only (it names the release source)

Expected: the dry run names the bump and versions; the PR has one commit; the checks pass and the bump commit lands on the release branch before the merge. If it fails: main already matches development means nothing to release. A red Regression gate means the development e2e run for that source failed; fix forward on development and re-run npm run release:pr (it rewrites the release branch). Never push to release/v*, main or development by hand.

3. Verify the test stack

gh run list --workflow cd-aws.yml --branch main --limit 1 --json databaseId,headSha,status
gh run watch <databaseId> --exit-status
git fetch origin && git rev-parse origin/main            # must equal the run's headSha

Expected: CD AWS Test succeeds with SAM deploy (test) (including Post-deploy smoke (GET /api/greet + SITE_URL) and, with MEDIA_SMOKE_ENABLED=1, a log ending media smoke: OK) and SAM dry-run production (no deploy). A Version sync PR into development follows.

Then check the stack as a user:

TEST_URL=$(aws cloudformation describe-stacks --stack-name portfolio-test --region us-west-2 \
  --query "Stacks[0].Outputs[?OutputKey=='HttpApiUrl'].OutputValue" --output text)
TEST_URL="${TEST_URL%/}"
curl -sfS "$TEST_URL/api/greet"                              # {"message":…}
npm run prod:preflight -- --stack portfolio-test --skip-sync  # OK, bucket-notification = managed config

Open $TEST_URL in a browser: the home page renders, navigation works, page source has siteUrl / og:url / canonical on the public https:// origin (never localhost), and /admin/media lists the catalog. Remember the headSha: it is the commit you tag in step 5.

If it fails: read the failed step’s log (gh run view <databaseId> --log-failed). The deploy step’s CloudFormation errors: aws cloudformation describe-stack-events --stack-name portfolio-test --region us-west-2 --max-items 30. Do not release; fix forward through development. Known open item: images under /_ipx answer 403 on test (TODO); decide whether it blocks the release.

Optional cleanup on test: once /aws/lambda/portfolio-test-nitro-logs and …-media-worker-logs receive events, delete the old default groups (aws logs describe-log-groups --log-group-name-prefix /aws/lambda/portfolio-test --region us-west-2, then aws logs delete-log-group --log-group-name <old name>).

4. Preflight production

npm run prod:preflight              # human report; add -- --json for a machine-readable summary

Read-only: it uses describe/get calls, an aws s3 sync --dryrun against the bucket, and gh api GETs (an allowlist refuses anything else before it runs), spawns aws and gh without a shell, and never prints secret values or URL credentials. --region / AWS_REGION, --environment, --stack, --repo and --public-dir (an existing directory) are validated first. Exit 0 = pass, 1 = a check failed (including aws CLI not found / gh not found), 2 = bad arguments. Expected on the first deploy:

prod preflight: portfolio-prod (us-west-2), GitHub OWNER/REPO environment production
first deploy: the stack does not exist yet
PASS  aws-credentials      account 000000000000, region us-west-2
PASS  stack                portfolio-prod does not exist: first deploy
PASS  leftovers            no tables, bucket, queues or log groups with the stack names
SKIP  queue-sse            first deploy: created by the deploy
SKIP  bucket-notification  first deploy: created by the deploy
SKIP  bucket-versioning    first deploy: created by the deploy
SKIP  tables-pitr          first deploy: created by the deploy
SKIP  sync-dryrun          first deploy: created by the deploy
PASS  github-reviewers     reviewers: <login>
PASS  github-secrets       AWS_ROLE_ARN, NUXT_ADMIN_TOKEN set (names only)
PASS  github-variables     CORS_ALLOW_ORIGIN https://…, domain …, MEDIA_SMOKE_ENABLED=1
PASS  site-url             SITE_URL https://… is https and resolves
OK: 7 passed, 0 warnings, 0 failed, 5 skipped

With the https://portfolio-prod.invalid placeholder (no custom domain), site-url is a WARN … does not resolve …; fine before the first deploy … and the verdict is OK: 6 passed, 1 warning, …. That warning is acceptable only before the first deploy; on a re-deploy it is a FAIL until step 6.2.

On a re-deploy, the stack checks run instead of leftovers: queue SSE, bucket notification (empty or exactly the managed config), versioning, PITR on all three tables, and the sync dry run (no deletes under m/, media/, i/, v/, d/). For an exact sync preview, stage the release build first (npm run sam:build -- production, from a checkout of the release commit); without it the check previews the --delete pass against an empty directory (worst case) and says so.

If it fails:

Check Fix
aws-credentials Sign in to the production account (step 0).
stack (ROLLBACK_COMPLETE, *_IN_PROGRESS) Wait for in-progress operations; a ROLLBACK_COMPLETE stack must be deleted first (step 8, failed first create).
leftovers Retained tables or bucket from an earlier failed create. Empty ones: delete them. With data: keep them and import them into the stack.
bucket-notification Someone added a notification by hand; CloudFormation will replace it. Move it into infra/sam/template.yaml or remove it.
sync-dryrun The sync would delete protected media. Do not deploy; fix the excludes in scripts/sam-aws-helpers.mjs (PUBLIC_ASSET_PROTECTED_PREFIXES).
github-*, site-url Set the missing reviewer, secret or variable (step 0). A site-url warning before the first deploy is fine when the deploy or a DNS record creates the host.

5. Publish the Release

gh release create v<ver> --target <headSha from step 3> --title v<ver> --generate-notes
gh run list --workflow cd-aws-prod.yml --limit 1 --json databaseId,status,headBranch

Expected: the release is published (not draft, not prerelease) on the commit that passed test CD, and CD AWS Production shows Waiting for the production review. Approve it in Actions → the run → Review deployments (as a reviewer who did not publish, if self-review is blocked). Then gh run watch <databaseId> --exit-status.

If it fails: a prerelease or draft does not deploy; edit the release or run gh workflow run cd-aws-prod.yml --ref v<ver> -f confirm=deploy. A wrong tag target: delete the release and tag (gh release delete v<ver> --cleanup-tag) and recreate. A failed deploy: step 8.

6. Verify production

Run 6.0 → 6.3 in order: the SITE_URL checks only pass once 6.1 / 6.2 are done.

6.0 The stack is up.

aws cloudformation describe-stacks --stack-name portfolio-prod --region us-west-2 \
  --query "Stacks[0].[StackStatus,Outputs[].[OutputKey,OutputValue]]" --output text
API_URL=<HttpApiUrl output without the trailing slash>
curl -sfS "$API_URL/api/greet"                                    # {"message":…}
curl -sSI "<AssetsCdnUrl output>/favicon.ico" | head -n 1        # HTTP/2 200 from CloudFront

Expected: CREATE_COMPLETE (first deploy) or UPDATE_COMPLETE; the outputs include HttpApiUrl, the three table names, MediaQueueUrl, AssetsBucketName, AssetsCdnUrl, AssetsDistributionId (and DomainTarget with a domain); the run log shows the greet smoke and, if enabled, media smoke: OK.

6.1 DNS (custom domain): with HOSTED_ZONE_ID the alias record is created; otherwise add a CNAME from DOMAIN_NAME to the DomainTarget output, then curl -sfS https://<domain>/api/greet.

6.2 SITE_URL second pass (no custom domain, first deploy only): replace the placeholder and redeploy the same tag:

gh variable set SITE_URL --env production --body "$API_URL"
gh variable set CORS_ALLOW_ORIGIN --env production --body "$API_URL"
gh workflow run cd-aws-prod.yml --ref v<ver> -f confirm=deploy      # then approve, and gh run watch

Until this pass, the preflight’s site-url check fails (the placeholder never resolves, and the stack now exists); every other check must already pass.

6.3 Verify as a user, with PROD_URL = https://<DOMAIN_NAME> or $API_URL:

curl -sS "$PROD_URL/" | grep -oE 'siteUrl"?:"[^"]*"'               # siteUrl:"<PROD_URL>", never localhost or the placeholder
npm run prod:preflight                                            # every check PASS (re-deploy checks)
IPX=$(curl -sS "$PROD_URL/work/spatial-experiences" | grep -oE '/_ipx/[^"'"'"' ,]+' | head -n 1 | sed 's/&amp;/\&/g')
if [ -z "$IPX" ]; then
  echo "FAIL: no /_ipx image found on /work/spatial-experiences"     # never curl "$PROD_URL" alone: the home page would answer 200
else
  echo "$IPX"                                                     # a NuxtImg URL, e.g. /_ipx/…/i/portfolio/tesseract/….png
  curl -sS -o /dev/null -L -w '%{http_code} %{content_type} %{url_effective}\n' "$PROD_URL$IPX"   # 200 image/…
fi

Expected: siteUrl is the public origin; the preflight ends OK: … 0 failed; the /_ipx image answers 200 with an image/* type (after any redirect to the CDN). FAIL: no /_ipx image found is a defect too: the page no longer renders NuxtImg images (or the grep missed them), so check the page source before releasing. In a browser: home, every nav item, a case study (its images load), /docs, and /admin/media sign-in; console without errors; og:url and canonical on the public origin.

A 403 (or any 4xx/5xx) on the /_ipx image is a defect. The same 403 is open on test (TODO: deployed test images). The owner fixes it before announcing the release, or explicitly accepts it in writing (in the release notes or the TODO entry), per the repo rule that no error status is “expected” without the owner’s say-so.

6.4 Follow-ups:

  1. Media base URL: set PRODUCTION_MEDIA_BASE_URL to the AssetsCdnUrl output in a follow-up PR (TODO).
  2. Optional: seed the catalog with npm run media:transfer.

If it fails: siteUrl on localhost, the placeholder or the wrong host means SITE_URL is wrong; fix the variable and redeploy the tag. Lambda errors: aws logs tail /aws/lambda/portfolio-prod-nitro-logs --since 30m --region us-west-2 (media: …-media-worker-logs). Anything worse: step 8.

7. After every release

Merge the Version sync PR into development when its checks pass, and update the TODO with anything found in steps 3–6.

8. Roll back

Tables and the assets bucket are Retain with PITR and versioning (data protection), so none of these steps loses data.

Promote media to production

Catalog media is not deployed by CD (the sync excludes m/* and media/*). To seed a stack’s catalog from a local one, run npm run media:transfer from a workstation. Swap --from and --to to pull a backup. The tool copies the catalog rows (DynamoDB) and the objects (S3) with the same ids, names and paths, so media URLs are identical in every environment. Rules: media.md § Transfer between backends.

Before you start

aws cloudformation describe-stacks --stack-name portfolio-prod --region us-west-2 \
  --query "Stacks[0].Outputs[?OutputKey=='MediaTableName' || OutputKey=='AssetsBucketName'].[OutputKey,OutputValue]" --output text

Which local catalog. sqlite:<dir> is a catalog folder holding media.sqlite and blobs/. Each app uses MEDIA_LOCAL_ROOT when it is set, otherwise:

App Run as Catalog folder
Admin (core/admin) Docker admin Compose (host ./data mounted at /app/data) data/admin-media
Admin nuxt dev in core/admin core/admin/data/admin-media (<cwd>/data/admin-media)
Web (core/web) npm run dev (Nuxt runs in core/web) core/web/.data/media (<cwd>/.data/media)
Web npm run docker:local data/media
Web npm run docker:dev with a non-DynamoDB backend named volume tgmc-dev-media-data (copy it to a folder first)

Endpoints. sqlite:<dir> (above), aws:<table>@<bucket> (a stack’s media table and assets bucket), and postgres:<postgres://… URL> for a PostgreSQL catalog with its objects in --blobs fs:<dir> or --blobs s3:<bucket> (--from-blobs / --to-blobs when both sides are Postgres; --s3-endpoint / --path-style for MinIO). A Postgres source must already have the media tables; a real Postgres target gets them on open, a dry-run one is never created. Passwords in the URL are masked in all output, but prefer PGPASSWORD so they stay out of shell history. Details: media-postgres § Transfer endpoint.

Promote. Every step has a gate: do not go on until it passes.

SRC=sqlite:./data/admin-media                       # the catalog folder from the table above
DST=aws:portfolio-prod-Media@<AssetsBucketName>

# 0. Stop the app that owns the source catalog, then back the folder up. Opening a SQLite
#    source runs its schema migrations in place, so keep an untouched copy.
mkdir -p ./backups && cp -a "${SRC#sqlite:}" "./backups/$(basename "${SRC#sqlite:}")-$(date +%Y%m%d-%H%M)"

# 1. Dry run: reads both sides, writes nothing.
npm run -s media:transfer -- --from "$SRC" --to "$DST" --dry-run > transfer-dry-run.json; echo "exit $?"
# 2. Real run: same arguments, without --dry-run. Keep the report.
npm run -s media:transfer -- --from "$SRC" --to "$DST" > transfer.json; echo "exit $?"

# 3. Smoke the stack's media API.
BASE_URL=https://<domain> ADMIN_TOKEN=<NUXT_ADMIN_TOKEN> npm run media:smoke

Backup pull (production into a new local catalog; also good for checking a promotion). Use read-only credentials: the pull never writes to its source, and read-only credentials make that a guarantee. The pull needs dynamodb:GetItem and dynamodb:Query on the media table (and its indexes), and s3:GetObject on m/* plus s3:ListBucket on the bucket.

npm run -s media:transfer -- --from aws:portfolio-prod-Media@<AssetsBucketName> --to "sqlite:./backups/media-$(date +%Y%m%d)"

Add --tenant <name> (repeatable: portfolio, example-organization) to pull only that tenant’s media, for example --to sqlite:./backups/ha --tenant example-organization; without it every tenant is copied (media.md § Transfer between backends).

A dry run never creates or migrates a sqlite: target: a missing target reads as empty, so everything is reported as would-copy.

Reading the report. stdout carries only the JSON report; library log lines go to stderr. Use npm run -s so npm’s banner stays out of the file. The report’s top level lists asset ids and children lists child ids, and sourceCount is the number of source assets seen. The exit code is 0 when nothing conflicted or failed, 1 otherwise (the report is still printed, even after a listing error), and 2 for bad arguments (with usage).

Local rehearsal: npm run docker:test, then run the transfer against aws:portfolio-test-Media@portfolio-assets --dynamodb-endpoint http://127.0.0.1:8000 --s3-endpoint http://127.0.0.1:4420 --path-style with AWS_ACCESS_KEY_ID=minioadmin AWS_SECRET_ACCESS_KEY=minioadmin, then npm run docker:test:smoke.

Local

cp .env.example .env          # SYS_ENV=local
npm run db:migrate:local
npm run dev                   # or npm run build

Development (Docker — local only)

Docker Compose is for local DynamoDB Local + MinIO + web/api stacks (Postgres opt-in) and Lambda test parity. Compose images are not built or used on GitHub Actions. The one container CI does run is a services: sidecar: amazon/dynamodb-local:2.5.2 in the local-parity job (below). Full regression (e2e) after merge to development (regression.yml) uses npm run e2e:regression: SYS_ENV=test identity, NITRO_PRESET=node-server, SQLite stores (migrated, plus one published fixture post from scripts/e2e-seed-posts.js — test data only, so the /blog/<slug> route is audited), Cypress on :4200, server entry .output/test/server/index.mjs. The suite checks the current primary navigation, every portfolio route at four viewport sizes, and the blog API store response. Do not set job-level NODE_ENV=production before npm ci — that omits root devDependencies (e.g. @nx/vite) and breaks postinstall → nuxt prepare.

cp .env.development.example .env.development
npm run docker:build:dev      # ensureNitroOutput(development) then web + api images
npm run docker:dev            # DynamoDB Local + MinIO + api (:4100) + web (:4200); exclusive with docker:local on :4200
npm run docker:dev:down

npm run docker:local          # DynamoDB Local + MinIO (DATA_BACKEND=sqlite for ./data); exclusive with docker:dev on :4200
npm run docker:local:down

# Full regression (same path as CI — no Docker required)
npm run e2e:regression

The regression runner requires a successful HTTP response (2xx) before starting Cypress. A server returning 500 stays unready and the timeout reports its last HTTP status, rather than producing a failure for every browser test.

The web and admin Nitro configurations bundle nuxt so Nuxt 4.6 renderer imports resolve their generated manifest/precomputed data instead of the package stubs. They also bundle @primeuix/utils so PrimeVue’s utility barrel and subpath exports resolve together; externalizing only the subpaths can produce missing-export errors when the built server starts. Validate these settings with the built-server regression command above, not only a successful build. Admin missed this when the lockfile moved to Nuxt 4.6 and returned 500 on every page in Docker; core/web/tests/nitro-externals.spec.ts and core/admin/tests/nitro-externals.spec.ts guard both inline sets.

Optional hot-reload profile (does not replace the built stack):

docker compose -f docker/docker-compose.dev.yml --project-directory . --profile hotreload up

Build images locally with npm run docker:build:dev (docker/Dockerfile.app with Compose args for api :4100 / web :4200). See docker/README.md.

Test / Production (SAM)

Canonical AWS region: us-west-2 (samconfig, CD workflows, .env.*.example).

npm run sam:build             # SYS_ENV=test → stage + sam build (fails hard unless SAM_BUILD_SOFT=1)
npm run sam:deploy:test
node scripts/sam-build.mjs production
npm run sam:deploy:prod

The SAM build launches Nuxt with a 6 GiB Node heap limit. Its server bundle can exceed Node’s default 2 GiB heap even when the workflow sets NODE_OPTIONS for the job.

Local machines without the SAM CLI can set SAM_BUILD_SOFT=1 to stage Nitro output and continue; CI/CD must never set this. Windows: install SAM via winget install -e --id Amazon.SAM-CLI (or the 64-bit MSI); enable long paths; see infra/sam/README.md.

Static assets (CloudFront + private S3)

Test (portfolio-test): hashed assets publish to the shared Example Organization/portfolio CDN (shared-cdn-test stack → s3://shared-cdn-test-assets-000000000000/ + https://cdn.example.com). Bucket root only (no portfolio/ prefix). Sync never uses --delete (would remove Example Organization objects; publisher IAM also omits s3:DeleteObject). Do not invalidate /* on that distribution. Stack-owned AssetsBucket / AssetsDistribution remain until cutover is fully verified; Lambda NUXT_*_CDN_URL imports shared-cdn-test-url.

Production (portfolio-prod): exclusive private S3 + CloudFront on the portfolio stack; sync uses --delete.

CD (scripts/sam-cd-deploy.mjs) and local npm run sam:deploy:*:

  1. Resolves NUXT_APP_CDN_URL: GitHub var → else shared-cdn-test export (test) / stack AssetsCdnUrl (prod) → else empty on pass 1
  2. Builds Nuxt with that CDN (if known), deploys the stack, then on first deploy rebuilds when CDN was unknown and redeploys
  3. Syncs infra/sam/.aws-sam/build-src/public → the CDN bucket root (no prefix). Test: shared bucket without --delete. Prod: stack assets bucket with --delete. Source is Nitro .output/public from scripts/sam-build.mjs. Every sync/repair pass excludes m/* (catalog blobs, m/{tenant}/{kind}/{name}.{ext}), the legacy media/* prefix and pending/* (presigned uploads waiting for complete), so CD never uploads, deletes or rewrites catalog media; the --delete pass and the repair also exclude the site media dirs i/*, v/*, d/* (their git-tracked files always upload through the typed passes; that split is permanent), so --delete can never remove media (No media work in GitHub Actions). Sync sets explicit Content-Type per extension (.js → text/javascript, .css → text/css, .mp4, .pdf, .html/.htm, …) and Cache-Control (public, max-age=31536000, immutable only for hashed _nuxt/**; public, max-age=3600, must-revalidate for everything else, including the unhashed _nuxt/builds/* manifest), and then REWRITES object metadata so identical-byte re-syncs cannot leave AWS CLI’s binary/octet-stream default in place (fatal under CloudFront nosniff for module scripts). On the shared test bucket the repair is limited to hashed _nuxt/* assets, so HyperActivity’s objects outside _nuxt/ keep their existing headers; the exclusive prod bucket is repaired everywhere outside the media prefixes (m/, media/, pending/, i/, v/, d/).
  4. Lambda runtime receives the same CDN URL from the template (NUXT_APP_CDN_URL / NUXT_PUBLIC_CDN_URL)

Local sync plan: npm run sam:sync:dryrun or node scripts/sam-sync-public.mjs --dryrun --shared. Heal existing wrong MIME without a rebuild: npm run sam:sync:repair-types (shared test CDN; never --delete). After a metadata-only heal, path-scoped CloudFront invalidation /_nuxt/* may be needed — never invalidate /* on the shared distribution. Docker docker:test does not emulate S3/CDN.

Set GitHub Environment test vars NUXT_APP_CDN_URL / NUXT_PUBLIC_CDN_URL to https://cdn.example.com so builds skip bootstrap guessing. Production still uses stack AssetsCdnUrl until a shared-cdn-production cutover is requested.

Test SITE_URL: set Environment test var SITE_URL to the stack HttpApiUrl (e.g. https://….execute-api.us-west-2.amazonaws.com, no trailing slash). cd-aws.yml no longer falls back to http://localhost:4200. When the var is empty or still localhost, scripts/sam-cd-deploy.mjs reads HttpApiUrl from the existing portfolio-test stack; if that is also missing (first-ever create), CD fails closed. Post-deploy smoke asserts the HTML siteUrl / og tags are not localhost and match HttpApiUrl. Background: aws-test-site-url-localhost.

Post-deploy smoke: CD curls GET {HttpApiUrl}/api/greet (HttpApiUrl uses the HTTP API $default stage — no /test or /prod path prefix).

No media work in GitHub Actions

Media never goes through a workflow: nothing in Actions ingests, enriches or transforms images, video or documents, and nothing touches the catalog (m/, media/). The only media files CD copies are the site’s own git-tracked ones under core/web/public/{i,v,d} (below).

Media storage deploy notes

What the admin / media hardening (2026-10) changes on a SAM deploy. One sam deploy carries the template and the code together; CloudFormation updates the IAM role and the bucket before the new function code serves traffic, so there is no manual ordering beyond deploying test before production.

Admin token

GitHub Environment secret NUXT_ADMIN_TOKEN reaches the Lambda as SAM parameter AdminToken → Lambda env NUXT_ADMIN_TOKEN / ADMIN_TOKEN, and only that way: it is runtime-only (review M5). cd-aws.yml / cd-aws-prod.yml pass it to the deploy step as ADMIN_TOKEN_SECRET (never as NUXT_ADMIN_TOKEN, which the Nuxt build reads); scripts/sam-cd-deploy.mjs puts it into the parameters and then removes ADMIN_TOKEN_SECRET / NUXT_ADMIN_TOKEN / ADMIN_TOKEN from the environment before the build runs, and core/web/nuxt.config.ts sets runtimeConfig.adminToken: '' (the server reads the env, resolveAdminToken). So .output never contains the token. scripts/cicd-workflow-policy.test.mjs fails if a deploy step sets NUXT_ADMIN_TOKEN or ADMIN_TOKEN. Empty = admin writes fail closed. It must be at least 32 characters and not a placeholder (openssl rand -hex 32): the server refuses a weak token like an empty one (401 with the reason), so check the secret before deploying this change (environment § ADMIN_TOKEN). npm run prod:preflight only sees that the secret exists, not its value.

Admin preSanitized secret

GitHub Environment secret MEDIA_PRESANITIZE_SECRET (and MEDIA_PRESANITIZE_SECRET_PREVIOUS during a rotation) on test and production. cd-aws.yml / cd-aws-prod.yml pass both to scripts/sam-cd-deploy.mjs, which sends them as the NoEcho SAM parameters MediaPresanitizeSecret / MediaPresanitizeSecretPrevious → Nitro Lambda env of the same names. The web verifies with them the HMAC the admin puts on the PDF and MP4 uploads it already sanitized (media-nuxt § Admin preSanitized signature). For now (expand phase) nothing is refused while either side lacks the secret; the web logs presanitize.no_key / presanitize.unsigned. A later step (Phase 7 S1-13) refuses unsigned admin PDF / MP4 uploads on Lambda and makes CD fail without the secret.

CORS

SAM parameter CorsAllowOrigin (default *). For production, set GitHub Environment var CORS_ALLOW_ORIGIN to the front-door origin(s) you serve from (single origin string). Leaving * in prod widens blast radius if a browser client ever carries Authorization.

The assets bucket’s S3 CORS uses the same origin (plus MediaUploadOrigins when set), allows GET/HEAD/PUT only (PUT is for presigned media uploads; POST is not allowed) and only the headers the presigned PUT needs (Content-Type, If-None-Match, Cache-Control, Content-Disposition, Content-MD5, x-amz-*; no bare * — scripts/sam-template.test.js checks the list against PRESIGN_REQUIRED_HEADERS in packages/media-aws/src/presign-headers.ts). The test stack still passes *, so its bucket stays open until CORS_ALLOW_ORIGIN is set for that Environment.

Details: infra/sam/README.md.

GitHub Environments + Rulesets setup

# Environments (test + production)
PROD_REVIEWERS=your-github-user npm run gh:setup-environments

# Branch Rulesets (PR-only development/main; print checklist, optional apply)
npm run gh:setup-rulesets
APPLY_RULESETS=1 npm run gh:setup-rulesets

Environments

Creates/updates:

Required reviewers need a GitHub plan that supports Environment protection rules (Team/Enterprise for private repos). On plans that return HTTP 422 for reviewers, keep the production branch/tag policy and gate prod with workflow approval in the Actions UI (and/or confirm=deploy on workflow_dispatch). The release: published path still runs under the production Environment.

Rulesets (branch protection)

Scripts: scripts/setup-github-rulesets.mjs. Recommended:

Ruleset Refs Key rules
devenv refs/heads/develop* Require PR; block force-push/delete; required check Local parity
main refs/heads/main Require PR; squash only; conversation resolution; checks Local parity + Regression gate

Admin/RepositoryRole bypass remains available for emergencies. If APPLY_RULESETS=1 returns 403/422, apply the checklist printed by the script in Settings → Rules → Rulesets.

Also create label skip-ci (the ruleset script attempts this) for version-bump PR detection.

Secrets / vars (per environment)

Repository-level (Actions):

Name Where Notes
NX_CLOUD_ACCESS_TOKEN Repository Actions secret Nx Cloud access token for CI remote cache

Prefer OIDC (no long-lived keys) for AWS deploy roles:

Name Where Notes
AWS_ROLE_ARN Environment secret IAM role trusted by repo:OWNER/REPO:environment:NAME
AWS_REGION Environment variable (preferred) or secret Default us-west-2 if unset
NUXT_ADMIN_TOKEN Environment secret Blog admin Bearer secret (test + production)
MEDIA_PRESANITIZE_SECRET / MEDIA_PRESANITIZE_SECRET_PREVIOUS Environment secrets (recommended; _PREVIOUS only during a rotation) Admin preSanitized HMAC keys, different per Environment (Admin preSanitized secret)
OPENAI_API_KEY Environment secret (optional) Required only when AI_LAB_LIVE_ENABLED=1
AI_LAB_SIGNING_SECRET Environment secret (optional) HMAC for AI Lab continuation tokens; required with live mode
NUXT_APP_CDN_URL Environment variable Test: https://cdn.example.com (shared-cdn-test). Prod: stack AssetsCdnUrl or override.
CORS_ALLOW_ORIGIN Environment variable (optional) HTTP API CORS origin; default *. Set for production.
DOMAIN_NAME / CERTIFICATE_ARN / HOSTED_ZONE_ID / API_THROTTLE_BURST / API_THROTTLE_RATE / MEDIA_UPLOAD_ORIGINS / MEDIA_SMOKE_ENABLED Environment variables (optional) See the production checklist.

OIDC bootstrap (account once)

  1. Create the GitHub OIDC provider: https://token.actions.githubusercontent.com (client ID sts.amazonaws.com).
  2. Create two roles with sts:AssumeRoleWithWebIdentity, conditioned on:
    • repo:OWNER/REPO:environment:test
    • repo:OWNER/REPO:environment:production
  3. Attach a deploy policy covering CloudFormation, SAM-managed services (Lambda, API Gateway, S3, DynamoDB, CloudFront, Logs), and IAM role management for CAPABILITY_IAM.
  4. Set each Environment secret AWS_ROLE_ARN to the matching role ARN. CD workflows already request id-token: write.

Trust policy sketch for the deploy role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Federated": "arn:aws:iam::ACCOUNT:oidc-provider/token.actions.githubusercontent.com" },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
        },
        "StringLike": {
          "token.actions.githubusercontent.com:sub": "repo:OWNER/REPO:environment:test"
        }
      }
    }
  ]
}

Use separate roles (or separate sub conditions) for environment:test and environment:production.

Node version

Floor: Node.js 24+. Local and tooling may use Node 24 through Current. Deploy/CI stay on 24 for Lambda and image parity.

Layer Pin
GitHub Actions (ci, cd-*, version-bump, update-agents) node-version: 24
Docker (Dockerfile.app / dev / production) node:24-bookworm-slim
SAM Lambda nodejs24.x
Local tip .nvmrc → 24 (or any >=24)
package.json engines + .npmrc >=24 + engine-strict=true

Node 20 is not supported. Installs fail under engine-strict if the runtime is below 24.

Why CI/Docker/Lambda stay on 24: Node 24 is Active LTS and available as nodejs24.x on Lambda. Newer Current releases (e.g. 26) are fine locally when engines allows >=24, but are not an AWS Lambda managed runtime yet. Node 20 Lambda runtime reached deprecation Phase 1 on 2026-04-30.

Nitro’s aws_lambda handler is async — compatible with Lambda’s Node 24 removal of callback-style handlers.