CI/CD & Environments

This guide covers the deployment pipeline, environment types, and automation workflows.

Environments

The monorepo supports three environment types:

Type Name Pattern Purpose Lifecycle
Production production Live user-facing environment Manual deploy via workflow_dispatch
Integration integration Staging / continuous integration Auto-deploys on merge to main
Ephemeral pr0001–pr9999 Per-PR preview environments Auto-created on PR, destroyed on close

Environment Differences

Aspect Production Integration Ephemeral
DynamoDB removal RETAIN DESTROY DESTROY
Log retention 1 year 2 weeks 2 weeks
Lambda minification Yes No No
Source maps No Yes Yes
Self-signup No No No
SSO sign-in Yes Yes No
Assets bucket Yes Yes No
Data seeding — — From integration, minus studio content
Docs publishing Yes (canonical) Yes (staging preview) No

CI/CD Workflows

Deployment Workflows

Workflow Trigger Environment
deploy-integration.yml Push to main; daily 05:00 UTC (fluency stream refresh) integration
deploy-production.yml Manual (workflow_dispatch) production
deploy-ephemeral.yml PR opened/updated ephemeral (per-PR)
destroy-ephemeral.yml PR closed; manual workflow_dispatch Destroys ephemeral stack
reconcile-ephemeral.yml Every 6h; manual workflow_dispatch Destroys orphaned stacks
prune-ai-games-branches.yml Every 6 h (03/09/15/21 UTC); manual workflow_dispatch Deletes games/pr*/* ai-games branches of dead ephemeral envs

Verification Workflow (verify.yml)

Runs on every PR (ready for review / synchronize):

  1. Lint — pnpm check --affected (lint + type-check across affected workspaces)
  2. Backend Tests — pnpm run test in apps/backend (with coverage)
  3. Check PR conventions — Validates conventional commit subjects and the pull request title

Deployment Pipeline (base-deploy.yml)

The reusable deployment workflow runs these jobs:

frontend-build ──┐
                  ├──► frontend-upload
infrastructure ──┘
     │
     └──► publish-docs (production + integration only)
  1. frontend-build — Installs deps, builds Vite app, uploads artifact
  2. infrastructure — Runs the deploy orchestration script (see Deploy Orchestration below)
  3. frontend-upload — Downloads build artifact, syncs to S3, invalidates CloudFront
  4. publish-docs — Publishes Scalar documentation (production and integration only)

Docs Publishing: Documentation is published for production and integration only. Each environment has its own Scalar project and custom domain: production at docs.gamecraft.learnwith.ai, integration at docs-integration.gamecraft.rp.devfactory.com. The API reference automatically points to the environment's own API, so reviewers can test endpoints directly from the docs. Ephemeral (PR) environments do not publish docs.

Deploy Orchestration

The infrastructure job runs pnpm run deploy:<type> from apps/infra/, which invokes deploy.orchestrator.ts. This orchestrator runs the full deploy lifecycle in a single process:

  1. Seed reservations — provisions shared account-level resources (CloudFront origin request policies) and writes SSM reservations to protect them from concurrent cleanup
  2. Synth — runs cdk synth to produce the CloudFormation template (separate step because pre-deploy reads the synthesized template)
  3. Pre-deploy (pre-deploy.hook.ts) — imports orphaned CloudWatch log groups so CDK can adopt them
  4. CDK deploy — applies the CloudFormation changeset (Lambda, API Gateway, DynamoDB, Cognito, CloudFront)
  5. Post-deploy (post-deploy.hook.ts):
    • Publishes environment config JSON to S3 (used by frontend for environment switching)
    • Runs optional external integrations via the integration registry (e.g., Slack notifications, Datadog dashboard upserts)
    • Seeds ephemeral environments (copies Cognito users and the DynamoDB table from integration, excluding environment-bound studio content — games, chats, shares — and re-applies admin designations)
    • Seeds the API and E2E test users (ephemeral and integration; never production) and stores their credentials in SSM, where pnpm script update-env and the post-deploy test jobs pick them up

Persistence

All environments run on DynamoDB. There is no runtime persistence selector and no alternate store path in the codebase.

Destroy Orchestration

When a PR is closed, destroy-ephemeral.yml runs pnpm run destroy:ephemeral from apps/infra/, which invokes destroy.orchestrator.ts. The same workflow also accepts manual workflow_dispatch for orphaned ephemeral stacks whose automatic destroy failed or never ran (for example, a closed PR whose stack still exists).

Manual dispatch must check out a fresh fixed ref (typically main at the current tip), not re-run an old failed workflow attempt. Destroy hooks evolve with the repo; replaying the code snapshot from the original PR-close run can miss pre-destroy teardown or grant assumptions that later commits fixed.

  1. Pre-destroy (pre-destroy.hook.ts): tears down hook-managed Studio AgentCore resources before CloudFormation deletes the stack:
    • Deletes the studio_<env> AgentCore runtime and its workload identity
    • Deletes the per-env studio execution role
    • Fail-closes: any failure aborts the orchestrator before cdk destroy (while the shared CI deploy role still holds the studio lifecycle grants)
  2. CDK destroy: tears down the CloudFormation stack
  3. Post-destroy (post-destroy.hook.ts): cleans up environment artifacts that outlive the stack:
    • Deletes S3 environment configs
    • Removes per-env SSM parameters (including the studio runtime ARN)
    • Cleans up orphaned CloudWatch log groups from CDK custom resources
    • Runs optional integration cleanup (e.g., revokes external resources registered by integrations)
    • Releases and sweeps SSM reservations for shared resources

Post-destroy is intentionally skipped when CDK destroy fails: the stack still exists, so cleaning up reservations or shared policies would be incorrect. On failure, a separate report-failure job (no uses: steps) opens a GitHub issue for manual intervention so a setup-stage Actions outage still surfaces.

Ephemeral reconciliation (reconcile-ephemeral.yml)

A single PR-close destroy shot can miss (dropped webhook, Actions infrastructure failure). Every six hours, reconcile-ephemeral.yml lists live gamecraft-pr* CloudFormation stacks in the integration account, resolves each stack's pull request, and dispatches destroy-ephemeral.yml for orphans whose PR closed outside a grace window (default 60 minutes). Refusal guards match prune-ai-games-branches: preview is the default for manual dispatch, --max-orphans bounds the verdict, the caller must be the integration account, and AWS clients are region-pinned. Scheduled runs always execute. Branch pruning stays a separate script (prune-ai-games-branches); this workflow does not schedule it — see ai-games branch pruning for the dedicated sweep that does.

# Preview the verdict (no destroys dispatched)
gh workflow run reconcile-ephemeral.yml --ref main

# Execute after reviewing a preview
gh workflow run reconcile-ephemeral.yml --ref main -f execute=true

The CI OIDC role holds cloudformation:ListStacks through the wseng-ci-deploy-gamecraft managed policy (see "Shared CI deploy-role permissions" below).

ai-games branch pruning (prune-ai-games-branches.yml)

The pre-destroy hook's ai-games branch deletion is non-fatal, so a failed deletion leaks games/pr*/* branches with nothing to retry it. Every six hours at 03/09/15/21 UTC (three hours after each reconcile-ephemeral.yml slot), prune-ai-games-branches.yml lists games/<env>/* on the ai-games remote, retains durable environments and live ephemeral stacks, and deletes branches of pr<digits> environments whose gamecraft-<env>-stack is absent. The three-hour offset means branches of an environment the stack sweep just destroyed are pruned in the same window rather than the next one. An environment whose stack is mid-delete (DELETE_IN_PROGRESS, or stuck DELETE_FAILED) is logged as Retaining live environment and swept at the following slot, so a leak's lifetime is one window and occasionally two. Scheduled runs always execute; manual dispatch previews by default. --max-dead (default 5) bounds the destructive verdict: a larger backlog refuses before deleting anything, the report-failure job opens an issue titled ai-games branch prune sweep is failing, and recovery is a preview followed by -f execute=true -f max-dead=<n>. No new IAM grant is involved: liveness uses DescribeStacks (already held by the CI OIDC role), and ref deletion uses the integration backend secret's bot token.

# Preview the verdict (no branches deleted)
gh workflow run prune-ai-games-branches.yml --ref main

# Execute after reviewing a preview
gh workflow run prune-ai-games-branches.yml --ref main -f execute=true

Shared CI deploy-role permissions

The shared GithubActionsOidcCdkDeployRole (one per account) is what CI deploys and workflow-dispatched scripts run under. gamecraft's stacks attach no policy to it: its permissions are customer-managed policies that wseng-platform's shared-infra stack declares and attaches, per the wseng-platform ITD. gamecraft's grants are the gamecraft entry of ci-deploy-role-policies.json in wseng-platform, rendered as wseng-ci-deploy-gamecraft.

A new CI capability (a new hook action, a new pooled resource, a new workflow script) is a PR to that file. In a development account such as tapdev, the PR's own shared-infra deploy applies the change before merge; production accounts change on merge. InfraStack's test refuses any stack-owned policy on the role.

Note: Manual API calls can be made via the Scalar interactive docs (/docs/viewer) or by running the auto-generated tests in packages/api-tests/.

Automation Workflows

Workflow Schedule Purpose
auto-update-branches.yml Every 30 min Keeps feature branches up-to-date with main
auto-release-accepted.yml Every 15 min Auto-releases accepted PRs
reconcile-ephemeral.yml Every 6 h Destroys orphaned ephemeral stacks
prune-ai-games-branches.yml Every 6 h Deletes ai-games branches of dead ephemeral envs

Deployment Flow (End-to-End)

A typical integration deployment follows this sequence:

  1. Developer merges a PR into main
  2. deploy-integration.yml triggers automatically
  3. It calls base-deploy.yml with environment: integration
  4. frontend-build compiles the React app with Vite
  5. infrastructure runs the deploy orchestration script (deploy:integration), which seeds account-level resources, synthesizes, runs pre-deploy hooks, deploys the CDK stack, and runs post-deploy hooks (environment config publish, integration sync)
  6. frontend-upload syncs the built assets to S3 and invalidates the CloudFront cache
  7. publish-docs generates and publishes Scalar documentation with the latest API spec
  8. post-deploy-tests (post-deploy-tests.yml) seeds credentials then runs API and E2E suites. Before seeding, two cleanup passes keep durable environments from accumulating test fixtures (gamecraft#535; ITD docs/itds/2026-08-12-integration-fixture-game-cleanup.md):
    • Purge leftover studio fixture games — authenticates as the studio-admin test identity and deletes games whose ai-games folder slug comes from a checked-in fixture (pnpm script cleanup-fixture-games --execute). Human-owned games, and human games under the admin identity (e.g. Trashcat), are retained: a game built while signed in as the studio-admin test identity survives only if its folder slug is not a fixture slug.
    • Prune orphaned ai-games game branches — deletes games/<env>/* refs on trilogy-group/ai-games that no live StudioGame record references (pnpm script prune-orphan-game-branches --execute, targeting the pipeline's ambient ENVIRONMENT).

For PR environments, the same pipeline runs with environment: prNNNN (the environment type is derived automatically from the name), but the publish-docs job is skipped (docs are only published for production and integration).

Managing Environments

Syncing Local .env from a Deployed Environment

# From the integration environment
pnpm script update-env integration

# From a specific PR environment
pnpm script update-env 20

# Wait for a stack that is still deploying
pnpm script update-env integration --wait

The script auto-selects the AWS profile matching the target environment's account (--env/--pr flags remain supported).

Rotating STUDIO_LEDGER_SIGNING_SECRET

The Studio ledger signing secret lives at SSM /gamecraft/<env>/studio/ledger-signing-secret. Post-deploy creates it once via ensureStudioLedgerSigningSecret and deliberately never regenerates it per revision — unlike STUDIO_GATE_MEMO_SECRET, which is fresh random bytes on every deploy and needs no operator runbook.

Rotate only when the secret may have leaked or when deliberately invalidating every signed playtest ledger and judge-dispute store in that environment:

# Ephemeral example — never run against integration or production without an explicit incident decision
aws ssm delete-parameter \
  --name /gamecraft/prNNNN/studio/ledger-signing-secret \
  --profile <account-profile> \
  --region us-east-1

# Redeploy so the runtime upsert re-mints the parameter
pnpm run deploy:ephemeral   # or the matching deploy:<type> for the environment

After redeploy:

  • Every existing playtest/playtest-ledger.json and playtest/judge-disputes.json in that environment verifies as empty (readVerifiedPlaytestLedger / readVerifiedDisputes fail safe toward a red gate). Retirements and disputes are re-earned on the next live runs; history is not silently trusted under the new key.
  • The same fail-safe applies when a game folder is moved or renamed: Phase 1 binds the HMAC key to the relative game path, so that one game's ledger and dispute store stop verifying until re-earned under the new path.

Confirm rotation on an ephemeral stack only: record a non-zero carried count on a game whose ledger already holds a failing row, rotate, redeploy, and expect a subsequent playtest technical line of 0 carried, 0 retired.

Accessing Deployed Environments

Environment Frontend URL API URL Docs URL
Production https://gamecraft.learnwith.ai https://api.gamecraft.learnwith.ai https://docs.gamecraft.learnwith.ai
Integration https://integration.gamecraft.rp.devfactory.com https://api-integration.gamecraft.rp.devfactory.com https://docs-integration.gamecraft.rp.devfactory.com
Ephemeral https://prNNNN.gamecraft.rp.devfactory.com https://api-prNNNN.gamecraft.rp.devfactory.com —

Ephemeral environment URLs are output by the deploy step and posted as a PR comment.