Skip to content

Deployment and Manual Operations

Tunnel hosts are immutable. A change means a new golden AMI and an instance refresh that rolls the ASG onto it. The tunnel credentials and the New Relic license key live in Secrets Manager and are pulled at boot, so replacing an instance never requires reconfiguration.

AMI build and fleet-roll pipeline BUILD THE IMAGE ROLL THE FLEET 1 Edit component YAML, bump the version 2 terraform apply imagebuilder 3 Run the pipeline to bake the AMI 4 Publish AMI ID to SSM ami-id-latest 5 terraform apply cflared-asg 6 ASG rolling refresh Applying the imagebuilder module re-uploads the hashed bootstrap and component versions but does not itself rebuild the AMI; rebuilds stay deliberate (weekly schedule or a manual pipeline run).

Image Builder components and recipes are immutable per version, so any YAML change requires a version bump in components.tf (component) and pipeline.tf (recipe). The bootstrap script is uploaded to S3 by Terraform (aws_s3_object with a content hash), so editing it and applying the imagebuilder module re-uploads it; the next build picks it up. Applying the module does not by itself rebuild the AMI: rebuilds stay deliberate, either the weekly pipeline schedule or a manual pipeline run.

Terminal window
cd ~/Repositories/GitHub/Adventive/adventive-platform-infra && . ~/Documents/Claude/.cowork-env && aws imagebuilder start-image-pipeline-execution --image-pipeline-arn "$(aws imagebuilder list-image-pipelines --query "imagePipelineList[?name=='adv-cflared-pipeline'].arn" --output text --region us-east-1)" --region us-east-1

When the build reaches AVAILABLE, the resulting AMI ID must land in the SSM parameter /adventive/cloudflared/ami-id-latest, which the launch templates read. An EventBridge rule and the adv-cflared-publish-ami Lambda are meant to do this on build success; until that path is confirmed working, set the parameter directly (this is supported: the parameter has ignore_changes = [value, description] so Terraform will not fight a hand-set value).

Terminal window
cd ~/Repositories/GitHub/Adventive/adventive-platform-infra && . ~/Documents/Claude/.cowork-env && aws ssm put-parameter --name /adventive/cloudflared/ami-id-latest --type String --overwrite --value ami-0123456789abcdef0 --region us-east-1

Then roll the fleet by applying the cflared-asg module: the launch template picks up the new AMI, its version bumps, and the ASG runs a rolling instance refresh. Apply order across environments is dev, then stg, then prd.

Terminal window
cd ~/Repositories/GitHub/Adventive/adventive-platform-infra/infra/cflared-asg && . ~/Documents/Claude/.cowork-env && terraform plan -out=tfplan

Review the plan (the image_id change and the launch-template version bump), then terraform apply tfplan, and watch the refresh:

Terminal window
cd ~/Repositories/GitHub/Adventive/adventive-platform-infra && . ~/Documents/Claude/.cowork-env && aws autoscaling describe-instance-refreshes --auto-scaling-group-name adv-cflared-<env> --region us-east-1 --query 'InstanceRefreshes[0].{Status:Status,Pct:PercentageComplete}' --output json

Because a component or recipe version cannot be deleted while an image built from it still exists, replacing a recipe whose prior build is still around can make terraform apply fail on the destroy of the old version. If that happens, delete the referencing image and re-apply:

Terminal window
cd ~/Repositories/GitHub/Adventive/adventive-platform-infra && . ~/Documents/Claude/.cowork-env && aws imagebuilder delete-image --image-build-version-arn "arn:aws:imagebuilder:us-east-1:ACCOUNT:image/adv-cflared-recipe/<version>/1" --region us-east-1

Reach for these when CI or the pipeline is unavailable, or during an incident. Host shell access, when genuinely needed, is through Session Manager, never SSH.

  • Force a fleet roll onto the current AMI. Start an instance refresh directly on the ASG.
  • Rotate the tunnel credentials. Change the tunnel in the cloudflare-tunnels module (the secret re-renders), then instance-refresh the fleet so new hosts fetch the new secret at boot.
  • Rotate the New Relic license key. Update the Secrets Manager secret value and instance-refresh; the bootstrap rewrites /etc/newrelic-infra.yml.
  • Roll back a bad AMI. Re-point the SSM ami-id-latest parameter to the previous AMI ID and apply cflared-asg (or start a refresh).
  • Replace one instance. Set its ASG health to Unhealthy; the ASG terminates and replaces it from the AMI.

No secret is baked into the AMI or committed to git. Everything the host needs is read from Secrets Manager at boot by the instance role.

NameLocationRead by
/adventive/cloudflared/{env}AWS Secrets ManagerTunnel config.yml + credentials, per environment
/adventive/newrelic/license-keyAWS Secrets ManagerNew Relic Infrastructure agent, shared
/adventive/cloudflared/ami-id-latestAWS SSM Parameter StoreLaunch template AMI lookup