SaltyCrane Blog — Notes on JavaScript and web development

Example Next.js GitLab CI/CD Amazon ECR and ECS deploy pipeline

Date:  |  Tags: aws, devops, docker, javascript

I've created an example Next.js project with a GitLab CI/CD pipeline that builds a Docker image, pushes it to Amazon ECR, deploys it to an Amazon ECS Fargate cluster, and uploads static assets (JS, CSS, etc.) to Amazon S3. The example GitLab repo is here: https://gitlab.com/saltycrane/next-aws-ecr-ecs-gitlab-ci-cd-example

Interesting files

Here are the interesting parts of some of the files. See the full source code in the GitLab repo.

  • .gitlab-ci.yml (view at gitlab)

    • the variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and ECR_HOST are set in the GitLab UI under "Settings" > "CI/CD" > "Variables"
    • this uses the saltycrane/aws-cli-and-docker Docker image which provides the aws v2 command line tools and docker in a single image. It is based on amazon/aws-cli and installs bc, curl, docker, jq, and tar. This idea is from Valentin's tutorial.
    variables:
  DOCKER_HOST: tcp://docker:2375
  DOCKER_TLS_CERTDIR: ""
  AWS_DEFAULT_REGION: "us-east-1"
  CI_APPLICATION_REPOSITORY: "$ECR_HOST/next-aws-ecr-ecs-gitlab-ci-cd-example"
  CI_APPLICATION_TAG: "$CI_PIPELINE_IID"
  CI_AWS_S3_BUCKET: "next-aws-ecr-ecs-gitlab-ci-cd-example"
  CI_AWS_ECS_CLUSTER: "next-aws-ecr-ecs-gitlab-ci-cd-example"
  CI_AWS_ECS_SERVICE: "next-aws-ecr-ecs-gitlab-ci-cd-example"
  CI_AWS_ECS_TASK_DEFINITION: "next-aws-ecr-ecs-gitlab-ci-cd-example"
  NEXT_JS_ASSET_URL: "https://$CI_AWS_S3_BUCKET.s3.amazonaws.com"

stages:
  - build
  - deploy

build:
  stage: build
  image: saltycrane/aws-cli-and-docker
  services:
    - docker:dind
  script:
    - ./bin/build-and-push-image-to-ecr
    - ./bin/upload-assets-to-s3

deploy:
  stage: deploy
  image: saltycrane/aws-cli-and-docker
  services:
    - docker:dind
  script:
    - ./bin/ecs update-task-definition

  • Dockerfile (view at gitlab)

    The value of NEXT_JS_ASSET_URL is passed in using the --build-arg option of the docker build command run in bin/build-and-push-image-to-ecr. It is used like an environment variable in the RUN npm run build command below. In this project it is assigned to assetPrefix in next.config.js.

    FROM node:14.16-alpine
ARG NEXT_JS_ASSET_URL
ENV NODE_ENV=production
WORKDIR /app
COPY ./package.json ./
COPY ./package-lock.json ./
RUN npm ci
COPY . ./
RUN npm run build
EXPOSE 3000
CMD ["npm", "start"]

  • bin/build-and-push-image-to-ecr (view at gitlab)

    # log in to the amazon ecr docker registry
aws ecr get-login-password | docker login --username AWS --password-stdin "$ECR_HOST"

# build docker image
docker pull "$CI_APPLICATION_REPOSITORY:latest" || true
docker build --build-arg "NEXT_JS_ASSET_URL=$NEXT_JS_ASSET_URL" --cache-from "$CI_APPLICATION_REPOSITORY:latest" -t "$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG"     -t "$CI_APPLICATION_REPOSITORY:latest" .

# push image to amazon ecr
docker push "$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG"
docker push "$CI_APPLICATION_REPOSITORY:latest"

  • bin/upload-assets-to-s3 (view at gitlab)

    LOCAL_ASSET_PATH=/tmp/upload-assets

mkdir $LOCAL_ASSET_PATH

# copy the generated assets out of the docker image
docker run --rm --entrypoint tar "$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG" cf - .next | tar xf - -C $LOCAL_ASSET_PATH

# rename .next to _next
mv "$LOCAL_ASSET_PATH/.next" "$LOCAL_ASSET_PATH/_next"

# remove directories that should not be uploaded to S3
rm -rf "$LOCAL_ASSET_PATH/_next/cache"
rm -rf "$LOCAL_ASSET_PATH/_next/server"

# gzip files
find $LOCAL_ASSET_PATH -regex ".*\.\(css\|svg\|js\)$" -exec gzip {} \;

# strip .gz extension off of gzipped files
find $LOCAL_ASSET_PATH -name "*.gz" -exec sh -c 'mv $1 `echo $1 | sed "s/.gz$//"`' - {} \;

# upload gzipped js, css, and svg assets
aws s3 sync --no-progress $LOCAL_ASSET_PATH "s3://$CI_AWS_S3_BUCKET" --cache-control max-age=31536000 --content-encoding gzip --exclude "*" --include "*.js"     --include "*.css" --include "*.svg"

# upload non-gzipped assets
aws s3 sync --no-progress $LOCAL_ASSET_PATH "s3://$CI_AWS_S3_BUCKET" --cache-control max-age=31536000 --exclude "*.js" --exclude "*.css" --exclude "*.svg" --exclude "*.map"

  • bin/ecs (view full file) (This file was copied from the gitlab-org repo)

    #!/bin/bash -e

update_task_definition() {
  local -A register_task_def_args=( \
    ['task-role-arn']='taskRoleArn' \
    ['execution-role-arn']='executionRoleArn' \
    ['network-mode']='networkMode' \
    ['cpu']='cpu' \
    ['memory']='memory' \
    ['pid-mode']='pidMode' \
    ['ipc-mode']='ipcMode' \
    ['proxy-configuration']='proxyConfiguration' \
    ['volumes']='volumes' \
    ['placement-constraints']='placementConstraints' \
    ['requires-compatibilities']='requiresCompatibilities' \
    ['inference-accelerators']='inferenceAccelerators' \
  )

  image_repository=$CI_APPLICATION_REPOSITORY
  image_tag=$CI_APPLICATION_TAG
  new_image_name="${image_repository}:${image_tag}"

  register_task_definition_from_remote

  new_task_definition=$(aws ecs register-task-definition "${args[@]}")
  new_task_revision=$(read_task "$new_task_definition" 'revision')
  new_task_definition_family=$(read_task "$new_task_definition" 'family')

  # Making sure that we at least have one running task (even if desiredCount gets updated again with new task definition below)
  service_task_count=$(aws ecs describe-services --cluster "$CI_AWS_ECS_CLUSTER" --services "$CI_AWS_ECS_SERVICE" --query "services[0].desiredCount")

  if [[ $service_task_count == 0 ]]; then
    aws ecs update-service --cluster "$CI_AWS_ECS_CLUSTER" --service "$CI_AWS_ECS_SERVICE" --desired-count 1
  fi

  # Update ECS service with newly created task defintion revision.
  aws ecs update-service \
            --cluster "$CI_AWS_ECS_CLUSTER" \
            --service "$CI_AWS_ECS_SERVICE" \
            --task-definition "$new_task_definition_family":"$new_task_revision"

  return 0
}

read_task() {
  val=$(echo "$1" | jq -r ".taskDefinition.$2")
  if [ "$val" == "null" ];then
    val=$(echo "$1" | jq -r ".$2")
  fi
  if [ "$val" != "null" ];then
    echo -n "${val}"
  fi
}

register_task_definition_from_remote() {
  task=$(aws ecs describe-task-definition --task-definition "$CI_AWS_ECS_TASK_DEFINITION")
  current_container_definitions=$(read_task "$task" 'containerDefinitions')
  new_container_definitions=$(echo "$current_container_definitions" | jq --arg val "$new_image_name" '.[0].image = $val')
  args+=("--family" "${CI_AWS_ECS_TASK_DEFINITION}")
  args+=("--container-definitions" "${new_container_definitions}")
  for option in "${!register_task_def_args[@]}"; do
    value=$(read_task "$task" "${register_task_def_args[$option]}")
    if [ -n "$value" ];then
      args+=("--${option}" "${value}")
    fi
  done
}

update_task_definition

Usage - set up AWS resources

Below are the minimum steps I needed to create the required AWS services for my example. I use the AWS region "us-east-1". For info about creating some of these services via the command line, see my Amazon ECS notes.

Create an ECR repository

Create an ECS Fargate cluster

Create an ECS task definition

  • click "Create new Task Definition" here: https://console.aws.amazon.com/ecs/home?region=us-east-1#/taskDefinitions
  • select "FARGATE" and click "Next step"
  • configure task
    • for "Task Definition Name" enter "next-aws-ecr-ecs-gitlab-ci-cd-example"
    • for "Task Role" select "None"
    • for "Task execution role" select "Create new role"
    • for "Task memory" select "0.5GB"
    • for "Task CPU" select "0.25 vCPU"
    • click "Add container"
      • for "Container Name" enter "next-aws-ecr-ecs-gitlab-ci-cd-example"
      • for "Image" enter "asdf" (this will be updated by the gitlab ci/cd job)
      • leave "Private repository authentication" unchecked
      • for "Port mappings" enter "3000"
      • click "Add"
    • click "Create"

Create an ECS service

  • click "Create" here: https://us-east-1.console.aws.amazon.com/ecs/home?region=us-east-1#/clusters/next-aws-ecr-ecs-gitlab-ci-cd-example/services
  • configure service
    • for "Launch type" select "FARGATE"
    • for "Task Definition" enter "next-aws-ecr-ecs-gitlab-ci-cd-example"
    • for "Cluster" select "next-aws-ecr-ecs-gitlab-ci-cd-example"
    • for "Service name" enter "next-aws-ecr-ecs-gitlab-ci-cd-example"
    • for "Number of tasks" enter 1
    • for "Deployment type" select "Rolling update"
    • click "Next step"
  • configure network
    • select the appropriate "Cluster VPC" and two "Subnets"
    • click "Next step"
  • set Auto Scaling
    • click "Next step"
  • review
    • click "Create Service"

Open port 3000

Create a S3 bucket

  • click "Create bucket" here: https://s3.console.aws.amazon.com/s3/home?region=us-east-1
  • for "Bucket name" enter "next-aws-ecr-ecs-gitlab-ci-cd-example"
  • uncheck "Block all public access"
  • check the "I acknowledge that the current settings might result in this bucket and the objects within becoming public" checkbox
  • click "Create bucket"

Update permissions for S3 bucket

Create an IAM user

  • create an IAM user. The user must have at least ECR, ECS, and S3 permissions.
  • take note of the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

Usage - run the CI/CD pipeline

Fork the example gitlab repo and configure CI/CD variables

  • fork https://gitlab.com/saltycrane/next-aws-ecr-ecs-gitlab-ci-cd-example
  • go to "Settings" > "CI/CD" > "Variables" and add the following variables. You can choose to "protect" and "mask" all of them.
    • AWS_ACCESS_KEY_ID
    • AWS_SECRET_ACCESS_KEY
    • ECR_HOST (This is the part of the ECR repository URI before the /. It looks something like XXXXXXXXXXXX.dkr.ecr.us-east-1.amazonaws.com)

Edit variables in .gitlab-ci.yml

If you used names other than "next-aws-ecr-ecs-gitlab-ci-cd-example", edit the variables in .gitlab-ci.yml.

Test it

References (build/push)

References (deploy)

Comments