How To Use Basecamp’s MRSK With AWS and GitHub

This article is outdated please read an updated version:

• https://jetrockets.com/blog/how-to-use-basecamp-s-kamal-with-aws-and-github

What we’ve used before?

What is MRSK and how it works?

MRSK + Amazon AWS possible schemas

AWS offers a plethora of networking options to organize your application infrastructure. The image below illustrates one of the simplest and most commonly used web application architectures.

Our domain is hosted on Route 53 in this network schema, pointing to an AWS ALB. All traffic is routed to the ALB, where SSL termination occurs. The HTTP traffic is directed to one or several EC2 instances managed with MRSK. Traefik on these instances routes the traffic to the Rails container, ensuring efficient request processing and optimal performance. This architecture combines AWS services, MRSK, and Traefik to create a reliable and scalable infrastructure for hosting our application.

ssh:
  proxy: "www@jumphost_public_ip"

To support such a setup, MRSK offers the ssh: proxy option, enabling seamless integration and efficient management of the infrastructure.

To make schema shown above even more robust AWS CloudFront can be added.

This approach allows us to differentiate traffic that benefits from caching on the CDN (Content Delivery Network) from traffic that needs to be directly forwarded to the application. In our specific scenario, we can utilize assets.application.site to distribute CSS and JS bundles, which will be served by the Rails application server upon each build and cached by CloudFront.

How to start using MRSK with Rails 7.0?

• Prepare the application to run in a container
• Create MRSK deploy config
• Prepare AWS EC2 instance(s) and AWS ALB
• Create a GitHub Actions scenario
• Deploy and manage the application
• Tweaks and Traefik configuration

Prepare the application to run in a container

# This first and main rule that we follow in our Dockerfile is to keep it
# without hardcoded dependencies and versions as much as possible.
# This file is an ideal – it has zero.

# It accepts selected Ruby version as an argument.
ARG RUBY_VERSION

# Create new stage from ruby-slim image with selected Ruby version.
FROM ruby:$RUBY_VERSION-slim as base

# We are Rails developers, so we want to keep our app under /rails directory
WORKDIR /rails

# Here is some Ruby magic and configuration options
ENV BUNDLE_DEPLOYMENT="1" \
  BUNDLE_PATH="/usr/local/bundle" \
  BUNDLE_WITHOUT="development" \
  HOME=/rails

# Throw-away build stage to reduce size of final image
FROM base as build

# Install dependencies that are needed only during build stage
RUN apt-get update -qq && \
  apt-get install -y build-essential curl libpq-dev nodejs git

# Install JavaScript dependencies
ARG NODE_VERSION
ARG YARN_VERSION
ENV PATH=/usr/local/node/bin:$PATH
RUN curl -sL https://github.com/nodenv/node-build/archive/master.tar.gz | tar xz -C /tmp/ && \
    /tmp/node-build-master/bin/node-build "${NODE_VERSION}" /usr/local/node && \
    npm install -g yarn@$YARN_VERSION && \
    npm install -g mjml && \
    rm -rf /tmp/node-build-master

# Install application gems
COPY Gemfile Gemfile.lock ./
RUN bundle install && \
    rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache "${BUNDLE_PATH}"/ruby/*/bundler/gems/*/.git

# Install node modules
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile

# Copy application code
COPY . .

ARG RAILS_ENV

# If you are already on Rails 7.1+,
# then you will be able to use new SECRET_KEY_BASE_DUMMY=1 variable,
# however below is a workaround for Rails 7.0 and below.
RUN SECRET_KEY_BASE=1 DATABASE_URL=postgresql://dummy@localhost/dummy RAILS_ENV=$RAILS_ENV \
  bundle exec rake assets:precompile

# Start again from base image to throw away anything that is not needed
# from build stage.
FROM base

# Install dependencies that we needed during application run.
# We use PostgresSQL for almost all our projects,
# this is why we install `postgresql-client`.
# Also almost all web applications have image upload and convert functionality,
# so you need to install `imagemagick`.
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y libvips postgresql-client imagemagick tzdata curl && \
    rm -rf /var/lib/apt/lists /var/cache/apt/archives

# Run and own the application files as a non-root user for security
RUN useradd rails --create-home --shell /bin/bash

# Copy built artifacts: node, gems, application
COPY --from=build --chown=rails:rails /usr/local/node /usr/local/node
COPY --from=build --chown=rails:rails /usr/local/bundle /usr/local/bundle
COPY --from=build --chown=rails:rails /rails /rails
RUN chown rails:rails /rails

USER rails:rails

# Entrypoint prepares the database.
ENTRYPOINT ["/rails/bin/docker-entrypoint"]

# Start the server by default, this can be overwritten at runtime
EXPOSE 3000
CMD ["./bin/rails", "server"]

#!/bin/bash -e

# If running the rails server then create or migrate existing database
if [ "${*}" == "./bin/rails server" ]; then
  ./bin/rails db:prepare
fi

exec "${@}"

# lib/middleware/assume_ssl.rb
#
class AssumeSSL
  def initialize(app)
    @app = app
  end

  def call(env)
    env['HTTPS'] = 'on'
    env['HTTP_X_FORWARDED_PORT'] = 443
    env['HTTP_X_FORWARDED_PROTO'] = 'https'
    env['rack.url_scheme'] = 'https'

    @app.call(env)
  end
end

# config/application.rb
#
Dir[Rails.root.join('lib/middleware/**/*.{rb}')].sort
  .each { |file| require file }

# config/environments/production.rb
#
Rails.application.config.middleware.insert_before(0, AssumeSSL)

Create MRSK deploy config

bundle add mrsk

gem 'mrsk'

mrsk init

# Name of your application. Used to uniquely configure containers.
service: onetribe

# Name of the container image.
image: onetribe

# Deploy to these servers.
servers:
  web:
    - "//web IP//"
  sidekiq:
    cmd: bin/sidekiq
    hosts:
      - "//sidekiq IP//"

# Use a different ssh user than root
ssh:
  user: www

# Credentials for your image host.
registry:
  # Specify the registry server, if you're not using Docker Hub
  server: AMAZON_ID.dkr.ecr.us-east-1.amazonaws.com
  username: AWS

  # Always use an access token rather than real password when possible.
  password:
    - MRSK_REGISTRY_PASSWORD

# Container builder setup.
builder:
  args:
    RUBY_VERSION: 3.2.2
    RAILS_ENV: production
		NODE_VERSION: 18.12.0
		YARN_VERSION: 1.22.19
  multiarch: false

# Container run setup
env:
  clear:
    RAILS_LOG_TO_STDOUT: 1
    RAILS_SERVE_STATIC_FILES: 1
    RAILS_ENV: production
  secret:
    - RAILS_MASTER_KEY
    - DATABASE_URL
    - REDIS_URL

# Configure a custom healthcheck (default is /up on port 3000)
healthcheck:
  path: /health

Prepare AWS EC2 instance(s) and AWS ALB

In JetRockets, we usually use Ansible and TerraForm to configure all virtual machines and networks. I will not go too deep into AWS configuration and will cover only the required things. We aim to create required EC2 instances, configure them to run Docker, and configure AWS ALB. We will go with Ubuntu 22.04.

After the instance has been created don’t forget to allocate and associate the Elastic IP address with it. Put the newly obtained IP address to your MRSK config into servers section. Once the instance is up and running, connect to it with the default user (usually ubuntu) and install Docker to run our containers. You can read more about this in Docker official manual. Long story short, you should install docker-ce and docker-ce-cli.

Once Docker is installed, create a user that will be used to run web services. I usually go with www username for this. 

ubuntu@ip-172-31-86-216:~$ sudo useradd -d /srv/www -m -s /bin/bash --groups docker www

www@ip-172-31-86-216:~$ docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
www@ip-172-31-86-216:~$

The instance is ready. Time to configure ALB. Create new Application Load Balancer and configure two listeners on it. I find it easier to create a balancer with just one listener and configure everything when balancer already exists.

The key point is that we redirect all HTTP/80 traffic to HTTPS with 301 response code. For HTTPS configuration is also very clear: we forward all HTTPS traffic that comes from onetribe.team and *.onetribe.team to mrsk target group (that includes the previously created EC2 instance) through HTTP 80 port. This is where SSL termination happens.

That covers the basics of the AWS setup. If you need additional instances to handle Sidekiq or to serve web requests, you can simply duplicate the existing EC2 instance.

GitHub Actions scenario

name: MRSK

on:
  push:
    branches:
    - feature/mrsk
  # I added workflow_dispatch trigger to be able to run workflow manually from GitHub webinterface 
  workflow_dispatch:

jobs:
  # To separate jobs that lint code and run specs are defined in separate files
  # spec:
  #   uses: ./.github/workflows/specs.yml
  # lint:
  #   uses: ./.github/workflows/lint_code.yml

  build_and_deploy:
    # needs: [spec, lint]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          ref: ${{ github.event.workflow_run.head_branch }}
			
			# Since MRSK uses SSH access to perform commands on remote server
			# we need to configure it with a valid SSH key 
      - uses: webfactory/ssh-agent@v0.7.0
        with:
          ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}

      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: 3.2.2
          bundler-cache: true
			
			# Since we want to build images directly on a GitHub runner,
			# we need to install Docker buildx.
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
			
			# For simplicity in this example I will use AWS access_key_id and secret
			# to authenticate in AWS ECR.
			#
			# There is an option to do it better and more secure by using
			# AWS OpenId Connect, we will cover this options in one of our future articles.
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id    : ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region           : us-east-1
          mask-aws-account-id  : 'no'

      - name: Login to Amazon ECR
        id  : login-ecr
        uses: aws-actions/amazon-ecr-login@v1

      - name: MRSK
        id  : mrsk
        env :
          MRSK_REGISTRY_PASSWORD: ${{ steps.login-ecr.outputs.docker_password_AMAZON_ID_dkr_ecr_us_east_1_amazonaws_com }}
          DATABASE_URL: ${{ secrets.DATABASE_URL }}
          REDIS_URL: ${{ secrets.REDIS_URL }}
          RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
          RAILS_ENV: production
          DOCKER_BUILDKIT: 1
        run: |
          ./bin/mrsk deploy --quiet

Deploy and manage the application

After all, preparations have been done; we are ready to deploy our app. After pushing code to the feature/mrsk branch our MRSK workflow will start automatically.

As you can see, the entire process was completed in approximately 6 minutes. This is quite impressive because we’ve used a micro instance to run the application and GitHub standard runners to build it.

Additional tweaks and Traefik configuration

With STDOUT logging you can access your application container logs from your host machine with docker logs <container_id> --follow, you will probably see something like this:

An alternative approach for accessing application logs is by using the mrsk app logs command directly from your development machine. This method offers a more developer-friendly experience.

The mrsk logs command provides convenient options to enhance log analysis. For instance, you can use the grep command to search for specific log entries, and filter logs by timestamp, hosts, or roles. To explore the available options, you can use the --help flag, which provides a comprehensive list of command options and their functionalities.

It's essential to consider that we are utilizing Docker, which provides a variety of logging drivers, including integration with AWS CloudWatch.

logging:
  driver: awslogs
  options:
    awslogs-region: us-east-1
    awslogs-group: /application
    awslogs-create-group: true
    tag: "{{.Name}}-{{.ID}}"

To enable awslogs on your host machine, you must provide AWS credentials. This can be achieved by installing awscli and configuring it from the root user.

root@app:~# apt install awscli
root@app:~# aws configure

Traefik Logging

traefik:
  args:
    log: true
    log.level: ERROR
    accesslog: true
    accesslog.format: json
    accesslog.filters.minduration: 10ms

Further Plans

Conclusions