Deploy rails app to AWS instances

0. Mở đầu

Hôm nay mình sẽ hướng dẫn các bạn config, viết code để deploy code từ github lên EC2 instances.

1. Chuẩn bị

• EC2 admin instance (Để ssh từ máy local của bạn)
• EC2 web instances (Có thể ssh từ admin instance)

2. config instances

-setup ssh key: https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys--2 -Test trên admin instance: ssh deploy@localhost -Test trên EC2 web instances: ssh deploy@instance_ip_1, ssh deploy@instance_ip_2 (Mình sẽ config để capitrano tự nhận EC2 web instances private IP) -Cài đặt các package của dự án: https://gorails.com/deploy/ubuntu/16.04

sudo apt install ruby-bundler sudo apt install ruby-whenever

sudo apt install mysql-client-5.7 sudo apt-get install libmysqlclient-dev sudo apt-get install redis-server sudo apt install ruby-sidekiq

-Tạo thư mục deploy code rails:

sudo mkdir /usr/local/rails_apps/ sudo chown -R deploy /usr/local/rails_apps

3. File .env, instances.env

Trên con admin, Tạo ra file .env, instances.env để lưu các biến môi trường dự án

file .env lưu các biến môi trường của con admin

export REPO_URL="git@github.com:framgia/your_app.git"
export RAILS_ENV=staging
export WEB_SERVER=passenger
export AWS_TARGET_GROUP_EC2="arn:aws:elasticloadbalancing:ap-southeastvvvvvvvvvvvv"
export AWS_REGION="ap-southeast-1"
export AWS_ACCESS_KEY_ID="xxxx"
export AWS_SECRET_ACCESS_KEY="xxxx"
....


export SIDEKIQ_NAME_SPACE="admin"

File instances.env lưu các biến môi trường của các con instances

export REPO_URL="git@github.com:framgia/your_app.git"
export RAILS_ENV=staging
export WEB_SERVER=passenger
export AWS_TARGET_GROUP_EC2="arn:aws:elasticloadbalancing:ap-southeastvvvvvvvvvvvv"
export AWS_REGION="ap-southeast-1"
export AWS_ACCESS_KEY_ID="xxxx"
export AWS_SECRET_ACCESS_KEY="xxxx"
....


export SIDEKIQ_NAME_SPACE="instances"

4. Source code

Gem:

gem "aws-sdk", "~> 3"
gem "sidekiq"
gem "redis-rack", git: "https://github.com/redis-store/redis-rack.git", branch: "master"
gem "redis-actionpack", git: "https://github.com/redis-store/redis-actionpack.git", branch: "master"
gem "redis-rails", git: "https://github.com/redis-store/redis-rails.git", branch: "master"
gem "redis-namespace"
gem "carrierwave"
gem "fog"
gem "capistrano-faster-assets"
gem "fog-aws"
gem "asset_sync"

group :staging, :production do
  gem "capistrano"
  gem "capistrano-bundler"
  gem "capistrano-rails"
  gem "capistrano-rvm"
  gem "capistrano-sidekiq"
  gem "capistrano-passenger"
  gem "passenger", ">= 5.0.25", require: "phusion_passenger/rack_handler"
  gem "capistrano3-unicorn"
  gem "unicorn"
end

File deploy.rb /config/deploy.rb

# config valid only for current version of Capistrano
lock "3.8.2"
require 'active_support/core_ext/string'

set :application, ENV["REPO_URL"].split("/").last.gsub(".git","").underscore.camelize
set :repo_url, ENV["REPO_URL"]
set :assets_roles, [:app]
set :deploy_ref, ENV["DEPLOY_REF"]
set :bundle_binstubs, ->{shared_path.join("bin")}
set :whenever_environment, ->{fetch(:stage)}
set :whenever_identifier, ->{"#{fetch(:application)}_#{fetch(:stage)}"}
set :whenever_roles, :whenever

if fetch(:deploy_ref)
  set :branch, fetch(:deploy_ref)
else
  raise "Please set $DEPLOY_REF"
end

set :rvm_ruby_version, "2.4.1"
set :deploy_to, "/usr/local/rails_apps/#{fetch :application}"
case ENV["WEB_SERVER"]
when "passenger"
  set :passenger_roles, :app
  set :passenger_restart_runner, :sequence
  set :passenger_restart_wait, 5
  set :passenger_restart_limit, 2
  set :passenger_restart_with_sudo, false
  set :passenger_environment_variables, {}
  set :passenger_restart_command, "passenger-config restart-app"
  set :passenger_restart_options, -> { "#{deploy_to} --ignore-app-not-running" }
when "unicorn"
  set :unicorn_rack_env, ENV["RAILS_ENV"] || "production"
  set :unicorn_config_path, "#{current_path}/config/unicorn.rb"
end

# Default value for linked_dirs is []
# NOTE: public/uploads IS USED ONLY FOR THE STAGING ENVIRONMENT
set :linked_dirs, %w(bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system public/uploads)

# Default value for default_env is {}

default_env_file_path = ENV["LOCAL_DEPLOY"] ? "/home/deploy/.env" : "/home/deploy/instances.env"

set :default_env, File.read(default_env_file_path).split("
").inject({}){|h,var|
  if var.present?
    k_v = var.gsub("export ","").split("=")
    h.merge k_v.first.downcase => k_v.last.gsub(""", "")
  else
    h
  end
}.symbolize_keys.merge(deploy_ref: ENV["DEPLOY_REF"], deploy_ref_type: ENV["DEPLOY_REF_TYPE"])

namespace :deploy do
  desc "create database"
  task :create_database do
    on roles(:db) do |host|
      within "#{release_path}" do
        with rails_env: ENV["RAILS_ENV"] do
          execute :rake, "db:create"
        end
      end
    end
  end
  before :migrate, :create_database

  desc "upload_s3"
  task :upload_s3 do
    on roles(:app) do
      within "#{release_path}" do
        with rails_env: ENV["RAILS_ENV"] do
          execute :rake, "assets:sync"
        end
      end
    end
  end
  after "deploy:assets:precompile", :upload_s3

  desc "link dotenv"
  task :link_dotenv do
    on roles(:app) do
      unless ENV["LOCAL_DEPLOY"]
        upload! "/home/deploy/instances.env", "/home/deploy/.env"
      end
      execute "ln -s /home/deploy/.env #{release_path}/.env"
    end
  end
  before "sidekiq:restart", "deploy:link_dotenv"

  desc "Restart application"
  task :restart do
    on roles(:app), in: :sequence, wait: 5 do
      case ENV["WEB_SERVER"]
      when "passenger"
        invoke "passenger:restart"
      else
        invoke "unicorn:restart"
      end
    end
  end
  after :publishing, :restart

  desc "update ec2 tags"
  task :update_ec2_tags do
    on roles(:app) do
      within "#{release_path}" do
        if fetch(:stage) == :production && !ENV["LOCAL_DEPLOY"]
          execute :rake, "tag:update_ec2_tags"
        end
      end
    end
  end
  after :restart, :update_ec2_tags
end

-Config namespace để các background job sidekiq trên các con web instances không nhảy lên admin instance (Vì admin instance có biến ENV giá trị khác)

config/initializers/sidekiq.rb

require "sidekiq"
require "sidekiq/web"

Sidekiq.configure_server do |config|
  config.redis = {url: "redis://#{ENV['REDIS_HOSTNAME']}:6379/0", namespace: ENV["SIDEKIQ_NAME_SPACE"]}
end

Sidekiq.configure_client do |config|
  config.redis = {url: "redis://#{ENV['REDIS_HOSTNAME']}:6379/0", namespace: ENV["SIDEKIQ_NAME_SPACE"]}
end

Code lấy private IP của các EC2 instances từ arn-group

/config/deploy/elb.rb

require "aws-sdk"
def get_ec2_targets
  region = ENV["AWS_REGION"]
  Aws.config.update({
    region: region,
    credentials: Aws::Credentials.new(ENV["AWS_ACCESS_KEY_ID"], ENV["AWS_SECRET_ACCESS_KEY"])
  })

  elb_v2 = Aws::ElasticLoadBalancingV2::Client.new(region: region)
  describe_targets = elb_v2.describe_target_health({target_group_arn: ENV["AWS_TARGET_GROUP_EC2"]})
  instances = describe_targets.target_health_descriptions.flat_map(&:target).map &:id
  ec2 = Aws::EC2::Resource.new region: region
  ec2.instances({filters: [{name: "instance-id", values: instances}]}).map do |instance|
    tags = Hash[instance.tags.map{|tag| [tag.key.downcase.to_sym, tag.value]}]
    tags.merge private_ip: instance.private_ip_address
  end
end

Phân chia role whenever để chỉ setup crontab trên 1 EC2 instance (admin instance), tránh việc trùng lặp schedule khi chạy app trên nhiều instances. config/deploy/staging.rb staging.rb

if ENV["LOCAL_DEPLOY"]
  server "localhost", user: "deploy", roles: %w(app db whenever)
else
  require_relative "elb"
  servers = get_ec2_targets
  servers.each do |sv|
    roles = ["app"]
    if sv[:name] == ENV["AWS_LOCAL_DEPLOY_EC2_NAME"]
      roles << "db"
      roles << "whenever"
    end
    server sv[:private_ip], user: "deploy", roles: roles
  end
end

/config/deploy/production.rb

if ENV["LOCAL_DEPLOY"]
  server "localhost", user: "deploy", roles: %w(app db whenever)
else
  require_relative "elb"
  servers = get_ec2_targets
  servers.each do |sv|
    roles = ["app"]
    if sv[:name] == ENV["AWS_LOCAL_DEPLOY_EC2_NAME"]
      roles << "db"
      roles << "whenever"
    end
    server sv[:private_ip], user: "deploy", roles: roles
  end
end

5. bash script

deploy bash (tự lấy code từ github về và deploy)

deploy_bin/deploy

function exit_failure() {
  echo "Aborted due to an error: $1"
  exit 1;
}

SOURCE_CODE_DIR=/home/deploy/$(echo "$REPO_URL" | grep -o "/[a-zA-Z0-9_-]+.git" | sed -r "s/^/|.git$//g")
echo "---> Move to $SOURCE_CODE_DIR";
cd $SOURCE_CODE_DIR || exit_failure "Move to $SOURCE_CODE_DIR";
echo "---> Fetch the codes from git repo";
git fetch origin && git fetch origin --tags || exit_failure "Fetch the codes from git repo";

echo "---> Pull the code from git repo";
case $1 in
tag) REF=$2;;
branch) REF=origin/$2;;
*) exit_failure "Please speify (tag|branch) as the first argument.";;
esac
git reset --hard $REF -- || exit_failure "Pull the codes from git repo";

echo "---> Bundle install";
bundle --path /home/deploy/bundle || exit_failure "Bundle install";
echo "---> Deploy";
DEPLOY_REF_TYPE=$1 DEPLOY_REF=$2 bundle exec cap $RAILS_ENV deploy || exit_failure "Deploy";
echo "---> Done"

Chỉ deploy trên admin instance: deploy_bin/localdeploy

LOCAL_DEPLOY=true deploy $1 $2

6. Deploy

-Lần đầu deploy:

cd 
git clone git@github.com:framgia/your_app.git

-Deploy admin

localdeploy branch develop hoặc localdeploy tag v1.0.0

-Deploy instances

deploy branch develop hoặc deploy tag v1.0.0

7. Kết luận

Như vậy mình vừa hướng dẫn các bạn cách deploy code lên AWS, tránh lỗi duplicate crontab, tránh lỗi background job nhận sai biến ENV

Hi vọng bài viết này hữu ích.