Packer EC2 support and Ansible for our Cassandra Database Clusters

Packer is used to generate machine and container images for multiple platforms from a single source configuration. We use Packer to create AWS EC2 AMIs (images) and Docker images. (We use Vagrant to setup dev images on Virtual Box.) 

Packer for AWS Cassandra Database EC2/AMI

This code listing is our Packer provisioning script to produce an EC2 AMI which we can later use to produce and EC2 instance with Cassandra installed. This script will install Cassandra on the EC2 image. 

packer-ec2.json - Packer creation script for EC2 Cassandra Database instance

{
  "variables": {
    "aws_access_key": "",
    "aws_secret_key": "",
    "aws_region": "us-west-2",
    "aws_ami_image": "ami-d2c924b2",
    "aws_instance_type": "m4.large",
    "image_version" : "0.2.2"
  },
  "builders": [
    {
      "type": "amazon-ebs",
      "access_key": "{{user `aws_access_key`}}",
      "secret_key": "{{user `aws_secret_key`}}",
      "region": "{{user `aws_region`}}",
      "source_ami": "{{user `aws_ami_image`}}",
      "instance_type": "{{user `aws_instance_type`}}",
      "ssh_username": "centos",
      "ami_name": "cloudurable-cassandra-{{user `image_version`}}",
      "tags": {
        "Name": "cloudurable-cassandra-{{user `image_version`}}",
        "OS_Version": "LinuxCentOs7",
        "Release": "7",
        "Description": "CentOS 7 image for Cloudurable Cassandra image"
      },
      "user_data_file": "config/user-data.sh"
    }
  ],
  "provisioners": [
    {
      "type": "file",
      "source": "scripts",
      "destination": "/home/centos/"
    },
    {
      "type": "file",
      "source": "resources",
      "destination": "/home/centos/"
    },
    {
      "type": "shell",
      "scripts": [
        "scripts/000-ec2-provision.sh"
      ]
    },
    {
      "type": "ansible",
      "playbook_file": "playbooks/ssh-addkey.yml"
    }
  ]
}

Notice that we are using a packer amazon-ebs builder to build an AMI image based on our local dev boxes EC2 setup.

Also, see that we use a series of Packer provisioners. The packer file provisioner can copy files or directories to a machine image. The packer shell provisioner can run shell scripts. Lastly the packer ansible provisioner can run ansible playbooks. We covered what playbooks/ssh-addkey.yml does in the previous article, but in short it sets up the keys so we use ansible with our the Cassandra Database cluster nodes.

Bash provisioning

Before we started applying ansible to do provisioning, we used bash scripts that get reused for packer/docker, packer/aws, and vagrant/virtual-box. The script 000-ec2-provision.sh invokes these provisioning scripts which the first three articles covered at varying degrees (skim those articles if you are curious or the source code, but you don’t need it per se to follow). This way we can use the same provisioning scripts with AMIs, VirtualBox, and AWS EC2.

scripts/000-ec2-provision.sh

#!/bin/bash
set -e

sudo cp -r /home/centos/resources/ /root/
sudo mv /home/centos/scripts/ /root/

echo RUNNING PROVISION
sudo /root/scripts/000-provision.sh
echo Building host file
sudo /root/scripts/002-hosts.sh
echo RUNNING TUNE OS
sudo /root/scripts/010-tune-os.sh
echo RUNNING INSTALL the Cassandra Database
sudo /root/scripts/020-cassandra.sh
echo RUNNING INSTALL CASSANDRA CLOUD
sudo /root/scripts/030-cassandra-cloud.sh
echo RUNNING INSTALL CERTS
sudo /root/scripts/040-install-certs.sh
echo RUNNING SYTSTEMD SETUP
sudo /root/scripts/050-systemd-setup.sh

sudo chown -R cassandra /opt/cassandra/
sudo chown -R cassandra /etc/cassandra/

We covered what each of those provisioning scripts does in the first three articles, but for those just joining us, they install packages, programs and configure stuff.

Using Packer to build our ec2 AMI

To build the AWS AMI, we use packer build as follows.

Building the AWS AMI

$ packer build packer-ec2.json

After the packer build completes, it will print out the name of the AMI image it created, e.g., ami-6db33abc.

Cassandra Tutorial: Cassandra Cluster DevOps/DBA series

The first tutorial in this Cassandra tutorial series was about setting up a Cassandra cluster with Vagrant (also appeared on DZone with some additional content DZone Setting up a Cassandra Cluster with Vagrant. The second article in this series was about setting up SSL for a Cassandra cluster using Vagrant (which also appeared with more content as DZone Setting up a Cassandra Cluster with SSL). The third article in this series was about configuring and using Ansible (building on the first two articles). This article (the 4th) will cover applying the tools and techniques from the first three articles to produce an image (EC2 AMI to be precise) that we can deploy to AWS/EC2. To do this explanation, we will use Packer, Ansible, and the Aws Command Line tools. The AWS command line tools are essential for doing DevOps with AWS.

Check out more information about the Cassandra Database

• Cassandra Consulting: Architecture Analysis
• Cassandra Consulting: Quick Start
• Cassandra Course
• Amazon Cassandra Support