My experiences with Fabric based deployment automation

Many good tools are available for configuration management and application deployment.
Puppet, Chef have attained cult status among the dev-ops team. There are good tools available in Python too. Salt may soon become a viable alternative and looks definitely promising to me. Push-Pull is commonly used to explain various types of tools in the eco-system.

Fabric is an excellent tool that allows you to weave operation locally and remotely on cluster of machines, allowing you to deploy applications, start/stop services and perform other operations on a cluster of machines. There are few good tutorials available to help familiarize with Fabric. If you haven't read it already, you should.

1. An example on deploying django using Fabric
2. A presentation on using Fabric
3. A video on Fabric usage

I have used Fabric to automate deployment of Hadoop / Hive application, Nagios deployment on cluster of machines on EC2, private cloud based on Cloudstack and commodity machines.

The code grew from nifty little commands / functions like setting up the "Fully qualified domain hostname" (FQDN), creating users and groups on Linux, installing yum packages to a complete system of commands that installs and brings up Kerberos enabled secure hadoop cluster using Cloudera hadoop packages.

Code soon became unwieldily.

There are a few practices that helps contain the level of complexity that grows when using Fabric enthusiastically.

Installing funkload on Mac

Funkload is  a useful tool for understanding the characteristics of the application server under stress and load conditions.

Installation of Funkload is very straightforward using virtualenv and macports on Mac.
If you aren't using it already, you should think of checking it out.

Create a isolated environment for installing Funkload.

virtualenv --no-site-packages loadtest
source loadtest/bin/activate
pip install yolk

Creating base box from scratch for Vagrant

At vagrantbox.es, you can find boxes for many flavours like CentOS, Ubuntu, Debian etc.

How ever, you might require a flavour of OS that is not available packaged for you already.
In such a case, you might want to package it for use with Vagrant.
I needed Oracle Enterprise Linux Box.

Following is a step by step approach to create a base box for Oracle Enterprise Linux 5.7 64 bit version.


Creating a VM on VirtualBox

Step 1: Get the ISO file from which we will install the Oracle Enterprise Linux.

Step 2: Create your virtual machine on VirtualBox.

  Create a new Virtual Machine 
      Type: VMDK
      Name : oel57
      Base memory size: 512 MB, Memory Space Maximum 40 GB
      Enable Host I/O cache

Using Vagrant

Vagrant is a great tool for creating vm at whim and tearing it down so that you could start all over again. It helps to start from a clean state, when you are trying to test deployment and setups. Vagrant requires VirtualBox and is written in Ruby.

Following is a step by step take down on how to setup and use vagrant on Mac