Setting up buildbot - customizing configuration file

The crux of BuildBot involves a master and multiple build slaves that can be distributed across many computers.

Each Builder is configured with a list of BuildSlaves that it will use for its builds. Within a single BuildSlave, each Builder creates its own SlaveBuilder instance.
Once a SlaveBuilder is available, the Builder pulls one or more BuildRequests off its incoming queue.These requests are merged into a single Build instance, which includes the SourceStamp that describes what exact version of the source code should be used for the build. The Build is then randomly assigned to a free SlaveBuilder and the build begins.

All this is configured via a single file called master.cfg which is a dictionary of various keys that is used to configure the buildbot when it starts up.
Open up the sample "master.cfg" that comes with the buildbot distribution, drop it in the master directory that you have created and start hacking it.

I have listed few important configuration that should get you started.
Below is instance of dictionary that is populated in the configuration file

c = BuildmasterConfig = {}

First we setup build slaves. I will setup a single build slave that resides on the same machine as the master.
Key is 'slave' and value is an instance of class BuildSlave with slave name, password and no of concurrent builds it can execute.
The slave is configured to receive build requests on a tcp port. This is same as the port that was given when creating the build slave during installation.

c['slaves'] = [BuildSlave("build-slave1", "webappslave", max_builds=1)]
c['slavePortnum'] = 9999

After this you setup the mechanism for buildmaster to detect source code changes. This section deals with version control integration.
You can either create a hook script that notifies the "change source" to BuildBot or allow BuildBot to poll the version repository.
The hook script should be able to send the command "buildbot sendchange".

from buildbot.changes.pb import PBChangeSource
c['change_source'] = PBChangeSource()

The other approach is to use a poller. I have used the svn poller here.

source_code_svn_url = "HTTP url for subversion repository"

svn_poller = SVNPoller(
                    svnurl=source_code_svn_url,
                    svnuser='name',
                    svnpasswd='password',
                    pollinterval=2*60, # seconds
                    histmax=20,
                    svnbin='/usr/bin/svn',
 )
c['sources'] = [ svn_poller ]

After this we configure the schedulers to which the change objects are passed by the class that implements "IChangeSource" (See above).
The scheduler then decides if this should trigger a build.

from buildbot.scheduler import Scheduler
c['schedulers'] = []
c['schedulers'].append(Scheduler(name="all",
                                 branch=None,
                                 treeStableTimer=0,
                                 builderNames=["Buildbot-App"]))

Now we configure the builders that can be invoked on the build slave.

from buildbot.process import factory
from buildbot.steps import source
from buildbot.steps.shell import ShellCommand

integratedBuildFactory = factory.BuildFactory()

After having setup a builder, we define the build steps that would be invoked when a build is triggered on the associated slave. These build steps are associated to a builder and can be different for each builder. This allows us to do different things for different environment like different os platforms.
There are various ways to build the steps. Some of the classes of interest is code checkout , ShellCommand etc. Failure of any step is marked as failure unless flunkOnFailure attribute is false for the following build step.

# Code Checkout
integratedBuildFactory.addStep(source.SVN(name="checkout",
                     mode='clobber',
                     baseURL='http://...',
                     defaultBranch='trunk'))
                     
#Build the code
integratedBuildFactory.addStep(ShellCommand(name="BuildAPP",
      description="Building application code",
      descriptionDone="build complete",
      command="shell commands to execute "
                                   )
                              )
                               
# Other test steps as required by the application

# Processing result
integratedBuildFactory.addStep(ShellCommand(
                          name="step description",
                          description="processing results",
                          descriptionDone="Results processed to html",
                          flunkOnFailure=False,
                          command="shell command to execute ...."
                       ))
                                                              
                     

By now, we have created a builder, build slave. Now we will be associating a particular builder with the build slave

b1 = {'name': "Buildbot-App",
      'slavename': "build-slave1",
      'builddir': "builds-directory-name",
      'factory': integratedBuildFactory,
      }
c['builders'] = [b1]

Now we setup some notification mechanism, for buildbot to notify the results of a build trigger.

c['status'] = []

Setup the web interface to access buildbot status.

from buildbot.status import html
c['status'].append(html.WebStatus(http_port=8010,allowForce=True))

To setup mail notification, the following should help. We will be creating a custom mail template for notification.

from buildbot.status import mail
def message(attrs):
        logLines = 20
        text = []
        color = {"Success":"#33CC66", "Other":"#FF6633"}
        stepresult = {
                    "checkout":"Code checkout",
   "CompareWebappResult":"App Test Results"
        }
    
      # rest of the textual template details
      
c['status'].append(mail.MailNotifier(
                     fromaddr="BuildBot@test.com",
                     extraRecipients=[aed@test.com, "bed@test.com"],
                     sendToInterestedUsers=True,
                     relayhost="relay server name",
                     customMesg=message))
    

Finally, you may want to create a customized web result display

c['projectName'] = "App Server BuildBot"
c['projectURL'] = " project web page"
c['buildbotURL'] = "http://buildbot_server_name:8010/"

With this the buildbot setup is complete.