Buildbot - Issue with svn poller

SVN poller may miss a check-in based on poll interval.

The current behavior of the poller is

The poller polls the version control system and stores last change (version number).  Subsequent changes are notified as log entries. These log entries are marked with the Time Stamp when the changes are noticed. These log entries are used to create change objects that is then passed to scheduler to trigger builds.Scheduler sees these change objects with the same timestamp and picks the latest change object to trigger the build.

The issue with this model is that if there are multiple changes within a single polling interval, this poller will result in triggering build only for the last one.

Some solution though half baked is to

1. Decrease the polling interval to say 30 sec. While this may decrease the number of errors or missed build. It does not guarantee that the builds may not be missed. It may only decrease their numbers.
2. The other solution is to use a script ( a subversion hook-script which submits changes to svn ) to trigger buildbot builds when a developer checks in the code. If this is server side hook then it is great. The issue is not every one has the privilege of setting up server side hooks.
3. Use a watcher which behaves like svn poller except that it needs to watch for every commit and send the change objects per commit to buildbot.

There is a file - "svn_watcher.py" distributed with buildbot. This scripts does the usual polling but suffers from the same lacunae as that of svn poller.

The way to run this file is

./svn_watcher.py subversion_url host_buildbot_master:port watch

Overcoming limitations of svn_watcher.py

Issue 1: Does not work in –non-interactive mode

Run the following command for the first time to enable appropriate authentication

svn log --xml --verbose --limit=1 subversion_url

Issue 2: Making it notice every commit and notify it to buildbot

# Check if multiple revisions have been made during this interval
    if oldRevision != -1:
        # Have to discount the state loss
        revJump = int(revision) - int(oldRevision)
        #print "Jump in Revision: " + str(revJump)
        if revJump > 0:
            # We have to get information about every change
            cmd_multilog = ["svn", 
                            "log", 
                            "--xml", 
                            "--verbose", 
                            "--non-interactive",
                            "--limit="+ str(revJump + 1), 
                            repo]
            if sys.platform == 'win32':
                f = win32pipe.popen(cmd_multilog)
                xml2 = ''.join(f.readlines())
                f.close()
            else:
                xml2 = getoutput(cmd_multilog)
            # Process multiple log Entries
            doc2 = xml.dom.minidom.parseString(xml2)
            logentries = doc2.getElementsByTagName("logentry")
            changedetails = []
            #print  "Len of logentries: " + str(len(logentries))
            for logentry in logentries:
                el_revision = logentry.getAttribute("revision")
                if int(el_revision) > int(oldRevision):
                    el_author = "".join([t.data for t in
                    logentry.getElementsByTagName("author")[0].childNodes])
                    el_comments = "".join([t.data for t in
                    logentry.getElementsByTagName("msg")[0].childNodes])
                    el_pathlist = logentry.getElementsByTagName("paths")[0]
                    el_paths = []
                    for p in el_pathlist.getElementsByTagName("path"):
                        el_paths.append(
                                  "".join([t.data for t in p.childNodes
                                   ]))
                    changedetails.append([el_revision, 
                                          el_author, 
                                          el_comments, 
                                          el_paths])
            changedetails.reverse()
            #print "Len of change details: " + str(len(changedetails))
            # Send change details
            for changedetail in changedetails:
                chng_cmd = ["../../bot/bin/buildbot", 
                            "sendchange", 
                            "--master=%s"%master,
                            "--revision=%s"%changedetail[0], 
                            "--username=%s"%changedetail[1],
                            "--comments=%s"%changedetail[2]
                           ]
                chng_cmd += changedetail[3]
                if verbose == True:
                    print chng_cmd
                if sys.platform == 'win32':
                    f = win32pipe.popen(chng_cmd)
                    print time.strftime("%H.%M.%S ") + \
                          "Revision "+changedetail[0]+ ": "+ \
                          ''.join(f.readlines())
                    f.close()
                else:
                    print time.strftime("%H.%M.%S ") + \
                                        "Revision "+changedetail[0]
                    # Send buildbot command
                    out_xml  = getoutput(chng_cmd)
                currentRevision = changedetail[0]
        else:
            print time.strftime("%H.%M.%S ") + \
                                "nothing has changed since revision "+ \
                                revision
            currentRevision =  revision
    else:
        #currentRevision = revision
        currentRevision = str(int(revision) - 2)
        print time.strftime("%H.%M.%S ") + \
                            "Monitoring for change after revision " + \
                            currentRevision

    return currentRevision