Behavior Driven Systems Monitoring for Telephony
While attending Euruko several weeks ago, I had the opportunity to see @aslak_hellesoy present the Cucumber Behavior Driven Development (BDD) framework. BDD is a part of Agile software development focused on bringing the domain expert into the specification process through the use of natural language. This is done through the users and developers creating a story that is directly parsed by the code for tests.
Aslak’s presentation was well done, giving some compelling reasons to adhere to BDD principals, showing that using Cucumber made it a joy. Thing is, Cucumber is already a part of testing for the Adhearsion framework and some specific projects we are working on. What was new to me, was a lightning talk given at the end of the day on Cucumber-Nagios.
Cucumber-Nagios is a project developed by @auxesis that bridges Cucumber test results into the Nagios monitoring tool. This allows you to take any test you have written and allow the steps to become part of an on-going systems monitoring strategy. While Cucumber-Nagios is geared towards web application testing, you may use this beyond to test virtually any application.
Naturally I have started using Cucumber-Nagios to create a series of tests to do application level monitoring of Adhearsion and Asterisk. This now allows one to not only ensure that these processes are running, but they are working as defined by the BDD specification.
To start, I created a couple of features that will connect to Adhearsion and/or Asterisk. Then ping the Asterisk Manager Interface (AMI) looking for a corresponding pong, ensuring the systems are still responsive.
The feature to ping the AMI uses the straight forward Gherkin language to create a feature file with the test specification:
The feature file is then parsed by the steps definitions as set out in the step file:
The methods called in the step definitions then invoke the system testing library I wrote specifically for these telephony systems. The result is having a standalone test framework that may provide direct input at an application level to Nagios, as well as provide plain English feedback to anyone looking to track down the cause of the problem.
First, if Asterisk is down we may see the result of running the Cucumber plain English test:
Then the corresponding Cucumber-Nagios test that also fails with a ‘Critical’ error:
Now, if Asterisk is up and all is fine we may see these same tests pass. First, for Cucumber all is green:
Then for Cucumber-Nagios we see all 3 tests passed:
The great thing about this approach is that it may be used to test systems using a standard open source dynamic language like Ruby. These tests are not limited to testing Ruby systems though, as you may see in this case I am testing the Asterisk open source telephony engine. Further, I may test not only the standard interfaces of various systems, but that the behavior of my application deployed within these systems is behaving as required from a specification perspective. This is a powerful approach to systems monitoring and troubleshooting, pulling the system administrators directly into the development process.
You may check out my telephony-system-tests project on Github. My plan is to greatly extend this system testing framework over time in order to have a stable of standard tests for Adhearsion, Asterisk, Freeswitch and others in the future. One of the next steps will be to integrate SIPr into the framework for testing SIP dialogs as well.