Bane is a tool that makes it easy to test your application in scenarios where any third-party server dependencies behave in unexpected ways. This post explains why you might want to use Bane, how to get started, and the design motivations behind Bane.
Problem: The World is Unsafe (For an Application)
If you are building an application, you may depend on third-party servers or other external sources for your data. You may use any of the publicly available web services from companies like Google, Yahoo!, or Twitter. Or perhaps you use web services in an internal business application to obtain data from authoritative sources. Most of the time these services are reliable and work as you expect.
Yet however reliable these services claim to be, at some point they’ll behave in an unusual manner. The servers may be busy and accept your connection but never respond, they might reply in an unexpected response format, or there might be network problems that require endless retransmits of the data - and that's just a few of the ways that things can go wrong. It’s up to you to program defensively so that if the remote service fails, your application doesn’t fail as well. Michael Nygard’s excellent book “Release It!” is full of patterns and anti-patterns for building applications that can survive in these situations; I can’t recommend it highly enough.
To make sure your application isn’t susceptible to external service failures, you should test your application under various hostile conditions. However, getting your external services to behave in the desired fashion can be a real challenge. Instead, Nygard recommends building a tool that can simulate various bad behaviors, as described in the chapter “Test Harness.”
Bane is an open source implementation of the test harness described in “Release It!” Bane is written in Ruby and is available as a Ruby gem. To install it, you’ll need a working installation of Ruby. Install Bane with “gem install bane” which makes an executable “bane” available from the command line. To make sure Bane is installed correctly, type “bane” with no arguments from the command line. You should see a usage message like this:
Usage: bane port_number <servers>
Bane requires you specify a port number on which to start listening and a behavior to use to respond on that port. A behavior is one of the several nefarious ways in which Bane can respond, such as never responding, sending random data, closing the connection immediately, etc. Behaviors are named in CamelCase (e.g. NeverRespond, NewlineResponseForEachLine) because they correspond to Ruby classes.
A Usage Example
Imagine you’ve built an application to track financial investment portfolios. All stock quotes are obtained via a third-party web service. You’d like to see how your application behaves when the web service accepts your request but never sends a byte of data in response. Let’s pretend that your stock quote service listens on port 8080.
Start up the NeverRespond behavior on port 8080, which accepts a connection but then never sends a response.
$ bane 8080 NeverRespond
[Tue Sep 28 10:49:25 2010] NeverRespond 127.0.0.1:8080 start
Point your application at port 8080 and see how your application behaves. Notice that Bane logs the incoming connections:
[Tue Sep 28 10:50:05 2010] NeverRespond 127.0.0.1:8080 client:61459 127.0.0.1<127.0.0.1> connect
You may need to add timeouts or other failure strategies to keep your application functioning under this scenario. Consult “Release It!” for more information and other alternatives.
The Design Motivations Behind Bane
Before I created Bane, I used to write little one-off scripts to create these badly behaving servers. They aren’t much work; Ruby’s standard library includes a GServer class that makes it easy to quickly create a multithreaded TCP/IP server. In fact, if you look at the source code, you’ll see that Bane uses GServer internally. So why not just create these little servers each time by hand?
It turns out that each time I had to create one, I’d take about five minutes to bring up the RDoc and get all the parameters right. So I created Bane to have all these common bad behaviors available in just a few seconds. Right now Bane only supports two response protocols: send a response immediately upon connection and respond after each newline sent by the client. I haven’t needed other response protocols on my projects yet, but it should be simple to create variations that respond after every n bytes, etc.
However, if it turns out there needs to be some advanced protocol (e.g. respond after certain keywords, preserve state, etc.) then you may be better off spending the five to ten minutes to write your own server in Ruby using GServer or Sinatra, or some other language of your choice. My goal is to keep Bane easy to use for common cases, to get a bad behavior up and running in a few seconds rather than five minutes.
Manual or Automated?
I use Bane primarily for manual exploratory tests, and write unit tests around my applications to create fast, repeatable automated tests. Typically I use Bane in my consulting work to test out an existing application and see how it behaves, find the failure modes, write automated unit tests to expose the behavior, and then fix the problem by applying the patterns described in “Release It!” Many of these unit tests use test doubles to throw exceptions that would be created by the I/O library in use.
That said, Bane provides an API through the Bane::Launcher class for automation in Ruby. Take a look at Bane’s integration tests and the examples folder in the source distribution for an example of this API in use.
How to Contribute
Bane is open source and I welcome your feedback or contributions. Here are a few ways you can contribute:
Provide Feedback: Please take Bane for a spin on your project and let me know if it helps, where it didn’t, and any suggestions for enhancements.
Write new behaviors: Fork Bane on GitHub and contribute useful bad behaviors or any other items described in “Release It!” which are not yet implemented (see the README for a list). In particular, Bane does not yet support any of the low-level protocol errors in “Release It!”, such as the remote end endlessly sending TCP/IP resend packets. To my knowledge, this low-level behavior is not possible with Ruby’s I/O library and requires a C extension.
Code Reviews: If you know Ruby and have some ideas about how to improve the code, please fork Bane and suggest some changes.