For a new project at New Relic, my new employer, I had to develop a test infrastructure for an existing product. I needed three services:
- Test HTTP app server
- Fake data collection service that gather data through a REST API
I also wanted to perform some headless testing, so I decided to use phantomjs. That meant I also needed to start a phantomjs service. So now I have to start four services before running any tests. This led me to investigate Whiskey. Whiskey has a really nice dependencies feature where you can define service dependencies of individual tests. It then looks up the services in a dependencies.json file and fires them up. Once the tests completed, it kills the services.
But I found three main issues with Whiskey. First, it only works properly if you use Whiskey’s built-in assertion library. I prefer the Chai assertion library, mostly because it has some extra useful assertions that Whiskey lacks. Secondly, it runs tests in separate processes, which is great for parallelism but painful if you need to debug your tests. Lastly, although Whiskey has the wonderful dependencies feature, it does a poor job of tearing them down when testing dies unexpectedly. After repeatedly checking and killing leftover processes after many tests, I decided to find something else.
I decided to rejoin the beaten path and check out the popular Mocha test framework. However, Mocha doesn’t have any dependency running functionality. After some searching and playing around, I found that the Grunt JS task runner has the perfect plugin: external_daemon. The combination of external_daemon then mocha wrapped in a Grunt task provides the dependency running I needed while allowing me to use the more stable and versatile Mocha test framework.
I’m not sure we’ll stay with this setup long term, but it seems to be working alright so far. We’ve integrated this test framework into a new continuous integration/deployment pipeline using Jenkins. So far, I’m quite pleased!