Behat is a relatively new testing framework, many developers are using it in personal projects and teams are beginning to use it for production projects as well. Recently a number of people have asked me about how Behat testing is integrated into larger projects involving teams of developers, testers and other stake holders. Common questions include:
How does a story get from inception to implementation?
Which members of the team are responsible for maintaining Gherkin and Steps?
What are the benefits of using an integration testing tool over just unit testing?
What role does Behat testing play in the sign off process?
I’ll answer some of these questions in this post by looking at the reasons Behat was adopted at my previous employer SkyBet* and the workflows developed to support it’s use. We started using Behat at the v1.0 stage and over time hammered out a process to get a story from inception to build, giving each member of the production team responsibilities and putting a process in place for creating acceptance criteria.
Some background is required as to the kind of applications and products developed at Sky to understand why Behat and integration testing were deemed a necessary part of our testing plan. SkyBet have a number of customer facing websites including Sky Vegas, Sky Bet and Sky Poker. As you can imagine with a gaming website, handling customer interactions, data from gaming information providers as well as general CMS content complexity builds up quickly in many areas of the architecture. With this context it’s easy to see why testing that each level of the architecture integrates correctly is important. This accomplished this by testing all internal API’s consumed by the various Sky products using Behat.
Behat proved to be a great tool as it allowed all members of a multi disciplined team to understand what was the required outcome of a story and what features of that story were covered in the tests. This was a benefit provided by Gherkin, the human readable test description language at the heart of Behat.
What do I mean by a multi disciplinary team? This is a team composed of a number of different roles: The product owner, business analysts, testers and developers. It’s common to find teams like this in bigger organisations or software houses, in smaller organisations the roles still exist but members of a team may ‘wear many hats’.
I’ll now walk through the process of defining a story, testing it and developing it with reference to the roles listed above.
At the beginning of a cycle the product owner will define objectives (usually taking them from a product backlog). These objectives are then formalised into stories by the business analysts. This process involves producing a small requirements document with information about the feature and suggestions on key points of functionality to test. They are well placed to do this because of their in depth product knowledge and experience of the business domain.
This document is then passed to the tester who will own the testing process for the story. The tester then translates the requirements doc into Gherkin. In theory BA’s could write the Gherkin, it’s human readable, they should just be able to write what they expect the test to accomplish. The reason why this isn’t the approach taken is that it is the testing teams responsibility to maintain the Gherkin dialect used in the test suite. Their in depth knowledge of which steps to use to accomplish certain tasks (data creation and verification steps in particular) ensure maximum step re-use. The output of this stage is a battery of Gherkin scenarios designed to test all the behaviour of the feature under test. The task is then passed to the developer/s responsible for implementing the feature.
The developer is in a great position at the start of the development phase of the cycle as they have a well defined feature to develop. In later stages of a project when a large number of steps have already been implemented the Behat tests may even run without the development of additional steps. This is one of the key benefits of having experts in a projects ‘dialect’ of Gherkin, step reuse.
At this point the developer would implement any undefined steps and then use the tests to develop the feature against.
The final stage of the test / development cycle is formal acceptance of a feature by testers and then BA’s. Providing the feature was well defined in the early stages of the cycle a green Behat test suite should prove a feature is complete. In reality manual testing is often required to verify the scenarios have been interpreted and implemented correctly.
Thats a summary of the integration and acceptance process employed at SkyBet. It provides real benefits: the time taken to develop a feature decreased (less back and forth between the parties involved in test/dev) and decreased defects in production (continually running the tests at build time catches errors pre deployment).
I think the process defined here also stays true to the BDD principles which Behat extolls. Stories (and behaviour) are defined by stakeholders and are written in Gherkin a language all stake holders can understand. Developers are supplied with tests ahead of feature development so they can use them to produce the feature closely to the specification.
Are you using Behat in production or within a larger team? Let me know what you think of these workflows in the comments section.
* My relationship with SkyBet: Until recently I worked as a Software Engineer at SkyBet. A move to London to and contracting meant I finished working there in Jan 2012. They are an excellent company and are also recruiting!
nice, where are you now?
I’m contracting for a media company in London. Exciting times!