crowd testing

How Google Tests Software – my review

 

 


James Whittacker, Jason Arbon & Jeff Carollo

First published 2012


What’s a developer doing reading a book on testing? Isn’t that something for the Tester to read?

…was how I once thought. These days my mindset is along the lines that a good developer is more than just being able to write good code. It’s more about being part of a team that delivers software. We need to be involved with defining the story of what our customer wants, writing the code using TDD techniques and being able to work with a tester to verify that we are delivering a piece of quality to our customer.

This book is focussed on the roles of the people who make Google’s software. All roles require some developer nous. Each part of the book goes into some detail, explaining what that role covers within Google. This book assumes you are familiar with development and testing practices.

The four main roles are:

  • The Software Engineer (SWE): A developer who delivers features. Of course done using TDD and also ‘participates in Tests’.
  • The Software Engineer in Test (SET): A developer focused on testability, quality and risk. Works closely with the SWE to help create test code, such as mocks and stubs. The book goes on to say they even create Testing Frameworks. Their goal is to test the developer and ensure code is of a high standard and consistent with the rest of the codebase.
  • The Test Engineer (TE):  Focussed on testing the user. i.e. ensuring that what’s getting delivered is what the user asked for and is of a good level of quality. They write test automation scripts. They are product experts. They are Quality and risk analysers.
  • The Test Engineer Manager (TEM): Primarily responsible for coordinating TE and SET. A take away for me was that these people are really product experts on top of really knowing their people.

Crowd Sourcing

Google also has the clever use of getting the public to tests its software for them. They call it crowd sourcing, it’s a natural extension of dog fooding (which they also do). The idea is to get users to run through tours, execute, specific testing and perform exploratory testing. They even have incentives for  bug finders! The Test Engineer’s role as I understood it was to coordinate this, to try and get a good level of coverage and facilitate how bugs are fed back to the developers.

The Future…

The final chapter is disappointingly not about killer robots, self driving cars or transportation devices. Whittacker waxes lyrical about how he sees the testing landscape at Google evolving over the next few years. Like combining SET and SWE roles. This was a thought I also had when reading the book –  if you’re a strong SWE – surely a lot of what the SET does are tasks you would do yourself. Especially writing mocks and stubs – for me an important tool I use when writing my code with a TDD approach. They are also using automation tools like Selenium and encouragingly he sees that having SWE/SETs contributing to open source frameworks such as Selenium is much more beneficial as opposed to the cost of innovating and maintaining an in-house testing framework.

Test engineers are reducing all the time. One key reason here is feedback. Making use of continuous deployment mechanisms they can get software to the customer (live, crowd testers, dog fooders) fast. Bugs can be raised fast and developers can fix these bugs and get them back out to production, fast. I read from this that TE will become more involved with orchestrating crowd tests and writing automation scripts and less involved with manual testing.

Google – just another software department?

This was an enjoyable and easy read but for me didn’t rock my world so to speak. It’s nice to know that the guys at Google aren’t doing anything significantly different to what I’ve seen in other good IT departments in terms of their approach. There is a focus on testing the ACC (Attributes, Components, Capabilities) of its software as opposed to test plans. I think this could read as Specification by Example, as the focus is on what problem the software is solving without getting bogged down in how it’s doing it. If this interests you, should read the book.

Advertisements