Ambysoft logo

Results from the November 2012 Agile Testing Survey

Follow @scottwambler on Twitter!

How to Measure Anything 2nd Edition This survey was performed late October through mid November 2012 and there was 178 respondents. The survey was announced on the IT Surveys page, via twitter, a posting to the Disciplined Agile Delivery (DAD) discussion forum, and to attendees of the Agile Testing Days 2012 conference held in Potsdam (hence the large percentage of European responses). The primary goal of this survey was to find out what testing and quality strategies people have adopted on agile project teams.

The Survey Results

Some findings include:

Figure 1. Primary approach to acceptance testing.

Acceptance Testing 2012

Figure 2. Primary approach to developer testing.

Developer Testing 2012

Table 1. The most difficult challenges when adopting agile testing approaches.

Responses Challenge
50% Getting all testing done in the current iteration/sprint
37% Adopting test-driven development (TDD) approaches
33% Validating non-functional requirements
33% Getting stakeholders/customers involved with testing
27% Getting developers to test their own code
21% User interface testing
16% Learning to test throughout the agile lifecycle
13% Adopting new agile testing tools
12% Migrating existing testing and quality professionals to agile
8% Using our existing testing tools to support agile development
8% Remaining regulatory compliant


Survey questions

The Survey Questions

Survey Data File

Raw Data

Survey Presentation

Summary Presentation

What You May Do With This Information

You may use this data as you see fit, but may not sell it in whole or in part. You may publish summaries of the findings, but if you do so you must reference the survey accordingly (include the name and the URL to this page). Feel free to contact me with questions. Better yet, if you publish, please let me know so I can link to your work.

Discussion of the Results

  1. People had a good idea what the topic of the survey was based on the title. Furthermore, many of the responses were from people attending an agile testing conference.
  2. Interestingly, given the predominance of "pro agile testing" people, it was interesting to see how many traditional testing and quality strategies are still being followed.
  3. This survey suffers from the fundamental challenges faced by all surveys.

Links to Other Articles/Surveys

  1. My other surveys

Why Share This Much Information?

I'm sharing the results, and in particular the source data, of my surveys for several reasons:

  1. Other people can do a much better job of analysis than I can. If they publish online, I am more than happy to include links to their articles/papers.
  2. Once I've published my column summarizing the data in DDJ, I really don't have any reason not to share the information.
  3. Too many traditionalists out there like to use the "where's the proof" question as an excuse not to adopt agile techniques. By providing some evidence that a wide range of organizations seem to be adopting these techniques maybe we can get them to rethink things a bit.
  4. I think that it's a good thing to do and I invite others to do the same.