Agile Testing: 2012 Open Research
The Survey Results
Some findings include:
- Figure 1 summarizes the responses regarding the primary approach to acceptance testing on agile teams. The good news is that the majority of agile teams are doing some sort of acceptance testing. Unfortunately, not as many as I would have hoped are taking a test first strategy, such as acceptance test-driven development (ATDD)/behavior driven development (BDD) as their primary approach. This is likely because these strategies require great discipline and skill to follow. It’s interesting that parallel independent testing approaches are still more popular than test-first approaches for acceptance testing.
- Figure 2 summarizes the responses regarding the primary approach to developer testing on agile teams. In this case test-first approaches such as TDD are a bit more common, although test-after still dominates. Table 1 summarizes what people believe to be the most significant challenges that they face when adopting Agile Testing and Quality Strategies.
Figure 1. Primary approach to acceptance testing.
Figure 2. Primary approach to developer testing.
Table 1. The most difficult challenges when adopting
agile testing approaches.
| Responses | Challenge |
| 50% | Getting all testing done in the current iteration/sprint |
| 37% | Adopting test-driven development (TDD) approaches |
| 33% | Validating non-functional requirements |
| 33% | Getting stakeholders/customers involved with testing |
| 27% | Getting developers to test their own code |
| 21% | User interface testing |
| 16% | Learning to test throughout the agile lifecycle |
| 13% | Adopting new agile testing tools |
| 12% | Migrating existing testing and quality rofessionals to agile |
| 8% | Using our existing testing tools to support agile development |
| 8% | Remaining regulatory compliant |
Downloads
|
|
|
What You May Do With This Information
You may use this data as you see fit, but may not sell it in whole or in part. You may publish summaries of the findings, but if you do so you must reference the survey accordingly (include the name and the URL to this page). Feel free to contact me with questions. Better yet, if you publish, please let me know so I can link to your work.
Discussion of the Results
- People had a good idea what the topic of the survey was based on the title. Furthermore, many of the responses were from people attending an agile testing conference.
- Interestingly, given the predominance of “pro agile testing” people, it was interesting to see how many traditional testing and quality strategies are still being followed.
- This survey suffers from the fundamental challenges faced by all surveys.
Links to Other Articles/Surveys
Why Share This Much Information?
I’m sharing the results, and in particular the source data, of my surveys for several reasons:
- Other people can do a much better job of analysis than I can. If they publish online, I am more than happy to include links to their articles/papers.
- Once I’ve published my column summarizing the data in DDJ, I really don’t have any reason not to share the information.
- Too many traditionalists out there like to use the “where’s the proof” question as an excuse not to adopt agile techniques. By
providing some evidence that a wide range of organizations seem to be adopting these techniques maybe we can get them to rethink things a bit. - I think that it’s a good thing to do and I invite others to do the same.