Process Frameworks in IT: 2008 Open Research
|This open research into process frameworks in IT was performed in mid January and there was 339 respondents. The survey was announced in the blog of Jon Erickson, the Dr. Dobb’s Journal editor, and in an email to the DDJ readership.
The results of the survey are summarized in my February through April 2008 newsletters, Agile CMMI?, How Effective are Process Frameworks?, and Repeatable Results over Repeatable Processes respectively.
Some findings include:
- 127 respondents indicated that their organizations were doing agile projects
- 13 (10%) indicated that they were exclusively doing CMMI-compliant agile projects (see Figure 1)
- 57 (45%) said they were exclusively doing non-CMMI agile projects (see Figure 1)
- 57 (45%) said that their organization had both CMMI compliant and non-CMMI projects (see Figure 1)
- On average, CMMI apparently has no statistical effect on project success rates in practice.
- Most people have a preference for repeatable results over repeatable processes
- 23% of all respondents hadn’t heard about CMMI
- 52% of IT managers had not heard about COBIT
- 32% of IT managers had not heard about ITIL
- 54% of architects hadn’t heard about TOGAF
- 38% of architects hadn’t heard about the Zachman Framework
- The various process frameworks appear to offer minimal benefit in practice.
You may use this data as you see fit, but may not sell it in whole or in part. You may publish summaries of the findings, but if you do so you must reference the survey accordingly (include the name and the URL to this page). Feel free to contact me with questions. Better yet, if you publish, please let me know so I can link to your work.
The response rate was unusual for us, I suspect because of the topic: Only 339 started the survey, the lowest survey response rate that we’ve ever gotten, and only 219 completed it. I suspect that many people were turned off by the second of five ages which explored the adoption rate and effectiveness of the frameworks (CMMI, COBIT, ITIL, PMI PMBoK, Prince2, TOGAF, and Zachman Framework).
- The survey results are likely biased because we told people up front that the survey focused on frameworks. Therefore, it’s likely that people with an interest in frameworks would respond to the survey.
- The CMMI results seem very high. When surveying the agile community, only 9% of people responded to being currently on an agile project that was CMMI complian.
- This survey suffers from the fundamental challenges faced by all surveys.
I’m sharing the results, and in particular the source data, of my surveys for several reasons:
- Other people can do a much better job of analysis than I can. If they publish online, I am more than happy to include links to their articles/papers.
- Once I’ve published my column summarizing the data in DDJ, I really don’t have any reason not to share the information.
- Too many traditionalists out there like to use the “where’s the proof” question as an excuse not to adopt agile techniques. By
providing some evidence that a wide range of organizations seem to be adopting these techniques maybe we can get them to rethink things a bit.
- I think that it’s a good thing to do and I invite others to do the same.