IT Project Success Rates: 2013 Open Research

This open research into IT project success rates was performed during November and December 2013 and there was 173 respondents. The survey was announced in my October 2013 DDJ artcile, on the Ambysoft announcements list, my Twitter feed, and several LinkedIn discussion forums (Disciplined Agile Delivery, Greater IBM connection, and Considerate Enterprise Architecture Group). The survey results Downloads What you may do with the information Discussion of the results Links to other articles/surveys Why share this much information?

The Survey Results

The survey results are summarized in my January 2014 Dr. Dobb’s Journal article The Non-Existent Software Crisis: Debunking the Chaos Report.

Some findings include:

• As you can see in Figure 1, Agile and Lean strategies are more effective than traditional strategies on average.
• As you can see in Figure 1, ad-hoc project teams (no defined process) and traditional project teams have lower success rates that agile/iterative project teams
• Figure 2 compares the effectiveness of the five paradigms for delivering in a timely manner, for providing good ROI, for delivering value to the stakeholders, and for producing a quality product.
• As you can see in Figure 3, there is no common definition of software development success.
  • When it comes to time/schedule, 16% prefer to deliver on time according to the schedule, 39% prefer to deliver when the system is ready to be shipped, and 42% say both are equally important
  • When it comes to ROI, 13% prefer to deliver within budget, 60% prefer to provide good return on investment (ROI), and 23% say both are equally important
  • When it comes to stakeholder value, 4% prefer to build the system to specification and 86% prefer to meet the actual needs of stakeholders, and 10% say both are equally important
  • When it comes to quality, 10% prefer to deliver on time and on budget and 56% prefer to deliver high-quality, easy-to-maintain systems, and 34% say both are equally important
  • Only 8% of respondents indicated that their definition of success included all three of delivering according to schedule, within budget, and to the specification (answers where both was indicated were included in this calculation).

Figure 1. Perceived IT project success rates by paradigm.

Figure 2. Success factors by paradigm (Scale is from -10 to +10).

Figure 3. How do people define success?

Downloads

<

The Survey Questions

Raw Data

Summary Presentation

What You May Do With This Information

You may use this data as you see fit, but may not sell it in whole or in part. You may publish summaries of the findings, but if you do so you must reference the survey accordingly (include the name and the URL to this page). Feel free to contact me with questions. Better yet, if you publish, please let me know so I can link to your work.

Discussion of the Results

1. It’s difficult to get a good estimate of project success rates because there isn’t a standard definition of success (nor will there ever be). So, if I define success specifically, for example as “reasonably on time, on budget, to specification” that definition will be applicable for some projects but not others. So, I could presumably get an accurate estimate of how well we’re doing against that criteria but it wouldn’t be the actual industry success rate. If, however, I define allow people to define success in terms of how it was defined for the actual projects then I’ll get a much more accurate estimate of project success rates but I won’t know exactly
2. This survey was huge, up to 50 questions, so the response rate was a bit lower than usual.
3. This survey suffers from the fundamental challenges faced by all surveys.

Links to Other Articles/Surveys

1. My other surveys

Why Share This Much Information?

I’m sharing the results, and in particular the source data, of my surveys for several reasons:

1. Other people can do a much better job of analysis than I can. If they publish online, I am more than happy to include links to their articles/papers.
2. Once I’ve published my column summarizing the data in DDJ, I really don’t have any reason not to share the information.
3. Too many traditionalists out there like to use the “where’s the proof” question as an excuse not to adopt agile techniques. By providing some evidence that a wide range of organizations seem to be adopting these techniques maybe we can get them to rethink things a bit.
4. I think that it’s a good thing to do and I invite others to do the same.