Comparing Approaches to Estimating Software Development

Software requirements elicitation, budgeting, and scheduling are highly interrelated – take the wrong approach with one of them and your entire software development process falls like a house of cards. This article argues that although traditional estimating approaches are comforting to senior managers, the research shows that they do not ensure predictability and worse yet result in spectacular levels of waste. Agile approaches to change management, modeling, and documentation are required if your organization is to succeed at becoming more agile with its approaches to estimating. This will require a significant paradigm shift on the part of both IT and business management.

Everyone claims they want to be agile, but when push comes to shove most people will choose the politically expedient approach over actual effectiveness. For example, what is more important to you: estimating the actual costs of an initiative and then bringing it in on budget or spending your IT investment wisely? If your answer is the first then you had might as well stop reading. If your answer is the second then this article is for you, and if you think that it’s possible to achieve both then you desperately need to read this article to dispel you naivety.

In this article I describe, and then compare, three approaches to estimating budget and time:

1. A traditional approach
2. An agile approach
3. A middle of the road approach
4. Comparing the strategies

1. A Traditional Approach to  Estimating

The traditional approach to estimating is to invest the time to develop what is considered to be an “accurate” budget and detailed schedule early in the lifecycle. To do this, a comprehensive definition of the requirements is typically produced during the initiation phase of the initiative, often simply called the requirements phase. Sometimes a detailed design document is developed, providing more information for the budgeting and scheduling process. “Smart” organizations will develop an initial estimate of your budget and schedule early in the effort, then refine it over time as more information becomes available. Not so smart organizations will assume that the initial estimates are official, and actually hold the team to them. This is particularly true of organizations which take a fixed bid approach to software development, a spectacularly questionable practice at best.

Traditional teams take a (near) serial approach to development in which requirements are defined and documented early in the initiative, the requirements may or may not be reviewed and accepted by the stakeholders, and then they’re provided to developers who are expected to build a system based on those requirements. Scope creep, also known as requirements creep, is contained by requiring stakeholders to follow a change management process. Traditional change management processes typically involve one or more people, often referred to as a change control board (CCB), who act as a gateway through which change requests are evaluated and potentially accepted. The goal is to minimize, if not prevent, scope creep so as to stay on budget and schedule. This approach sounds great in theory, but in practice proves to work poorly. In Examining the Big Requirements Up Front (BRUF) Approach I describe Figure 1 in detail, which shows that when you take a traditional approach to requirements elicitation and management that 45% of functionality delivered is never used and a further 19% is rarely used. In other words, a serial approach to development results in nearly 2/3 wastage (and that’s only considering the teams which actually deliver into production, so the actual figure is much worse).

Figure 1. The cost of serial approaches to requirements.

Fixed price bids/contracts are bad ideas for both IT and the business stakeholders. Sadly, the business thinks that it’s reducing its financial risk with a fixed bid, but the reality is that it forces them into a position where their money is almost always wasted (see Figure 1). For IT, it almost always motivates them to reduce the delivered quality due to the realities of the iron triangle. It is management decisions like that that have led to the high levels of data technical debt in organizations.

2. An Agile Approach to Budgeting and Scheduling

Now let’s consider the other end of the spectrum. Here, we stop worrying about how much things are going to cost, or how long it will take, and instead focus on spending your money wisely and working in a timely manner. Easy to say, but hard to do. Agile software development teams do this by accepting the fact that requirements change and therefore choose to adopt practices which allow them to react accordingly to those changes. These practices include an agile approach to change management, to modeling, and to documentation. However, to achieve this management must adopt new ways of budgeting and scheduling in parallel.

Because requirements change frequently you need a streamlined, flexible approach to requirements change management. Agilists want to develop software which is both high-quality and high-value, and the easiest way to develop high-value software is to implement the highest priority requirements first. Agilists strive to truly manage change, not to prevent it, enabling them to maximize stakeholder investment. Figure 2 overviews the agile approach to managing requirements, reflecting both Extreme Programming (XP)’s planning game and the Scrum methodology’s approach. Your software development team has a stack of prioritized and estimated requirements which needs to be implemented – many XP teams will literally have a stack of user stories written on index cards although some teams use spreadsheets or databases to manage their requirements stack. The team takes the highest priority requirements from the top of the stack which they believe they can implement within the current iteration. Each requirement is initially estimated when it is identified and put on the stack, but when it comes time to implement the requirement some model storming may occur to ensure that the estimate is accurate. Scrum suggests that you freeze the requirements for the current iteration to provide a level of stability for the developers whereas in XP the current requirements can also change as long as you’re willing to drop some requirements out of the iteration/timebox if needed.

Figure 2. Agile change management.

Developers are responsible for estimating the effort required to implement the requirements on which they will work. Although you may fear that developers don’t have the requisite estimating skills, and this is often true at first, the fact is that it doesn’t take long for people to get pretty good at estimating when they know that they’re going to have to live up to those estimates. Smaller requirements are easier to estimate. Shall statements, such as “the system shall convert feet to meters”, are an example of very small requirements. User stories are a little larger but still relatively easy to estimate. Use cases, a staple of the Rational Unified Process (RUP), can become too large to estimate effectively although you can reorganize them into smaller and more manageable artifacts if you’re flexible. I describe a wide array of modeling techniques, including the requirements models mentioned in this paragraph, at AgileModeling.com/artifacts.

So how does this pertain to budgeting and scheduling? An agile approach to requirements enables you to take an agile approach to budgeting and scheduling because of the higher level of control it provides business stakeholders:

1. Stakeholders get working software on a regular basis, enabling them to determine if the team is going in the right direction.
2. Stakeholders get their requirements implemented in the priority order which they define, enabling them to prevent wastage.
3. Stakeholder can control how their money is being spent because they can always reduce/remove funding to the team.

In other words, if you don’t like what you’re getting for your IT investment you simply reduce funding to the team, and even potentially stop funding it. If you like what you’re getting you can increase funding to the team. In other words, stakeholders now have complete control over how they invest their money in IT, enabling them to get the best value for their investment. With this approach, your budget is always whatever stakeholders are willing to spend and the schedule is always however long it takes to build. When you think about it, isn’t that the true answer for any software development effort, regardless of the approach? The primary difference is that with the agile approach we maximize stakeholder investment.

With this level of control, do you really need to develop a fictional budget, or schedule, at the beginning of the initiative? Unfortunately, the answer is probably still yes in most organizations. Even though the financial people within your organization hold out little hope that the IT department will actually get the estimates and schedules right, they naively think that they’re reducing financial risk by doing so. This leads us to the “middle of the road” approach to budgeting and scheduling.

3. A “Middle of The Road” Approach to Estimating

With a middle of the road approach you do just enough requirements envisioning up front to identify the scope of the initiative, and just enough architectural modeling to determine a potential strategy to build the system. This modeling efforts provides just enough understanding for estimating an initial budget and schedule without paying the price of excessive documentation. The most effective strategy to put the initial budget and schedule together is to gather a small group of people who have relevant experience from similar efforts in the past, and more importantly have a stake in the new initiative (e.g. they’re on the team and accountable), and ask them to provide a good guess. Realistically at the beginning of the initiative this guess should have a +/- 30% range, some research suggests a 50% range, but unfortunately on average organizations ask for a +/- 11% initial estimate even though their actuals come in at +/- 19% after considerable “fudging of the numbers” in many cases.

Agile modelers understand that because requirements evolve over time that any early investment in detailed documentation will only be wasted. Instead they will do just enough initial modeling to identify their scope and develop a high-level schedule and budget: all you can really hope to achieve at this point is a good “ballpark” estimate, so you should only do enough work to get you to that point. Figure 3 overviews this Agile Model Driven Development (AMDD) approach.

Figure 3. The AMDD lifecycle.

During development they will model storm in a just-in-time (JIT) manner to explore each requirement in the necessary detail, and at that point produce an accurate estimate for that bit of work. The developers should still be responsible for estimating their work, as described above, and stakeholders responsible for prioritizing the requirements.

The good news is that is still reasonably agile: remember, agility is situational. At first, you only need a high-level, loosely detailed schedule; the details are identified throughout the effort JIT. Similarly, at first your estimate will involve a large range (if “greater accuracy” is needed, provide a figure closer to the top of the range and hope for the best) which will be narrowed down as the team progresses. The end result is that based on your actuals you will update your estimates throughout the initiative, narrowing the estimate range over time. These narrowing estimates are referred to as a “cone of uncertainty”, a phrase coined by Barry Boehm.

Better yet, forgo the questionable concepts of schedules and estimates completely and simply choose to spend your money wisely as implied by Figure 2.

4. Comparing the Approaches

The following table compares and contrasts the three approaches. Each approach has its advantages and disadvantages, you need to identify which one is most suitable for your situation and then tailor it to meet your exactly needs.