Project to Product: Why Software Development Isn’t a Project
Since the publication of Mik Kersten’s book, Project to Product, in 2018 we’ve seen this very shift in the software and systems development space. Software initiatives, in particularly the creation, support, and evolution of software-based solutions (think apps, IT infrastructure, data warehouses, and so on) are being organized as long-term product efforts rather than finite projects (or a series of finite projects). This article examines why this is happening in practice.
Why Projects Are Ill-Suited for Software Development
The following diagram depicts an “idealized project lifecycle.” It has several critical features:
- The stages aren’t explicitly named. I did this on purpose because the exact name of the stages (for example Discovery, Design, …) depends on both the context of your project and the methodology you’re currently following. The point is that software projects tend to work in stages/phases, even agile project teams (regardless of the semantic hoops they’ll jump through to avoid admitting this), and these stages occur in a serial manner one after the other.
- An explicit start and end. Projects have an explicit start and end date, although the end date may slip when the priority is on meeting customer requirements rather than hitting an often fictional “required” date. Having said that, it is common for software development project teams to sacrifice something, typically either functionality or quality (or both), so as to appear to be on time. I say appear because if you need to have a second project (or more) to deliver what you originally promised, or to clean up the mess that you made, then you clearly didn’t deliver on time or on budget regardless of whatever story your project management office (PMO) spins (and nobody really believes the story anyway).
- Feedback cycles exist. As you can see, feedback cycles do exist in the project lifecycle, regardless of the claims of agile purists. But these feedback cycles tend to be weak and onerous, particularly when a traditional governance strategy is in place that is based on milestone sign-offs. Worse yet, when there is a strong desire to deliver on time, all but the most critical feedback is pushed off to a future clean-up project (which may never occur).
- Limited customer involvement. Customer involvement, if you can get it, tends to be early on in a project. Once you get past your Discovery/ Inception/ Requirements stage customer involvement tends to be reduced to receiving project status reports, which rarely contain any interesting information and have likely green shifted or even quantum shifted, and attending milestone reviews which they may not understand. The challenge with inviting customer/ stakeholder feedback is that they often give it to you, and to act on it (and thereby increase the chance that you deliver something of value for them) you will very likely make your project over budget and late (which the team, and particularly the project manager, will likely be blamed for). In short, the project team is typically set up to fail on this critical success factor.
- The team (typically) doesn’t live with the consequences. Yes, your bonus is likely be tied to the success of the project, so there’s that. However, it is often the case that most, if not all, of the members of a software development PROJECT team move on to something else after the project is completed. A small handful of people may remain with the resulting solution, at least for a “warranty period” after delivery, and sometimes joining the maintenance team responsible for operating it. This is required for a “successful handoff” that typically results in the solution being added as a responsibility to an already overburdened maintenance team, thereby reducing any hopes of a timely follow-up project to clean up the mess or add any dropped scope (there isn’t any budget allocated, maybe next year, yadda yadda yadda).
There are several significant reasons why a project lifecycle is ill-suited for software initiatives:
- Software is long-lived. The actual software lifecycle, rather than the software development project lifecycle, runs for many years and there may not even be an end date for it. For example, consider Microsoft Word which has been evolving longer than many people alive today. The concept of a “project” within that lifecycle injects needless risk, cost, and time in practice.
- Software requirements evolve. The reality is that software requirements evolve, for many good reasons as well as a few bad ones. If your project approach is based on the idea that you’ll be able to “lock down” or baseline the requirements then at best you’re fooling yourself and at worst you’re unethical (if you’re delivering something to specification that’s not what they really need, in effect you’re forcing them to pay for something they don’t want and that’s not ethical). Effective software development approaches accept that requirements evolve and therefore your budget and delivery date may similarly need to be flexible, a concept that is opposite the all-too-common “on time and on budget” philosophy of project management.
- A project-based strategy increases technical debt. For your project to remain reasonably on time and on budget something typically needs to give, and that is usually a combination of scope (functionality is dropped) and quality (time for testing and fixing is often cut short). As I mentioned earlier, the naive promise that “we’ll finish up in a future project” is typically held out to placate stakeholders but unless it’s budgeted for and scheduled you’d be a fool to believe that it’s actually going to happen.
Why a Product-Based Approach is Well-Suited for Software Development
The following diagram presents an idealized product lifecycle. It has several critical features:
- The activities aren’t explicitly named. As with the project lifecycle, I’ve chosen not to name the activities as they will reflect your context and the methodology that you follow. One example of a product lifecycle, or more accurately a value stream lifecycle, can be found in the value stream layer of PMI’s Disciplined Agile (DA) toolkit.
- The lifecycle is continuous. Product lifecycles are explicitly continuous, starting and hopefully continuing with your customers. And to be clear, although you could try to attempt this as a series of projects the net effect of doing so would be to inject time, cost, and risk into your overall effort.
- Customers are (typically) more involved. Product lifecycles tend to have a greater focus on the customer, and more importantly with a continuing relationship with customers. The feedback cycles tend to be shorter, resulting from greater motivation to be competitive via delivering value as responsively as possible.
- The product team has to live with the consequences. This is a critical aspect of the product lifecycle – because it is continuous, the work that the team is doing today will need to be operated, supported, and evolved tomorrow by the team itself. This happens because the team stays together, there’s no “project end” where the team disbands. The implication is that poor quality work will soon come back to haunt them, unlike with a project team that is likely to have been dispersed by the time long-term issues become apparent.
There are several reasons why a product lifecycle is well-suited for software development initiatives:
- Product-based approaches reflect the long-life of software assets. Software typically exists for years, and often decades. It needs to continuously evolve to reflect the changing needs of your stakeholders, as I show next. The continuous approach of the product lifecycle reflects that reality.
- Product-based approaches explicitly assumes the solution will evolve. The needs of your customers evolve over time, and to remain competitive you need to evolve your offerings to them to follow suit. This is true both of internal software as well as external software that is used by your organization’s customers. With a product-based approach the team stays together and focuses on addressing their needs of their customers.
- Product-based approaches reduces, and often avoids, technical debt. When a team knows that they will need to live with any poor quality that they inject into their work, and when they know that they will be the ones that suffer the pain of cleaning up their mess, they are far less likely to tolerate the injection of poor quality to begin with.
From Project to Product
A project-based approach has proven to be ill-suited in practice for software development. Software is a long-term asset for your organization, one that will need to evolve over time as the needs of its stakeholders evolve. The short-term, transactional thinking of project-based approaches aren’t appropriate for the development of dynamic and malleable assets such as software. The long-term, customer-oriented strategy of product-based approaches prove to be a better fit for software in practice.
I really can’t say enough good things about Mik Kersten’s book, Project to Product. If you’re in the software field you really want to read this.