Projects considered harmful - Part 2

In the previous post, we discussed how the broken feedback loops that software development projects create lead to a continuously deteriorating IT system landscape, resulting in an ever-shrinking dependability – which is probably the by far most important runtime property of software.

In this post, we will discuss the other three reasons why I consider projects harmful:

• Due to the project blinders effect, local optimizations are enforced, leading to a complexity explosion of the related IT system landscape.
• Software engineers, doing the actual work are tossed around from project to project which is problematic in terms of individual motivation and performance as well as in terms of team dynamics – both leading to suboptimal results, compromising the quality of the related IT system landscape.
• If everything needs to be a project, if nothing can be accomplished without setting up a project, the regular organizational structure is completely inadequate. Instead of setting up more projects, the regular organizational structure needs to be fixed.

Finally, we will summarize our findings. Let us get started …

Local optimizations

While the effects described in the previous post are bad enough, software development projects cause more harm due to their project blinkers: The sole purpose of a project is to achieve its goal, in the case of software development projects to deliver some software in time and budget.

That is it. Nothing else is relevant for a project. Projects do not care about anything outside their scope and they do not care about anything that happens after the project end date. Projects have a laser-sharp focus: Their goals and objectives. Nothing else. Caring about anything that lies outside the scope of a project contradicts the purpose of a project. In short:

Projects only optimize for their concerns and blank out everything else.

While this property of projects is needed to get ambitious ventures successfully completed, it tends to backfire when it comes to software development projects.

Due to its sole focus on its own targets, each software development project pushes for local optimizations, blanking out the effects of such local optimizations for the rest of the IT system landscape as well as the effects for future changes of the software concerned. This leads to effects like, e.g.:

• A less effortful implementation of the API saved a few implementation days, doubling the efforts needed to use the API for every other system.
• A quick hack, (re)using a database field for a second purpose saved a few implementation days, causing lots of bugs and downtimes after going live and doubling the times needed to fix the bugs or changing anything related to that database field because it is extremely cumbersome to comprehend the implementation.
• The introduction of a new technology, deviating from the widespread company standards, saved a few implementation days. However, every software developer working for that part of the IT system landscape from now on needs to understand an additional technology, leading to significantly increased software development costs and reduced dependability due to increased mental load, resulting in more errors and reduced productivity,.
• And so on …

Note that all these local optimizations make the IT landscape more complex because they are hard to understand, deviate from agreed upon standards, increase mental load, increase efforts to integrate systems, increase efforts to maintain the IT system landscape, and more. And of course, all those local optimizations tend to be poorly documented (if at all) because documentation for people outside the project scope – including future maintainers of the software – is not a project goal.

Also note that these effects do not add up. The multiply up!

It is quite easy to keep track of a single shortcut, a single deviation from the norm. Three are still okay-ish. Five are marginally manageable. Beyond seven, things start to spin out of control. Additionally, all these shortcuts, hacks and deviations tend to influence each other in unpredictable way, further increasing the accidental complexity.

Over time, all these local optimizations increase the complexity of the IT system landscape massively, accelerating its deterioration and reducing its dependability. Due to the exploding complexity, future changes become harder and harder, increasing the pressure for quick hacks and workarounds, accelerating this downward spiral.

Looking at the state of the average IT system landscapes of companies that accomplish software development using projects – well, it shows.

Detrimental effects on people

A lot has been written about strengthening and adverse effects regarding the productivity of individuals and teams. I will not repeat all this because other people have discussed it much better and in much more detail than I could ever do it.

Just be aware that the typical software development project setup has rather adverse effects on the motivation and productivity of people and teams. Being tossed around between projects as a “resource” like a pen or a memory upgrade makes you feel like being company livestock which is a huge motivation killer. It costs a lot of energy to compensate this continuously paraded disregard of individual value and it is hard to motivate oneself if one is treated like an exchangeable piece of meat.

I know it is not always that bad. However, this “people are resources” thinking which is typical for software development projects always includes a sizable disrespect for the value of individuals, continuously scraping at your motivation (and in bad cases even at your self-esteem).

If you need a significant amount of energy to motivate yourself against the adverse conditions of your work environment, you lack this part of your energy when it comes to your productivity and drive. And the longer you live in such a context, the harder the self-motivation tends to become.

Additionally, we know that teams require some time until they become productive: Forming – storming – norming – performing. Most of us know it. Yet, project-driven environments tend to completely ignore this, assuming that project teams can be put together and remixed all the time without any productivity impact as if the project members were soulless parts of a machine.

Finally, there is a strong relationship between team productivity and psychological safety. However, project-driven organizations do not provide psychological safety. Remember, you are a replaceable resource expected to function perfectly on command in any arbitrary project context, neither getting the time nor the room to build any psychological safety.

Overall, it can be said that all these adverse effects of project-driven software development environments result in a significant loss of productivity as well as solution quality: How great a solution can you expect from human beings you treat all the time like a replaceable cogs in a machine? It should be obvious that high performance cannot be achieved this way. At best mediocre performance is possible in terms of output and quality.

Project-driven software development persistently ignores these well-known and often discussed facts, the resulting IT system landscapes bluntly revealing the consequences.

Declaration of organizational bankruptcy

The last point is a bit different. It is not so much about the negative effects of projects on the IT system landscape. It rather is about the organizational insanity of organizing all software development as projects in the first place.

My point is that based on the definition of a project, a project normally is only set up if the regular organizational structure is not able to accomplish the task. Otherwise, a sane organization would never invest all the extra effort and money required to set up and run this additional temporary organization structure on top of its regular organization structure. It would simply use the existing organization structure to accomplish the task because it is a lot more efficient and less cost-intensive.

This is the purpose of projects: To implement the uncommon tasks that would overstrain the regular organization while the regular organization takes care of all common tasks. In other words: The regular organization does most of the work and projects are used only for the rare tasks the regular organizational structure cannot accomplish.

This is how projects are normally used – except in software development.

In software development, project-driven organizations are not able to accomplish anything with their regular organizational structure. They basically need to set up a project for every line of code they want to write or change ¹. In other words: Their organizational structure obviously is completely inadequate for the task it was established for.

This means, companies with such a project-driven software development maintain and pay for a regular software development organization structure that is not able to accomplish anything because for some reason of the past these companies decided that each and every software development task needs to be wrapped in a project. They decided to go with (and pay for) an utterly useless organizational structure.

In such a situation, I tend to ask why they afford an obviously useless regular organizational structure. My recommendation then is to take a few steps back and rethink the organization and processes. If you need projects for everything and anything, this is a sign that something is utterly wrong with your organization and you should fix it unless you are a company that does nothing but extraordinary tasks (e.g., building power plants).

However, software development is not an extraordinary task. We do it for more than 70 years meanwhile. It is quite well understood (even if it often is poorly implemented). There is nothing really challenging about software development (even if some people claim it would be). If software development is a problem, it is because the specifics of software are ignored.

To make software development a better controllable task, we do not need to wrap it into projects, multiplying the problems. We only need to take the specific properties of software into account – which inevitably leads to an organization structure that does not need projects to get things done.

Wrapping all software development in projects is a clear sign of not having understood software, leading to quickly deteriorating software landscapes – and most of the times it is a declaration of organizational bankruptcy.

Summary

I started with the claim that software development projects are harmful.

To prove the claim, we discussed the most relevant reasons I see:

• Due to broken feedback loops, wrong incentives are strengthened. As a consequence, the viability of the related IT system landscape is compromised.
• Due to the project blinders effect, local optimizations are enforced, leading to a complexity explosion of the related IT system landscape.
• Software engineers, doing the actual work are tossed around from project to project which is problematic in terms of individual motivation and performance as well as in terms of team dynamics – both leading to suboptimal results, compromising the quality of the related IT system landscape.
• If everything needs to be a project, if nothing can be accomplished without setting up a project, the regular organizational structure is completely inadequate. Instead of setting up more projects, the regular organizational structure needs to be fixed.

There are more problems, software development projects tend to cause. I discussed some of them as part of my “Simplify!” blog series.

But also without diving into the additional problems projects cause, like a counterproductive responsibility split, long lead times, and more, I hope it became clear that software development projects create more problems than they solve.

But what is the alternative?

Well, a lot of companies have understood that projects in software development are problematic. They moved from project-centric to product-centric. They organize around (business level) products and build cross-functional teams, consisting of all people needed to successfully implement, run and further evolve the respective product. Those teams consist of business experts, software development experts, software runtime experts, potentially people from finance, legal and whoever is required to drive the evolution of the product.

The teams tend to be long-lasting, creating the required motivation and productivity at the individual as well as at the team level. The feedback loops are set correctly. If those teams cut efforts for the wrong reasons, it is them who feel the pain resulting from it. Approaches like platform engineering that help to contain the exploding IT landscape complexity while reducing the mental load at the same time work well in such settings because – again – the feedback loops are set correctly. And so on.

In short: Alternatives exist and are proven to be superior to a project-centric software development approach in basically every regard (see, e.g., Team Topologies or the unFIX model for recommendations how to set up such an alternative organization).

But why do we still see so many software development projects?

Well, I guess change is hard …