Responsible IT - Part 2

In the previous post, we discussed the biggest challenges, IT organizations face these days. We have seen four big challenges, each with its own peculiarities:

1. Post-industrial markets and the resulting decision uncertainty (which equally affects IT due to the consequences of digital transformation).
2. Digital transformation and its consequences, the fact that business and IT have become inseparable and especially the indispensability of IT.
3. Highly complex, highly diverse and outdated IT system landscapes, held together with lots of duct tape and kept running which WD-40 all over the place.
4. Demographic change and the resulting skills shortage (which is a big issue at least for the Western hemisphere).

In this post, we look at the typical response patterns, why they usually only reinforce the problems and then try to rethink the solution space from first principles.

Let us begin …

Typical response patterns

When looking at the typical response patterns, maybe a bit surprised we realize that the response patterns often are the same patterns that created the mess in the first place, e.g.:

• The typical response to post-industrial markets is adding features in a shotgun fashion (“If I fire a dozen features at the users, one of them will hopefully stick”), often without measuring the impact of the features and most definitely without any plan or budget to remove features that do not create value. This habit reinforces the complexity of the system landscape and the problems resulting from the skills shortage. Additionally, little thought is spent on making systems dependable (because all money and capacity is required for adding features), counteracting the indispensability of IT.
• Because adding more and more features in a shotgun fashion requires more and more development capacity but fewer and fewer developers are available due to the skills shortage, the aim is to increase development efficiency more and more and more. However, as I already described in a prior post and will detail a bit more when discussing effectiveness further down this post, it does not solve any problem if you become more efficient without being effective. In such a setting, much of your work does not create value but is either an idle or even a value-destroying performance. Due to the erroneous assumption that effectiveness is a given (stemming from outdated industrial market habits), most companies neglect effectiveness and solely focus on improving efficiency, i.e., they attempt to implement more features with less money without checking if the features create any value. As a consequence, each time efficiency is improved, the pile of idle and value-destroying features grows faster. In the end nothing is won. Quite the contrary, the system landscapes’ complexity grows disproportionally and the dependability of IT is actively impaired.
• However, the hope is still to find a cheap and easy silver bullet that cures all perceived efficiency shortcomings, be it, e.g., No Code/Low code or more recently (Gen)AI solutions of all kinds. While those solutions have their benefits, bluntly introduced with the sole expectation to increase developer efficiency in order to shoot out even more features, it only reinforces the problems.
• Simplifying the system landscape is usually equated with standard software. However, looking at the fate of the typical SAP or Siebel implementation, it becomes obvious that the problem is not custom software development. Rather, it is the unbridled demand of the business departments that the software solutions must support all idiosyncrasies and quirks of “their way of doing things” which they developed over the decades, including even the rarest and most unlikely business exceptions. This demand to let software solutions perfectly mimic one’s own idiosyncratic ways instead of focusing on the job that needs to be done ¹ leads to overly complex solutions – no matter if custom-built or standard software. If using standard software, all these quirks and exceptions will also be demanded by the business experts, leading to highly customized standard software solutions, not being “standard” anymore by any means, often working against their built-in concepts, resulting in a loss of upgradeability – which is one of the most important reasons to choose a standard solution in the first place. Additionally, lots of diverging paradigms need to be integrated into the IT system landscape, one more each time another standard solution is integrated, resulting in even more complexity at the system landscape’s level. In the end, adding standard software to the game rarely reduces any complexity. Instead, it rather compromises the ability to respond swiftly to the market uncertainty. It reinforces the skills shortage because now you need experts to maintain and change all the standard software solutions’ customizations. In the worst case it even gambles with the indispensability of IT. ²
• And so on …

Overall, it can be said that the typical response patterns do not solve any of the problems. Quite the contrary, they only reinforce the problems.

As nothing gets better, one would expect a change of the approach. However, company realities often remind of the famous quote, often incorrectly attributed to Albert Einstein:

“The definition of insanity is doing the same thing over and over again and expecting a different result.”

The typical “solution” is to increase the pressure further, to amplify the unsuccessful patterns, to press harder for the “proven” and accepted, for the things that obviously do not solve the problems.

To be fair: The usual response patterns become very understandable when looking at

• the different response patterns needed to successfully address predictable, industrial markets and unpredictable, post-industrial markets.
• the fact that change is hard for humans and even more for higher-level systems like companies.
• the need-vs-want dilemma, i.e., that the more short-term oriented wants tend to trump the longer-term oriented and more sustainable ³ needs. Again, this applies to an individual level as well as to a company level. Most of the usual response patterns feed the company’s wants, not its needs.

However, even if these behavioral patterns are very understandable, they do not solve any problem in a sustainable way.

Thinking from first principles

The obvious question is: How can we do better?

To answer this question, let us take a few steps back and try to look at the challenges and possible solution approaches from first principles. Here are the challenges again:

1. Post-industrial markets and the resulting decision uncertainty (which equally affects IT due to the consequences of digital transformation).
2. Digital transformation and its consequences, the fact that business and IT have become inseparable and especially the indispensability of IT.
3. Highly complex, highly diverse and outdated IT system landscapes, held together with lots of duct tape and kept running which WD-40 all over the place.
4. Demographic change and the resulting skills shortage (which is at least for the Western hemisphere a big issue).

Resilience

Let us start with the post-industrial markets and the resulting decision uncertainty: We cannot say upfront anymore what will be successful. This is not easy to handle, but we know methods to deal with such a decision uncertainty. However, what makes things much harder is the fact that we can hit by a surprise, an adverse event any time. We do not know when they will happen and where they will come from.

It does not always need to be a full-blown crisis like, e.g., the supply chain crisis that messed with availability and prices of goods in unexpected ways. It is enough if a competitor launches a new product that becomes highly popular and you did not see it coming. In short:

The planability of the past is gone. Surprises and lack of predictability are the new normal.

This means, we need to find ways to deal successfully with surprises, with unexpected adverse events and situations.

These ways are called resilience.

Resilience is the ability to successfully deal with expected and especially also with unexpected adverse events and situations.

So, we have our first response to the IT challenges of today. Resilience is a big topic and we need to break it down in different response patterns, in different targets, and more. I already discussed some aspects of resilience in this introductory post, this post series about resilient software design and I will discuss it in much more detail in some future posts. ⁴

Still, even if we do not discuss resilience in greater detail here, it should be obvious that thinking in these lines will lead us to very different actions than blindly shooting more and more features at the customers in a shotgun fashion and pressing to increase efficiency by all means.

Sustainability

Let us move on to digital transformation and its consequences, that business and IT have become inseparable and IT has become indispensable. If something has become indispensable, we need to make sure it is available and works in the long run: In a year, in 5 years, in 10 years and beyond. We need to take care of its long-term well-being.

This is called sustainability.

These days, many people equate sustainability with ecological sustainability only. However, this is just one dimension of sustainability. Traditionally, sustainability has three pillars and in IT, we need to add a fourth one:

1. Economical sustainability – making sure we are financially healthy in the long run.
2. Ecological sustainability – making sure our acting does not harm our environment, not now and not in the future.
3. Social sustainability – making sure our acting is tenable for all people involved and affected, preserving a “sustainable pace”.
4. Technical sustainability – making sure our IT systems are dependable now and in the future.

The first three pillars are the traditional pillars of sustainability and are required whenever you want to play an endless game – what should be the goal of all companies, even if we sometimes may get a different impression. As business and IT have become inseparable, IT must also play an endless game if business plays an endless game. The fourth pillar is IT-specific and related to the indispensability of IT.

Sustainability leads to very different directions than the typical response patterns which try to find a quick and short-sighted band aid to mitigate symptoms instead of trying to fix the cause. E.g., sustainability points towards less complexity and less stress, looking for more effective solutions instead of solely focusing on implementation efficiency. I will also dive deeper into this topic in future blog posts.

Simplification

When it comes to the highly complex, highly diverse and outdated IT system landscapes, the required response seems to be too simplistic at first sight: simplification.

We need to simplify our system landscapes. However, simple solutions tend to be the opposite to easy. I made the same observation over and over again:

Every moron can create an overly complex solutions for a given problem.

But it takes a lot of effort and brainpower to come up with a simple solution.

While this observation may feel a bit pithy (and admittedly, it is to a certain degree), it reflects what I have seen time and again over the years: Hastily designed, poorly pondered, overly complex solutions, often driven by what can be done instead of what needs to be done. Additionally, many other unhealthy company habits at all levels tend to reinforce the resulting system landscape complexity.

I wrote a whole blog series consisting of 15 (!) parts about the problem and discussed some ideas how to do things better. Without giving away too much: The ideas that point towards less complexity are very different from the typical short-sighted response patterns we tend to see.

Effectiveness

This leaves us with the fourth problem, the demographic change and the resulting skills shortage. When it comes to getting done more with less people, the usual response pattern is to increase efficiency which is a well-trained pattern from our industrial production past.

This is exactly what we see if IT decision makers clutch at straws like No code, GenAI and alike. They hope to further increase the efficiency of their available workforce.

However, our problem is not a lack of efficiency but a lack of effectiveness. As I discussed in my blog post “Forget efficiency”, the rate of value-increasing features in an average company lies between 10%-20%.

In other words: 80%-90% of the features implemented and released are waste at best and value-destroying at worst.

If you increase your efficiency, e.g., by 10%, you increase your business value by 1%-2%. Unfortunately, at the same time you increase your idle and value-reducing production by 8%-9%, usually overcompensating your wins.

Now imagine, we could improve our effectiveness by 10%. This would mean, our rate of value-increasing features would increase from an average of 10%-20% to 20%-30%. This would mean a real productiveness increase of 50%-100% in terms of business value – which is our actual measure of success, not the amount of work accomplished. On top, we would also reduce the amount of value-destroying work which would further increase our overall productiveness.

Additionally imagine, at the same time we would establish measures to detect idle and value-reducing features early and stop their implementation at, e.g., 20% of their overall implementation effort in average. We would save 80% of the money, resources and capacity, we usually spend on idle and value-reducing features. All this money would be available for actually useful stuff.

Overall, we would need to spend a lot less money and skills shortage would be a problem of the past.

It is known how to establish such a system (see, e.g., the book “Trustworthy online controlled experiments” by Ron Kohavi et al.). Unfortunately, most companies only attempt to increase their efficiency, completely ignoring their effectiveness.

Responsible IT

This was a very short derivation of sustainable answers to the problems, we discussed in the previous post. Let us summarize the responses:

1. Resilience as response to the inevitable surprises due to post-industrial markets and other influences, often summarized as VUCA.
2. Sustainability as response to the effects of the ongoing digital transformation, the inseparability of business and IT and the indispensability of IT.
3. Simplification as response to overly complex, highly diverse and outdated IT system landscapes.
4. Effectiveness as response to the demographic change and the resulting skills shortage.

When pondering these four responses for a moment, it becomes clear that the first three responses create a self-balancing and self-reinforcing system. Resilience, sustainability and simplification as guiding forces ensure that none of the three forces leave their respective sweet spot.

E.g., if you would try to optimize for simplification only, the best possible response would be to eliminate all IT. However, economic sustainability as well as resilience balance this force. Even regarding ecological sustainability, it is questionable if the alternative efforts needed to achieve the same business outcomes would result in a better ecological footprint.

Or if you would try to maximize the runtime robustness of your applications as part of resilience as much as possible, this would result in very complex solutions that may become hard to change. However, the other levels of resilience as well as simplification balance this force. Additionally, it would compromise social and probably also economical sustainability.

On the other hand, while staying in their sweet spots, the three forces reinforce each other. E.g., simplicity fosters robustness. Simpler solutions are also easier to change and reduce the mental load on the software engineers affected, improving human/social sustainability. And so on.

Effectiveness acts as a North star. It creates the overarching direction and boundaries for all our activities. Are we really working towards more effectiveness? Do we attempt to improve outcome and impact instead of output and work? Do we measure the effects of our activities at the market side rather than internally? Or did we accidentally fall back into the habit of solely focusing on efficiency while ignoring the effectiveness of our actions?

I call this way of responding to the most pressing challenges of IT using first principle thinking Responsible IT.

Yes, here we finally got to the title of these two posts!

I chose this name because I think it is a much more responsible way to respond to our challenges in IT than the usual, often very short-sighted response patterns.

I also chose it because I was not able to come up with a cooler and shinier name. After all, I am not Gartner or McKinsey. They would have come up with a much shinier and much more important-sounding name, of course … ;)

Summing up

These days, most IT organizations face the following pressing challenges:

1. Post-industrial markets and the resulting decision uncertainty (which equally affects IT due to the consequences of digital transformation).
2. Digital transformation and its consequences, the fact that business and IT have become inseparable and especially the indispensability of IT.
3. Highly complex, highly diverse and outdated IT system landscapes, held together with lots of duct tape and kept running which WD-40 all over the place.
4. Demographic change and the resulting skills shortage (which is at least for the Western hemisphere a big issue).

While most IT decision makers immediately name the 3rd and 4th challenge, they usually miss the first two challenges because their effects are a bit harder to spot – however not less impactful – than the effects of the last two challenges.

The typical response patterns are not suitable to solve any of these challenges. On the contrary, they tend to reinforce the problems.

Therefore, I tried to take a few steps back and ponder from first principles how to respond to the challenges in a more sensible, more sustainable way and came up with the following responses:

1. Resilience as response to the inevitable surprises due to post-industrial markets and other influences, often summarized as VUCA.
2. Sustainability as response to the effects of the ongoing digital transformation, the inseparability of business and IT and the indispensability of IT.
3. Simplification as response to overly complex, highly diverse and outdated IT system landscapes.
4. Effectiveness as response to the demographic change and the resulting skills shortage.

Following these ideas leads to very different responses and actions to tackle the aforementioned challenges than the usual response patterns we tend to see these days. Put together, they form a self-balancing system with a clear North star which I call Responsible IT.

As mentioned before, I already wrote about most of the responses in more detail and I will continue to write about them. These two blog posts are meant to explain the overarching ideas that stand behind many of my blog posts. I will continue to explore the details of these ideas, and maybe I sparked some curiosity in you to do the same – which would be great because from all I see, we desperately need a more Responsible IT.