The essence of architectural work - Part 4

In the previous post, we discussed the economic purpose of architecture. The economic purpose is probably the most emphasized facet regarding the Why of architectural work, yet hardest to grasp in its entirety and the hardest to get right. However, this does not mean that the other facets would be of less importance. But as they do not immediately emphasize time and money, they are often pushed aside in our highly commercial environments. Nevertheless, depending on the situation, getting these facets right can have an even bigger impact on time and especially money than the economic facet. Hence, let us discuss these other two facets of the purpose of architectural work in this post:

• The cognitive purpose
• The humane purpose

The cognitive purpose of architectural work

At a very basic level, software development is a mapping from a problem domain to a solution domain.

The problem domain consists of all the needs and demands that define and constrain the solution. Many stakeholder groups frame a vast amount of immediate and contextual requirements. This comprises not only the functional requirements but all explicit and implicit requirements that shape the solution. Additionally, the problem domain consists of myriads of facts as well as interconnections and dependencies between them that define the problem domain and in which all the aforementioned requirements are embedded.

Ideally, we would take all these needs and demands and directly translate them into a solution implementation, taking all the encompassing facts and their interrelations into account.

Drowning in problem domain details

The problem is the sheer mass of details on either side of the mapping. We are confronted with thousands, if not millions, of facts, interrelations, requirements, explicit and implicit wishes, demands, whims, and more on the problem domain side. What makes things even more complicated:

• Many of the demands contradict each other. Sometimes the contradictions stem from different stakeholders. Sometimes, it is even the same person not realizing (or caring) that they created contradicting demands.
• Many of the demands do not fit into the existing solution design (if a solution already exists). Most people only care about their current demands. They do not care about how their demands fit into an existing solution. They only care that their demands get implemented.
• Many of the demands are never spoken out. They are just silent expectations that will be held against you later (“It should be obvious that …”) if you fail to elicit them from the respective stakeholders. Good architects develop a sixth sense for these kinds of demands over time.
• And so on.

It is an art in itself to make all relevant needs, wishes, and demands explicit, resolve the contradiction and organize them in a comprehensible way. Personally, I doubt that AI agents will become good at that because all this has to do with the “messiness” of humans, their egos, and more ¹. But this is a different story, I do not intend to discuss here.

Let us simply assume we would be able to gather all the required facts and interrelations, elicit all needs, wishes, and demands from the stakeholders involved, resolve all contradictions, and organize everything in a comprehensible way. These would be many more details than any human can grasp and consider simultaneously while designing and implementing a solution.

Software engineers drown in an abundance of problem domain facts and details.

Drowning in solution domain details

It does not look any better on the solution domain side:

• We need to understand the details and intricacies of the programming language used.
• We need to understand the possibilities and intricacies of the language’s ecosystem.
• We need to understand the frameworks and libraries used.
• We need to understand the runtime infrastructure, middleware, and environment, its products, tools, and applications.
• We need to understand the development environment, its options, challenges, and quirks.
• We need to understand the existing solution: what is put where, how the former facts and requirements were translated into code.
• We need to understand the APIs we need to connect to, their commands and data structures, and how to access them.
• And so on.

It is not only that we need to know where all the already implemented facts and requirements are located in the solution. We also need to understand everything else that shapes the solution domain. These are many more details than any human can keep in their head at once. In short:

Software engineers drown in an abundance of solution domain facts and details.

A losing game

Now, let us assume we would try to map all the requirements directly onto the solution with nothing between them. We would need to map a myriad of facts and details on the problem side onto a myriad of facts and details on the solution side. This would result in a mapping so huge, nobody would be able to even cover a tiny fraction of it, let alone the whole mapping.

Whenever we would create a part of the solution, we would forget the majority of needs and demands affecting that part of the solution. We would need to rewrite all parts of our solution over and over again because we would always forget to take relevant needs and demands or other relevant solution details into account. The entire software development process would result in a mess.

Hence, we need a smaller mapping. We need some kind of compression that fits into a human brain and allows us to quickly identify the currently relevant solution parts and to focus on their relevant details. We need a mapping that requires only reduced mental bandwidth, a mapping of a size that does not result in cognitive overload.

Architecture as compression and guidance

That is exactly what architecture provides (or should provide): Architecture basically provides a sort of compressed construction plan we put between all the facts, needs, and demands of the problem domain and the implementation details of the solution domain. It condenses and structures all the facts, needs, and demands (also those pointing into the future) we find in the problem domain, in a way that it provides guidance and orientation when implementing the solution. It reduces the mental bandwidth required to map between problem domain facts and details and solution domain facts and details. Thus, it reduces the likelihood of missing essential demands while implementing the solution.

This also means that an architecture needs to be simple enough to understand it completely, because otherwise it is of little use. If the architecture results in cognitive overload, it does not fulfill its cognitive purpose (a fact still too often neglected by software architects).

Also note that condense and structure includes removing ambiguity from the requirements and prioritizing contradicting demands to enable the implementation of a solution. Ideally, this should not be part of architectural work but be done in business analysis, requirements engineering, or product management. However, based on my experience, more often than not a flood of ambiguous and contradicting demands hits the architects. Thus, be prepared to do the work and do not rely on others doing it.

Unfortunately, most of the discussions regarding software architecture only focus on the provide guidance and orientation part, including lots of suggestions on how to measure compliance with the architecture and the like. Architects tend to think more often in terms of how to control developers than how to support them in doing their work. The best way to ensure that developers adhere to an architecture is to give them something they understand and that makes their lives easier. An architecture only achieves this if we do both parts of the work:

• Condense and structure the facts, needs, and demands of the problem domain into an architecture
• Adding guidance and orientation that supports developers in doing their work

We may still add some fitness functions that are measured automatically during CI/CD. But they should aim at providing guidance and orientation, not at controlling developers.

There are many ways to describe such an architecture. Diagrams of the intended system structure and behavior may be part of it (the latter forgotten most of the time). But an architectural description is not limited to diagrams. Guiding principles and constraints are even more important. Structures and flows often evolve and change quickly. But styles tend to be much more durable. Interface descriptions and code skeletons/sketches can also be very useful. Some metrics that help to figure out if a demand is met. And so on.

The core point is that an architecture should provide guidance and reduces cognitive load.

The humane purpose of architectural work

With that, let us move on to the final facet of the Why of architectural work: the humane purpose. This facet is usually neglected, but based on my perception, it can be the most powerful lever of architecture. Thus, let us have a closer look.

Personally, I made the experience that we can often learn a lot from looking into other domains and understanding what they are doing, why, and how. This was one of the reasons why I watched a little anthology called “Abstract” on Netflix. The anthology portrayed designers from many domains. One of the episodes portrayed Ilse Crawford, a famous interior designer. At one place, she said:

“Ultimately, design is a tool to enhance our humanity” – Ilse Crawford

This sentence resonated deeply with me. If we adopt this stance, architecture, being a vital part of software design, ultimately should improve the lives of the people affected by the solutions we create. If you let this sink in for a moment, you will realize how beautiful and powerful this approach is – a very humane one and a lot less technocratic than most other approaches (including my first one). It also encompasses the cognitive purpose we discussed before because it improved the lives of the developers.

You may be tempted to dismiss this perspective as mere idealistic thinking in a cynical world. But before doing so, it is important to understand its vast economic power. Software solutions affect people: the people contributing to them, the people building them, the people running them, the people using them, and other people who have stakes in the solutions.

• If we improve the lives of the people influencing, building, and running the solutions, we reduce friction and increase the probability of sustainable success. We reduce effort by reducing the amount of discussion and stress. People are happier in general, which is a massively underrated driver of productivity.
• If we improve the lives of our users, we increase impact. If our users love our solutions, if they feel good, smooth, and frictionless, we will get more users using our solutions more often. I.e., we do not only influence the cost side with our architecture, which has a limited leverage. We influence the revenue side, which has a much bigger lever than the cost side.

Therefore, we should not easily dismiss this humane facet of the purpose of architectural work. Quite the opposite, we should emphasize it a lot more.

However, in this short-sighted, capitalistic world most of us are living in, the predominant cynical stance is that quick money is more important than the well-being of humans. Personally, I love the irony that the cynics deprive themselves of the possibility of making a lot more money because of their short-sighted worldview.

Be aware that following this humane approach ultimately will lead to different priorities regarding the required activities, when and how to carry them out. It may also create a powerful counterbalance to the purely financial perspective of the economic approach discussed in the previous post.

As this facet of architectural work is so massively neglected, there is still a lot to explore. Based on what I have seen, it is more than worth it.

AI and the purpose of architectural work

These days, it is almost impossible to discuss any aspect of software engineering without mentioning AI. And there are a lot of people around who claim that architectural work can be taken over by AI agents. Thus, let us briefly examine this claim in the light of the purpose of architectural work (we will discuss the topic in more depth at the end of this blog series).

Personally, I think that those people never pondered the purpose of architectural work. If you let the three facets we have discussed sink in, it should be clear that the existing AI solutions are far from being able to implement them even remotely. If you feel obliged to contradict, try to make the AI agent of your choice design an architecture that:

• Minimizes the cumulative costs of a system over its lifetime without compromising the correctness of behavior at runtime
• Condenses and structures the problem domain and provides guidance and orientation in the solution domain
• Improves the lives of the people affected

I think you will be out of luck quickly.

This does not mean that every human architect creates architectures that satisfy these criteria. To be frank, most architectures are probably far from it. Still, if we try to satisfy these traits of a good architecture that fulfill the purpose of architectural work, we need humans for it. Humans are not a guarantee for getting a good architecture, but machines are simply not able to navigate all these too human needs, demands, and resulting depths and pitfalls.

On the other hand, this does not mean that AI agents are useless when it comes to architectural work. Quite the opposite. They can support architectural work in many ways. However, they are not able to implement the desired traits of a good architecture on their own.

Always remember:

Ultimately, we create software for humans.

If we forget this, we are in deep trouble – for many reasons.

Reinventing the wheel

When reading the last section, you may want to contradict: But in my project, AI took over architectural work. Of course, you need the expensive frontier models for that. But they are doing it successfully!

The question is: What are those projects doing where AI takes over “architectural work” successfully?

Most of the time, they are reinventing the wheel – a problem I already discussed a while ago in my blog post “Solving the wrong problem. If we look around, we see companies building the millionth instance of an e-commerce, ERP, SCM, CRM, or the like solution. We see them solving the same problems time and again. For the wrong reasons, we failed to create good software components that encapsulate this logic, as we did with technical components like HTTP servers or message brokers. Usually, the fallacy was (and still is) that companies insist on their “uniqueness”, most of the time only nurturing their habits and whims (and thus also disdaining standard software solutions like, e.g., SAP or Salesforce).

I will not delve deeper into the reasoning why we solve the same problems time and again. This might be a discussion for a different post. The point is: We did, we do, and most likely, we will continue doing so. We solve the same kinds of problems time and again. We solved them so often meanwhile that they are deeply ingrained into the training corpus of any LLM.

Still, this is what we do most of the time: solving solved problems. Building the same solution time and again, just with a few insignificant bells and whistles added. If this is what you are doing all the time – well, then of course your AI agent can define your architecture. But then you do not need architectural work. You do not need an architect. You rather need a librarian who pulls the right book from the bookshelf, a book that was written a long time ago, opens it, and starts reading it. This is a job, LLMs are better at than any human.

But again, this is not architectural work. This is just implementing an existing, worn-out blueprint yet another time – a blueprint someone created a long, long time ago.

Interlude

We concluded the discussion of the Why, the purpose of architectural work, by looking at the cognitive and the humane facets of architectural work. It became clear that AI agents are not able to satisfy these traits of architectural work (unless you are solving a solved problem). Still, they can support architectural work in many ways.

In the next post (link will follow), we will move on to the next dimension of architectural work, the What. Stay tuned …