Open Source Software – Part 1

When I started to discuss accidental complexity on the architectural level in my “Simplify!” blog series, I quickly realized that I needed to discuss Open Source Software (OSS) first as it often acts as a driver of accidental complexity on the architectural level today.

So, I started to discuss OSS inside the post. While doing this, I quickly realized that OSS deserves a more detailed discussion than just a few paragraphs in a post about a different topic. Especially, as so often it is important to understand the past evolution of a topic – here OSS – to be able to assess today’s situation correctly.

Therefore, I started to write this post. While writing, I realized that discussing OSS would become too much for a single post ¹. Thus, I split it up in 3 parts:

• The rise of OSS and its benefits (this post)
• Misconceptions regarding OSS (next post)
• OSS in the world of today (next but one post)

This post will be about the past of OSS, its rise and the effects it had on software development.

The time before OSS

Before we dive into the misconceptions regarding OSS that drive complexity, let us first have a brief look at the evolution of OSS and how it influenced software development.

In the early times of software development, OSS did not exist. All software was the property of the company that developed it. Sharing of software beyond algorithms shared via textbooks or alike was the exception. This also meant that companies needed to implement basically everything above the level of a 3GL programming language on their own, including all required base libraries and frameworks. ²

This limited the size of problems that could be solved reasonably with software ³. Quite some effort had to be spent on implementing base functionalities instead of solving the business problem at hand. In the beginning the code ratio probably was 90% base functionalities and 10% business problem. ⁴

Over time the 3GL programming language providers added more and more libraries, frameworks and other base functionalities to their compilers. Some vendors – typically also supplying the underlying OS – even provided whole programming and runtime environments including powerful abstractions. E.g., think about TSO as batch scheduler or IMS and later CICS as transaction monitors by IBM, including lots of libraries, frameworks and more base functionalities up to full-flavored IDEs and runtime support.

Roughly around the same time, standard software emerged. Instead of leaving all the business to the OS vendors, independent vendors started to implement software that could be installed on a given OS. This also gave rise to a completely new type of software that did not focus on IT productivity but on addressing business topics. The probably most prominent representative of this kind of software is SAP, but also lots of other standard software products emerged.

All these developments increased the productivity of software development a lot, i.e., the ratio of time spent on solving the actual business problem compared to implementing base functionalities rose. On the other hand, all these productivity enhancers were closed source and cost a lot of money. Additionally, hardware was still so expensive that usually only companies and other organizations of a certain size like, e.g., universities or government organizations were able to afford them.

This meant, the productivity of software development was bound to what vendors provided and what your respective company was able to buy which capped the overall productivity gain.

The rise of OSS

Over time computers became affordable for regular persons. Microcomputers emerged. Sharing of digital data became easier for individuals due to the rise of floppy disks acting as a cheap data exchange medium and the Internet (not yet the WWW, but all the protocols that existed before ⁵).

This evolution supported the forming of like-minded communities like, e.g. the GNU project that started to (re)build software solutions with the ideal of free access and usage for everyone. Simply put, starting with this “Public Domain” ideal over time several other variants emerged, with OSS becoming sort of an umbrella term. ⁶

In the beginning, relatively simple software solutions were (re)created as OSS solutions, some libraries or smaller applications. But over time, the solutions – often built on top of previously created OSS solutions – became more and more powerful.

Eventually, OSS solutions became so powerful and reliable that companies started picking up OSS – usually not relying on the powerful OSS community but requiring a support contract as they were used to from commercial software. ⁷

This in turn created a whole industry specialized in providing OSS support contracts. Yet, only a few companies were able to successfully implement this business model, i.e., were able to live from OSS support contracts. E.g., Red Hat was able to create a sustainable business model for a long time by licensing their own Linux distribution to enterprise customers, plus optional support contracts. But most companies were not able to create such a sustainable business model.

OSS today

As a consequence the business models around OSS changed over time. Today, many of the popular, widespread “OSS” solutions are actually commercial solutions. Additionally, the companies behind the solutions also offer an OSS edition of the product, typically lacking essential enterprise features like, e.g., monitoring, HA capabilities or integrations with other widespread (commercial) products.

While it would be possible to use the OSS version of the software and add the missing enterprise features on your own, very few companies choose this path. Instead, they usually buy the commercial variant of the solution – which conveniently usually also includes a support contract option. The idea of the OSS edition in these cases primarily is to attract software engineers with the goal to eventually sell the commercial version of the product. ⁸

But also many other OSS solutions exist that are not connected to commercial offerings and do not incur license or subscription fees if used in an enterprise context. These solutions are backed in various ways, ranging from a single or a few idealistic individuals who work on the solution in their spare time to companies who pay some of their employees to work on an OSS solution (their motivations again ranging from quite idealistic to quite selfish – in the latter case usually being heavy users of the solution themselves).

Today, you find an OSS variant of almost everything imaginable on a technical level, often in very high quality and maturity. Also quite some business-related offerings are available as OSS. But the coverage is by far not as high as on the technical side. ⁹

Many companies implemented an “OSS first” strategy in the last 10 years. Yet, the reasons are usually not rooted in a deep conviction on the decision maker level regarding OSS. It rather is a mixture of developers preferring OSS solutions because they have better access to it and trust it more, and decision makers hoping for risk-free license cost reduction. Also, project managers and architects tend to like OSS as it often spares them tedious discussions about license costs. ¹⁰

Effects of OSS

The effects of OSS on software development cannot be overestimated. OSS was sort of a democratization wave regarding software development. As all democratization waves this had a lot, primarily positive effects. E.g.:

• Many more developers had access to programming languages, tooling, libraries and more. This increased the number of qualified software developers a lot. ¹¹
• As a developer you were no longer dependent on the software your company was willing to buy. You had access to a lot of software building blocks, often in high quality.
• The productivity of developers increased a lot. They did not have to build all the functionalities from scratch anymore. They were able to start at a lot higher abstraction level.
• OSS created new widely accepted software and tooling baselines that further increased productivity. Just compare a C++ project of the 1990s that usually had to build their own string management classes (I had to do it several times) with the freely available ecosystems you find, e.g., in Java, Javascript or Python where you can create a working HTTP server with a single line of code.
• OSS boosted competition. Commercial vendors could no longer rest on their traditional monetization models selling relatively low-level software solutions at high prices (and higher-level solutions at extremely high prices). Low-level software solutions became available for free which urged the commercial vendors into higher-level segments, also making it harder to realize very high prices for their solutions.
• The whole software ecosystem evolution was accelerated. Building on lower-level OSS solutions, higher-level solutions were created up to complete application servers and more. The widespread availability of high quality OSS solutions acted as an accelerator for the whole industry.

Probably I missed some relevant effects, but I think the ones I listed already make clear, how much OSS influenced the whole software development domain. Especially in the early 2000s when companies started to pick up OSS at an increased rate, OSS created a huge productivity boost.

Summing up

In this post we discussed the origins and rise of OSS. We can state that OSS was a tremendous success stories that brought a lot of positive effects. Especially in the period between approx. 1990 and 2010 it changed software development dramatically in many ways and shaped a whole generation of developers.

As so often, also some misconceptions sneaked in. But that is the story of the next post. So, stay tuned …