Prologue

One of the issues that you encounter on a daily as a SWE is how complex software gets over time. From a single file e.g main.go or index.js, intuitively, you've added external API calls/RPCs, database calls, micro-services context sync ups, state handling, error handling, logic testing and verification, security, e.t.c.

2 years down the road, you (could be as an individual or a company) have a couple hundreds of files. Suddenly, the new developer on your team takes a considerable amount of time to effect something as "simple" as an error text fix within the software system.

In your mind, you question silently, "Why is he/she such a snail?".

The truth is you do not recognize how compex the codebase has gotten because you've been working on it for a longer time. The new developer needs to create a mental map of the entire codebase, understand where to find which part and what every module truly does.

Bottomline is, recognizing how complex a software system has gotten, at first glance, is hard.

So, what do we do? Write software that's not complex? Can this be achieved? In this article, I explained that:

...software complexity can not be avoided but deciding whether the complexity/abstractions you are about to introduce into the system are essential or accidental is a call you have to make at some point.

I have gone on to digest more literature that I could find about what really software complexity is, how to avoid/embrace it and how complexity affects the overall developer experience. One illuninating resource I found is a research paper titled "Out of the Tar Pit" written by Ben Moseley and Peter Marks.

For one to truly understand the "software industry", they should be conversant with the two principles listed below:

i ) The Law of Continuous Change

Every software system in use today goes through continous change; updates, new features, bug fixes, e.tc. If it's obvious that a piece of software must changes over time, how often is this principle ignored?

Product managers:

We've got to be faster. Quick, quick, quick, let's merge straight to production.

Or,

Be agile and nimble. We made a promise to our customers. We need this ASAP!

A couple of seconds later, "Ooops 😓, the system is down." Let's roll back the changes.

As the "pressure" to deliver working software or making changes to the already existing one increases, so does the sloppiness and short-sightedness about the system's maturity. One would argue, it's a linear relationship.

This is why some startups make a choice to fully re-write the software powering their products. The software goes through many changes until it becomes spaghetti.

Whether you're a "10x engineer" or not, you will still fall victim to not knowing the future requirements of your system. As the business grows, the software requirements and specifications change and that clever code you spent time writing will no longer be relevant.

ii ) The Law of Increasing Complexity

As a piece of software evolves, so does the complexity.

Software systems are destined to grow and evolve over time. All source software projects have one thing in common: their complexity seems to follow an upward curve as the number of engineers working on the code base increases. To this end, having a comprehensive set of tests and establishing software design paradigms for the code base does wonders.

Causes of software complexity

• Complexity caused by state management
• Complexity caused by control
• Complexity caused by code volume

Each of the causes above deserves it's own book.

Other causes of complexity could be:

• Duplicated code
• Code which is never actually used (“dead code”)
• Unnecessary abstraction
• Missed abstraction
• Poor modularity
• Poor documentation
• Bad coding practices

Lessons and take-aways

1. Simplicity is hard to achieve

In his 1990 lecture, Corbato pointed out:

I conclude that there are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.

As Dijkstra noted:

...we have to keep it crisp, disentangled, and simple if we refuse to be crushed by the complexities of our own making...

Strive to keep things simpler.

2. Think twice

Before you write that clever code, think twice about the next engineer to succeed you. Or, you stay long enough to be bitten by the very tech debt you have introduced.

3. Care about the system's maturity.

Look beyond the present. However, make sure not to fall into the pre-mature optimizations trap.

Premature optimization is the root of all evil - Donald Knuth