I have worked in the software industry for two decades now.
Over time, I have developed a clear view on things that contribute to achieving goals – and which do not.
This series is about how we, as developers or software architects, burn company money (while having fun).
You can find and discuss all the other parts with us on LinkedIn:
Part 1 – Trends
Part 3 – Docker + Kubernetes
Part 4 – Kafka
Part 5 – Low abstraction technologies like Go & Typescript
Part 6 – Using Slack for Monitoring
Part 7 – Cross-Team Collaboration
10-15 years ago, I was mostly working on monolithic applications.
And, boy, was I excited when the microservice trend hit the industry.
It promised to free us from many things that – in hindsight – were not really problems, but we considered annoyances.
We thought:
– These are more easy to manage.
– We can replace software modules more easily this way.
– Deployment is less of a risk and a hassle.
– Cross-team-collab will be easier/there will be reduced cross-team alignment-needs.
– They make it easier to scale certain software modules as they were deployed independently.
One of those is true: it allows you to manage a software module independently from others easily.
Another thing that is also true: Microservices come at a crazy price.
You pay with testability, for instance.
Younger audience will not remember, but 10 years ago most bigger applications could be tested automatically in full integration of all modules continuously.
When I look at companies today, the coverage of automated testing is terrible. The amount of incidents and defects is shameful.
You pay with maintainability. Yes, you heard me right. Most microservice-heavy environments end up being locked in their setup a lot harder than the worst designed monolithic architectures I have seen.
You pay with cash. To AWS, Datadog and all the services you need surrounding your core architecture.
Most microservice-heavy environments I have seen burn between 50k and 100k EUR. A month.
Funnily the spendings were unrelated to team size and load. Some of those had 10 engineers, others had 50. Some of them had millions of daily users, others – just a few thousand.
The requirements to the system seem to decouple from the technical effort and Resource usage. The opposite of what we wanted to achieve.
You pay in payroll as you need crazy amounts of engineers to barely be on top of the complexity of your software stack and architecture.
Don’t get me wrong. If you have a piece of software that can live on its own, it should live on its own.
But I would like to get across a few important things.
The most important aspect is that the architecture of different layers of your application and also the architecture of different lifecycle phases has different requirements.
Microservices are a RUNTIME Architecture pattern. They describe how you structure the deployables of your application.
Microservices are infrastructure modularization.
If you use them to modularize your code, to enforce clean software modularization and avoid module coupling, you are misusing a crazy expensive tool to work around inefficiencies and seniority problems in your team.
This will fail. If your team is not senior enough to structure code well in one repository, don’t expose them to a highly coupled codebase spread over a dozen repositories.
Do not overload them with the struggles of integrating these functional fragments of your application.
You can modularize your code well while it lives together.
And even in scenarios in which you want runtime modularization you can get away with keeping the code close well.
Service borders should be meaningful. Keep elements that interact to solve a shared problem close together.
Make a clear delimitation between services that exist to solve different problems.
Have you ever seen an Enterprise Technology Developer refactoring a huge enterprise application that was built over 10 or 15 years?
In a lot of cases it will just be a few shortcuts to change an application from database layer to the point of exposure (frontend, API, …).
The IDE understands the code, its coupling and its dependencies. It understands how software modules relate and interact with each other.
This provides massive value and an incredible productivity advantage.
Another important problem of microservices is integration.
Firstly, integration of software modules should be highly standardised and technology-inherent.
Secondly, you want the integration point to be as close as possible in terms of software architecture (within the application ideally).
Thirdly, the integration of software modules should happen as early as possible in the development lifecycle.
If software is linked at compile-time, the compiler will catch all glitches and make you aware of problems.
Linking at compile-time, allows testing the integrated software modules in the build-phase, long before deployment.
If you link late, in most microservice scenarios through HTTP or messaging and streaming infrastructure, you lose all that.
Btw: the significant efforts of consumer testing of APIs also do derive from the aspect that you decided to link your software modules at the latest point in time (runtime in production) and through the wrong layer of your software (network).
As easy as that. Don’t go into code or deployable separation, unless you have a really strong reason for it.
If you went into microservices without a good reason, congratulations, you burnt a crazy amount of company cash and left a way-too-big engineering team stranded in unnecessary complexity.
You probably can guess my view at this point: as microservices often are terrible due to too many loose ends to stay in control, Lambdas/Serverless functions do not help.
Further fragmentation of software modules leads to more chaos and even lower potential for automated testing.
Lambdas are a nice thing to use in certain scenarios (small operations within data pipelines could be one example). But breaking bigger business applications into nano-chunks to be operated in a serverless hell will not answer the problems you try to escape after having lost the microservice-battle, this much I can promise.
Can you see what I am talking about? Do you agree? Did you also fall for premature deployable separation with the struggles that come with it over time?
I for sure have regretted that “yes” to microservices once or twice in my career which resulted in 5* as big engineering teams with ⅕ of the productivity (giving a divide-by-25 return on invest in engineering salaries to the companies I worked for).
I am looking forward for heated discussions on LinkedIn!:
And the other parts:
Part 1 – Trends
Part 3 – Docker + Kubernetes
Part 4 – Kafka
Part 5 – Low abstraction technologies like Go & Typescript
Part 6 – Using Slack for Monitoring
Part 7 – Cross-Team Collaboration
