“Microservices = Multirepo”
It seems obvious, why centralize components whose purpose is to be distributed?
However, this is the choice that Theodo had to make for one of its customers in order to be able to continue to develop a growing product.
Darya Talanina shared the reasons for migrating 30 microservices to a Monorepo during a Devoxx conference edition.
This article shares their story illustrating the practices and results obtained through the prism of Quality Engineering.
Follow the QE Unit for more exclusive content of Quality Engineering.
The project starts with one repository per service
Some choices seem to come naturally.
In this case, that of having a repository per back-end project is set up, facing a front-end.
The Multirepo architecture then remains easily understandable.
The branching model is also distributed, based on:
- One to several feature branches per project
- Two stable branches dedicated to the “preprod” and “prod” environments
- A merge-request process by order of deployment.
This model allows developers to release features in their own repositories.
The first problems are starting to materialize
It’s comfortable to develop in your corner, without having to synchronize too much with the rest of the team.
Bad luck, collaboration is necessary to build a properly organized product to satisfy users.
The first need encountered is to centralize common rules in a common project, thus avoiding duplication.
On paper, following the principle of avoiding code duplication sounds like a good idea.
The first pains are nevertheless felt by the developers, even to do simple things.
Each change for a referenced component involves updating it in the various repositories concerned.
This means several minutes systematically lost, far from value creation.
Product growth accelerates reaching the breaking point
There is good news, however: the product finds its market and requires further development.
The project architecture goes from 11 to 31 microservices organized in Multirepo in the same technological standard.
On the other hand, the team is starting to be saturated:
- The initial build of 5 minutes leaving time to have a coffee now takes up to 2h30, enough to have an overdose
- Some pull-requests are forgotten in such distributed ecosystem, regularly creating bugs in production
- The implementation of common practices requires 30 actions in different repositories, which no one wants to carry out.
Darya clearly sums up the situation.
Heavy git workflow prevents us from improving developer experience and code quality.
Darya Talanina, Fullstack Developer @ Theodo
In addition to the developers, the product owners and the customers who are getting impatient.
All this time lost in updates is far from bringing the expected value on time.
Services are migrated from Multirepo to Monorepo
The decision is made to switch from Multirepo to Monorepo for all back-end services.
Migration consists of switching projects one by one to the new centralized repository.
This centralization also raises the question of build and deployment when more changes will be made on the same basis.
A continuous integration pipeline is set up to quickly give feedback on code pushes.
The continuous integration pipeline requires optimizations
The transition from a decentralized to a centralized model does not solve the problem of project build time.
The Monorepo poses the problem of knowing which projects to compile during a change, and which to do in parallel.
The first solution is to use tags to specify the structuring updates of the repository.
The explanation of the dependencies then makes it possible to compile only the projects impacted by a change.
The parallelization is then taken care of by the continuous integration pipeline.
Developers can finally deliver the expected value on time.
Quality at Speed becomes accessible after migration
The Monorepo continuous pipeline makes it possible to deliver changes in 10 minutes.
It is the day and the night when it used to be hours.
The team still has challenges to solve: automation of tests and their integration into the pipeline, progressive deployment, etc.
The important thing is that to date the team is able to deliver value just in time with minimal efforts.
Quality Engineering does not need to be complex, it must constrain the software chain to Quality at Speed with the minimum of Complexity.
And you, do you need a Monorepo?
References
Devoxx, Migrating 30 Microservices to a Monorepository https://www.softdevtube.com/2021/10/14/migrating-30-microservices-to-a-monorepository/