The debates of mono and multi repos are often focused on application code, but what about the test repository?
This is what we share in this article, analyzing the possible choices of repository and storage of our tests, in connection with our applications.
We assume here that your test configuration files are accessible to store in a repository.
You can preamble to this introductory article The Hands-on Mainstream Repo Models You Need To Know.
Join the QE Unit to access to more exclusive community content.
Why keep our tests in a repository?
Let’s reverse the question, what happens without a repository of our tests?
Each member of the team will store his test files, which can create several short and medium-term problems:
- Duplication of test cases
- Non-existence of test modules
- Loss of files in the various storages
- Conflict on changes of the same tests
- Inability to use the tests in the absence of a collaborator
- Waste of time in research, sharing and use
We can do better for our tests, what is more, are real assets of our companies.
Generally, they should be stored in a durable, secure, versioned, accessible and maintainable mechanism.
The need for portability may also be relevant, to keep control of your test repository outside of a specific tool, giving you more flexibility to change.
Once properly centralized, it becomes possible to support sharing and collaboration around your tests, solving some of the problems identified.
The software versioning mechanisms are also carriers of processes improving the quality of our tests, such as peer review supported by merge requests in Git.
You also have easy access to history, rollback capability, conflict management, on extended collaboration.
Does the choice of mono or multi-repos also apply to the test?
Our test files are stored in a defined tree structure, often grouped together in the concept of a project.
We, therefore, have the same choice of organization in mono or multi repos for our tests, with the same trade-offs.
Similarly, the different files can be organized at our convenience within a repository, their assignment to a particular repo being at another level of granularity.
For the tests, we add the question of application code inclusion and coupling: should we keep our tests with our application code?
What alternatives between code and test repositories?
We have two possible choices for each situation encountered.
The first is to maintain our tests with the code of our application within the same project, like a single mono-repo.
The second alternative will be to keep our tests in a separate project, whether it is in the same or different repo.
Through the advantages and disadvantages of each solution, we understand the importance of the context and accept its limits.
Which decision criteria to use?
The current context and its possible evolution are the first elements to clarify in order to articulate your decision-making.
Our objectives will highlight which points we value the most while accepting the limitations of a model.
The good news is, we don’t normally need a complicated decision matrix.
As in everyday life, we normally need 3 arguments to lean towards a specific alternative.
Here are the choices we are faced with:
- Should we keep our tests within our application?
- Should we use a mono or multi-repos for our tests?
Two choices are necessary for our test repos: mono or multi repos, mono or multi projects.
Antoine Craske
We can already proceed by elimination with the unit tests which must be stored with the project to be integrated into the build cycle.
Conversely, we can store code and tests separately if we want to keep our build pipelines free from test folder exclusions or other conflicts.
Taking into account the organizational context is for me a priority over technical considerations.
If the QA team is not familiar with distributed branching, versioning and collaboration models, the separation of code and tests is preferable so as not to impact development flows, while waiting to train them.
We have to consider the current and future organizational model that we want to incentivize, not limited to QA only.
A single repo of code and tests for a company promotes visibility, collaboration and cross-functional sharing, at the cost of managing a single repo.
This model may be suitable for companies pursuing these objectives, with a centralized rather than decentralized culture, whether it is a start-up or a large group like Google.
We can have the objective of accelerating the developments in parallel by small units having the different functions present. In this case, a multi-repos model will leave the freedom to include or not the tests in the application projects.
Our decision criteria are strongly linked to the culture, behavior and delivery models that we want to achieve, supported by a more or less strong level of coupling of delivery cycles.
What good practices to apply?
Good practices remain valid regardless of the model chosen.
A holistic vision of the teams, of the current and future organization, will guide us in the choice and evolution of the repositories.
Limiting the complexity to the initial needs will allow it to quickly iterate and create value. Keeping the rest in mind will have allowed us to keep flexibility.
For example, wanting too quickly to set up a complete model of Dev and QA multi-repos turns out to be risky for an unaligned organization lacking in collaborative maturity.
The basics of naming convention and versioning are structuring in all models, making it easy to find test projects linked to development projects for example.
Having a similar lifecycle, repository and branching model between code and test is desirable to facilitate understanding, alignment and maintainability.
QA repositories require real organization
The choice of mono and multi-repos is therefore also structuring for QA artifacts.
One could also wonder if there is a real difference in choice between the application code and the tests.
From a local point of view, we are indeed dealing with text files whose life cycle we have to organize.
On the other hand, the performance, ease and understanding of the integration of QA projects impact the development and deployment process, hence the need to keep this broader perspective.
It is with a holistic view of a system that we can align local choices for more global benefits.