The Round Table of the Quality Engineering Unit met to discuss the transition from Quality Assurance (QA) to Quality Engineering (QE).
The area of Quality Engineering Software aims to include quality in the transversality of the teams, product construction processes, and technologies.
However, having the word “engineering” in its name, does not mean that it is limited to implementing technologies.
We share the following themes:
- What themes for a transition from QA to QE?
- Which practices can be used, with which criteria of attention?
- How to define your priorities?
I thank each participant for their participation and contribution:
- João Proença, Principal Quality Engineering at Outsystems and International Speaker
- Diogo Rede, Engineering Lead & Test Engineer at Farfetch
- Luis Bastião Silva, CTO at BMD Software
- Filipa Nogueira, Engineering Team Lead at La Redoute
- André Macedo, Solution Architect at Sky Technology Center
Contextualization of the event
To start off, we share with João Proença what Quality Engineering is, more content in this short episode.
We started the event by identifying the topics of interest in this prioritization matrix.
Luís Bastião Silva started our conversation about the transversal risk matrix.
Risk awareness is essential
Are you familiar with the rule of not releasing the product on Friday?
Many experiences report a complicated weekend when this rule was not followed.
This rule is common sense in a complex ecosystem that is not yet at the expected quality level.
Returning back to the question, does the whole team have this risk awareness?
Risk awareness is fundamental for the whole team, in addition to operations teams.
Luís Bastião Silva
Luis shared that teams tend to have different risk definitions and scopes, ultimately resulting in concrete problems in operations.
A transversal risk matrix comes in this context to help to define, share and encourage a sharing of the risks involved within the development process.
The question of risks must become a reflex of the team over time.
Points that you can reflect in your context, which we discuss afterward:
- Are you personally aware of the various risks in the value chain?
- Is there a formalization of these risks, with prioritization?
- How can you define, share and animate your use with the team?
With the power to deploy to production, comes responsibility
Who is responsible for installing the production software in your organization?
For historical rather than technical reasons, we often find operations with this role.
The problem is not related to the team that makes it, but rather who is in the first line to follow.
There, too, developers may miss out, leaving that job to top-notch operations and support teams.
Being end-to-end accountable for a product is critical to start increasing its quality.
Diogo Rede
This view can be hard to fit into specific organizational models or processes, such as separation of concerns, shared services.
They have in fact other limitations, such as a product owner or a developer who has no direct feedback on the actual use of their product.
We share the fundamental need for the team to have the feedback loop of use in production, from users to the most technical part.
It can involve reviewing processes and the organization, and we can often find solutions that maintain the advantages of the models and that allow this feedback loop.
What can you ask yourself in your context:
- Who is responsible today for performing the deployment, and following it?
- Are there still valid reasons for the developer to don’t access operations?
- What automatic loop feedback mechanisms can you experiment?
Closer to the client, closer to the risk perception
We are human and adapt our behavior based on experience and often on personal emotional feedback.
Therefore, in order to improve the risk inclusion, the team must be in contact with those risks.
Risks in the software industry materialize themselves in its use: blocked, degraded features, slowness, data loss, etc.
How do you get this information?
A team must be as close to production as possible.
João Proença
Here we point out the need for Observability of the product, an area of great acceleration due to the benefits it can bring.
It is also parallel with the shift-right pattern.
However, until we have our dashboards automatically made, optimized, and managed with Data Science, we must act.
A concrete practice is to put the team in contact with the customer, directly when it is possible, indirectly when it is not.
We had complaints of slowness, which the product team did not recognize. They spent a week with the users and came back with points to improve.
Antoine Craske
You can materialize with simple practices such as placing users with the product team, organizing sessions with a beta-tester panel, …
There are also plenty of pragmatic tools for screen recording, analytics, and graphics that can focus on particular points of use.
Points you can take in your context:
- How does the operation of my product can be materialized?
- Who are the end-users?
- How could the team receive feedback from operations and users?
Quality Engineering involves a team responsibility and mindset
Do you feel like a united team in terms of product quality?
It is one of the priority actions to be taken before starting changes to processes or tools.
The team must feel like a unit from the conception to the operation of the software, beginning to include risk across the board, and learning from their experience.
Quality Ownership is the team’s responsibility for the quality, not just of one person.
João Proença
The evolution of the team’s thinking and dynamics are often put in the background.
In reality, they are the foundations to allow a team to improve the quality of their product, finding the best cross-cutting mechanisms themselves.
Diogo Rede shared cases in which the team ends up mitigating risk from conception with mechanisms such as feature-toggles.
Mitigation mechanisms must be provided beforehand by the team.
Diogo Rede
With maturity, a team manages to have an approach of continuous improvement based on the operations of the product, in which they iterate on quality practices.
This is the result that should be sought more than a perfect technical architecture at a particular moment in time.
What can you work with your team:
- What is the level of the transversality of my team today?
- Which interactions would be most valuable to initiate?
- What examples and concrete cases can I use as first successes?
Getting started and showing value is the first priority
“One last optimization and it will be ready”.
You must have heard this sentence many times, even by the same person for days, without delivering something functional.
Several factors bring into play here, the personality of the person, the culture of the organization, the objectives, and the perspective that was given to the project.
Pulling for a product as quickly as possible and testable in operations is a priority in Quality Engineering.
“Start small”, a transition to QE is an iterative process.
João Proença
João shared the need to see QE as an incremental and iterative path, it is not a project with a fixed end date.
You need to experiment with different techniques in order to identify and develop what works best for your product.
The focus should be on the value generated, not on a technical optimization in a silo that we can easily fall into.
What you can do in your context:
- What is your product as simple as possible that brings value to the business?
- How can you implement it as soon as possible?
- How to allow the team to learn from the first experience to define the next iteration?
More quality does not emerge only by Design
Does it take more quality by design to be included in the process?
It is a possibility that seems of common sense.
However, counterintuitively, theory should not be the first action to be taken.
We need to confront ourselves to reality as quickly as possible.
You need to have the courage to confront the reality early.
Antoine Craske
This is a link to the necessity of the team to be in contact with customers and the operation of their product early.
It also applies to the implementation of its processes, technologies, and quality tools within the software life cycle.
Concretely, how can you, more quickly and more regularly, confront the reality of your hypotheses, models, and products?
Which tests to prioritize in Quality Engineering
We started by sharing the end-to-end testing necessity.
Luís shared his experience where testing from end to end was the first priority that ended up bringing a lot of value to the business.
In the particular context, it allowed validating the stability of the product’s main functionalities.
Without necessarily finding the exact source of the problem in question, they were providing value to the business and the team.
Adapting to the business context is key to creating value.
Diogo Rede
Over time, they have been complemented by other types of tests with different objectives.
Unit tests were one of them and proved to be useful to improve the developers’ focus on use-cases and modularization of their code.
We agree on the mistake of wanting to automate everything, especially when the test plan exists and comes from manual tests.
We proceed to the testing pyramids, aligned which are useful models realizing the assumptions that have for each type of test.
João shared that the pyramid is a metaphor, in which the granularity of the components to be tested comes into play
One should not test with e2e what can be tested in lower layers.
João Proença
The balance of cost and benefit of the tests is a factor that ends up including a perspective of value for the business, forcing them to be formalized.
What matters is the risk we are reducing and at what cost.
As we shared in the test automation round table, quantity is not quality, quite the contrary.
In addition, we found that running the tests right from the start, whether end-to-end or not, allows you to quickly validate the incorporation of testability into your product.
A transition to QE is a transversal and incremental dynamic
Returning to our initial question, “From QA to QE, where to start?”, We address several points here.
We have identified that a transversal responsibility of the team having access to the operations of your product is a prerequisite.
Focusing on delivering a product that is quickly testable in reality is one of the keys to iterating, whether in the final software or in the support processes.
The question of testing needs the same reasoning: what is the value for the business? What is the cost-benefit? Which solution is the most suitable?
In the implementation, QE builds upon existing QA and DevOps practices, for testing automation or deployment pipeline for example.
We see market standards such as shift-left, shift-right also end up materializing in QE, both in organizational and technical terms.
For your success, your transition to QE is an adventure that must be conducted incrementally.