Starting a QA practice alone is not easy. The true challenge is to bring value to the organization’s purpose and mission, aligning the various stakeholders on the right priorities.
Lina Kulakova is the founder of the QualityTalks community. She joined tb.lx to take on the challenge of driving quality in the company. In this interview, we share her perspective, learnings and convictions about her experience.
We cover the following themes through this interview:
- Understanding the company context, purpose and mission
- Aligning stakeholders expectations on the value to deliver
- The importance to develop habits, autonomy and collaboration
- How to implement fast feedback loops to experiment the value of quality
- Select the architecture and technology to support the overall why
- Focus on prevention rather than reaction, focusing on having impacts
- Use wisely measurement conscious on its limitations
Follow the QE Unit to access exclusive and regular content from the community.
Lina Kulakova
Antoine: Can you start by introducing yourself?
I have been in QA for several years now. I don’t have an engineering background, but I fell in love with IT and QA especially. I started as a manual tester, and then I expanded my knowledge quickly because I was driven and motivated by everything I saw daily. I tried to learn different techniques quickly.
Since I was usually the only QA in the team, I was struggling to get feedback on my work; there was no one to compare with. This is how I started QualityTalks back in 2017 as a Portuguese QA community. We do provide content to a larger audience in English. We were used to monthly meetups. It has evolved right now with more talks, interviews and a Youtube channel.
In parallel, I assumed the role of head of QA in a swiss startup. I had to define the strategy from scratch, build the whole QA department from zero, and then grow the team. It was a huge challenge for me. I have learned a lot. Since then, I have discovered my niche and another passion within QA: define, think and align the strategy focusing on the big picture of companies.
Besides that, I had been working with Portuguese Women In Tech (PWIT) as a mentor, participating in differentiating initiatives. We are working to attract more women within the Tech world.
Antoine: Interesting what you said on having a transversal perspective and bringing value to the company as a whole.
It is a very interesting exercise. I have to put myself aside, think of the big picture. It’s very intriguing and stressful at the same time. I think it’s also why I like it so much. In the end, it is really rewarding when you start to get the fruits.
Antoine: I understand your company – tb.lx – is based in Lisbon, part of a larger group in the transportation sector. Can you tell us a bit more about it? What are you trying to solve, for whom?
I definitely enjoy tb.lx. We are in the sweet spot being a startup within a corporate world. We still have the freedom to make technology choices, experiment and learn fast in an agile environment. At the same time, we have the backup of a very large group, Daimler. Our mission is to build sustainable transport solutions to Mercedes-Benz trucks, fusos, and freightliners.
Tb.lx is focused on a great mission and I’m very proud to be part of it. We are working on sustainable transportation for our future which is crucial. We focus on the trucks and buses, which are inevitable and necessary to deliver food and goods. It is amazing to be part of a company with that purpose.
Our company is not that big, we are over 60 people and continue to grow. We are based in Lisbon with a strong culture of trust. We work in hybrid setups between on-site and remote work, keeping a focus on diversity. People in our team are from diverse horizons and are also based outside of Lisbon.
Antoine: Your challenge was to set up a Quality practice within the company, why?
The company had a fantastic engineering team, conscious of the need for quality. More and more, we see in different companies the need for quality. I would say that the picture has changed a lot over the last couple of years. While some times ago we had to convince people that we need a quality department, now it’s more natural for everybody. It is like a common ground that we need to have a QA team.
Although the engineering team started doing things without someone officially affected as a QA, they felt they would need somebody to channel their efforts and structure the approach. The company and the team were growing with an increasing amount of delivery. They thought they would need more structure and benefit from an advocate who would guide them through the quality process. For example, they wanted to build more reliable products.
Antoine: The growth was driving the need for someone with a broader perspective, connecting the dots between the different perspectives of value. Was this culture of quality something you incorporated from the beginning? How did it all start? What were your priorities during the first 90 days?
Exactly, I pushed for quality to not appear as a police checker in the company. I am not the gatekeeper, the person who says go or no go, but much more about favoring team collaboration, giving them more responsibility, guiding them when they need it.
Although they had unit and integration tests in place, we had to specify the common ground among us. It started by aligning where we are and where we want to be, what kind of tests we required, which level of testing. Even aligning the naming conventions and vocabulary was not an easy job. Integration tests mean different things to different people; some would call them end-to-end. The first thing was really to define this shared ground among all of us.
“The first thing was really to align a shared ground among all of the stakeholders: our current situation, a shared vocabulary, their expectations.”
Lina Kulakova
After that, I worked at understanding what the constraints are and what we want to deliver. There was no 1-size-fits-all solution. We had to understand the business perspective and drivers. I had to analyze and feel what we needed and what would be the main difficulties. I knew that we wanted to deliver the best products for the Daimler world, that we had a high degree of interactions with a variety of stakeholders.
That was like a guiding star for me because I knew that we needed something able to provide a good level of communication among different people.
Antoine: I imagine that relationships management was key. How did you address this topic? Did you favor 1to1 or more collaborative ways to align a shared definition of quality and value?
I had to interact with a broad set of stakeholders, not only in 1to1. I think a lot of people were expecting me to start. By the time I started, everyone already shared with me to book meetings with me, for some in groups. I introduced myself and what my priorities were overall.
My first weeks and months were really to understand what we wanted to deliver as a company. Then when I had a very clear understanding of our goals, purpose, and values, I gradually met more stakeholders from the outside world of tb.lx, namely Daimler.
Antoine: What were your quality priorities? How did you align them on business priorities? How did you prioritize the effort?
One major priority for QA was to align the priorities through communication. We are a company with internal and external actors. We had to guarantee that we were all on the same page and sometimes communication is just not easy. We are in a remote setup; we work in different locations and have different time zones. The objective was to align everyone on the same vision of what we are delivering and what we were expected to deliver.
From there, I focused our strategy more on the TDD (Test Driven Development) and ATDD (Acceptance Test Driven Development) approaches to align with the development team. We really needed their effort to be focused on what the customer would expect from us. I found a tool that is not well known or broadly adopted, KarateDSL. It is a variation of the old known Cucumber with Gherkin. It has one layer less than Cucumber. It is therefore a lot easier to implement, code and write tests.
I was able to guarantee that we would go faster because of that simplicity advantage. Additionally, we would have living documentation with clear test cases that anybody within the team would understand. That was also the case for developers. Even if they are not in the QA practice daily, they can quickly open the tests and understand exactly what we are testing.
“Quality is an enabler of fast feedback loops of communication between the various stakeholders.”
Lina Kulakova
That was our starting point: aligning with the business partners what exactly they were expecting from us, then transforming that into tests. From there, the development team could work and do their magic. They have the freedom to choose the best programming language, frameworks etc, as long as they guarantee to answer to the business requirements of our stakeholders.
Also, performance is key as we collect massive data from the fleet of trucks; we have invested in performance tests with Gatling. It is possible to link the KarateDSL scenarios to the performance tests to keep an alignment. Security is also critical. To that effect, we try to cover different types of domains of security. João Proença shares various themes in QA and in particular about risk-storming sessions.
Since security and performance were so important for us, we work at integrating risk-storming processes within our team. We start all development with risk-storming, then share with the stakeholders like product owners to define the different tests. We continue with the development and the coding parts, including static and security analysis at different levels. Now I am focusing on systematic ways to ensure security by implementing a DevTestSecOps framework rather than just a QA framework.
Antoine: From a business and product owner, the impact of quality was to share their expectations and being involved in risk-storming sessions. What were the most valuable results for them?
The business sees and values a transversal alignment of the work. The most significant change we could notice is for the engineering team themselves. They had no broad idea of what they were expected to deliver. They were mostly focused on engineering, technology and solution design instead of the more global business and customer perspectives. For example, they started to think much more about what can happen in scenarios of truck failures. That was something not happening in the past. They now have a much better structure, like a clear recipe and process they can compose on, add ingredients and variations.
We are working at expanding our practice at every level and also maintain it while we are growing. One thing I like in our context is that we accept to iterate and fail fast if necessary. We are focusing on performing fast feedback loops. We guarantee to have time to reflect on the problem and fix them properly. A statistic says that if you test the deliverables right away, it takes 4 minutes for a developer to fix the bug. But if you find out like a day after, it will already become one hour and so on up to weeks if found even later. The later we discover the bug, the more time-consuming it is for everybody.
Today, the development team knows exactly the expected tests and must be green when performing changes; it is part of our definition of done. Then we complement our approach with exploratory testing not necessarily systematized nor the development responsibility. The main value is that they can constantly verify by themselves the test results.
Antoine: Metrics and KPIs can be both powerful and dangerous in their usage. Do you use specific metrics and KPIs to support the quality initiative?
We would like to and we are using some. Our main focus was to guarantee that we had the DevTestSecOps and automated tests, dividing my time between teams. Ensuring the autonomy of the teams was a key point to consider. We follow for example the nightly builds and results of the first tests performed. I would say that we are not metric-focused yet.
“In the end, quality processes must become natural and part of our culture.”
Lina Kulakova
We have more emphasis on preventing overreacting as early as possible in the process. We measure coverage at different layers and different test levels, including the acceptance ones. We also try to guarantee that the errors and bugs we find are way much more than the ones found by our stakeholders.
Our main goal is to ensure a strong routine, quality standards and habits in the organization. It is about ensuring that we favor quality, sharing between teams and that teams are clear on what is expected from them. In the end, the process must become more natural and part of our culture.
Antoine: We regularly see the questions of who should perform the tests. I think it depends on the context. You shared some elements of yours, are the tests implemented by the software engineers for example?
I agree with you that it depends on the context. In our context, we have an engineering team focusing on the unit and integration tests, and I am more focused on the functional test levels. Although I am just the only QA for now for different teams within the company, my focus on ATDD brings critical thinking to testing. The tests themselves in gherkin are not too long to write; we pass more time on writing the test cases. I collaborate with various product owners to help me think about this.
One amazing thing we achieve is to commit the test definition that is then reviewed by the product owner, raising new questions and asking as a peer review. Developers are checking the code itself and we have the PO checking the use-cases. The tests are open to the whole team. That is a true collaboration between us. I am the owner and everybody contributes.
Antoine: You selected a more usable and understandable tool for the team rather than a perfect technology toolbox. It is an excellent example of a valuable quality decision on your journey. Which challenges did you face? What were the learnings on that cultural change process?
We had a lot of things that didn’t work as expected for several reasons. First of all, I would say that one of the biggest challenges is to work with a lot of data. It is live streaming data and live streaming technology. I had to talk to different people to understand how we could test it.
Live streaming is not easy. You cannot exactly predict what you will be receiving. That complicates how you are supposed to test it right. One of the biggest struggles I had to solve was that we don’t have a real available vehicle for testing. That was a problem to define the data set we could rely on. We needed to guarantee sufficient test benches and test data. A key learning was that we were not able to test everything and guarantee fast and reliable executions. We had to build datasets representing different scenarios. We relied on engineers to get realistic scenarios and build up our data sets.
The testing of streaming architecture is definitely not easy. I do remember that I started to write some tests. At first, it was a waste because it was not working well in a majority of cases. I tried to leverage postman, but due to the distributed technology, this is much less of a synchronous request/response verification. After a series of fast iterations, we ended up building the right framework and solution.
Antoine: What are the key takeaways from this experience? Which advice would you give to someone starting like you as a Head of QA?
First things first, we need to understand what we have to deliver. I believe that sometimes we want to focus directly on automated tests, implement pipelines, and get a lot of automation in place for each build. But we can quickly forget why we test in the first place. This is something to keep in mind at all stages.
When I selected the tooling for example, I was constantly assessing the capability of collaboration, being one of our major priorities. I discarded some of them that were not answering these requirements by performing quick experiments. I was regularly stepping back on what I was trying to achieve. We can quickly fall into worrying too much about fancy dashboards and tooling rather than the value we create.
It’s not a competition. It’s not about having the maximum number of automated tests. Automation comes for example to remove repetitive and boring tasks. It is not the goal in itself. We are not aiming at a particular number of automated tests. Sometimes it is so exciting that we can quickly focus on optimizing technology instead of the true quality.
Antoine: We should all put a post-it and daily reminder with “Step back on the why”. Closing up with a personal note, do you have specific inspirational content, like quotes, books, even outside of Quality?
That’s not an easy answer, not by lacking references but to the contrary, many things inspire me.
I feel inspired a lot by the team that I have. There is no perfect company, team or circumstances; we know that the grass is always greener on the other side. It is crucial to work in an environment where you can trust your people and they can trust you. You need to be able to communicate clearly, get there, and get things done. Having the right context, the team and a driving purpose are essential to me.
There is also the external world. I have the QualityTalks and I feel very inspired both by the speakers that I receive and from people who attend. They ask questions, bring their doubts, and are seeking for mentorship, help or advice. That inspires me a lot to keep pushing and to learn. Sometimes I learn more from questions rather than giving information. Receiving training and asking questions is for me a great way of learning. It also tends to put everything in a different perspective. It is very cool to have this QualityTalks community.
“Purpose is essential. We need to have an impact in this world.”
Lina Kulakova
I actually like to share ideas with different professionals in the area like João Proença. I also know an inspiring guy called Antoine, I don’t know if you know him in the QE Unit 🙂 People in IT, especially in Quality are approachable. I recently shared for example with Lisa Crispin who was greatly motivated by doing a talk together.
Books from Lisa Crispin and Janet Grégory inspire me a lot. Angie Jones is also a great source of inspiration. I tend to look for leadership and non-technical inspiration. Technical topics and matters change; one day it is DevOps, Cypress, and other tools. I am much more driven by purpose, aligning on the why and the impact. It is important to have an impact.
By developing a mobile app, you can help someone access a particular service they could not previously. By working on a travel platform, you are helping people have their dreams come true. I have the massive honor to be able to transform the future of transportation and this is amazing. I look for people that inspire me in terms of motivation, why we are in the QA in the first place. People that I mentioned here are great people in this field, and most importantly, a great source of inspiration.
Antoine: Purpose enables us to continue our effort even through hard times. I truly believe we share this importance of purpose for individuals as well as for organizations.
Definitely. At the end of the day, we have the ability to do things that we love which are testing and quality for us. We also have the opportunity to select between different businesses. When there is a true connection with our purpose, it brings the full potential of what we do.
That is also critical for the work-life blend. For me, it is more a blend than a balance because I do not stop quality after 5 or 6 pm. I still attend different conferences, meetups, or podcasts. Purpose is essential. We need to have an impact in this world.
Antoine: Thanks Lina for sharing your experience with humility. We learn by sharing and building up with communities; I truly appreciate what you shared and also thanks tb.lx for this openness. Have a great journey in building up your QA practice during your growth, and all the best with QualityTalks.
You can follow Lina on Linkedin, Twitter and the QualityTalks.
Useful resources
KarateDSL https://github.com/intuit/karate
João Proença (2020). What is risk storming? How to begin your Quality career? How to define a QA strategy? Quality Talks.
Lisa Crispin, Janet Grégory (2008), Agile Testing: A Practical Guide for Testers and Agile Teams. Addison Wesley.
Lisa Crispin, Janet Grégory (2019), The Agile Testing Condensed. Addison Wesley.
ATDD et BDD quelle différence. https://latavernedutesteur.fr/2019/02/25/atdd-et-bdd-quelle-difference/ La Taverne du Testeur