The World Quality Report 2021-2022 is a widely known analysis within the industry, conducted every year. Capgemini and Microfocus are sponsoring the initiatives that you can find on this page.
The main findings and takeaways from the report are the following:
- Focus on what matters.
- Standardize the use of test automation in QA, and use it from end to end.
- Stop talking about AI and ML, and start doing.
- Emphasize the availability of test environments and test data.
- Get management buy-in for intelligent industry initiatives.
Industry analytics and trends are valuable to step back on the main priorities to address and our day-to-day activities. But the difficulty is usually to define a concrete action plan in our context from general insights and data.
This article aims to provide you with concrete actions to progress towards Quality at Speed leveraging Quality Engineering. We will rely on MAMOS, the Quality Engineering Framework, to draw out an action plan:
- Methods: Systematically keep the focus on end-to-end business outcomes
- Architecture: Only retain the solutions that will survive for Quality at Speed
- Management: Keep transversal iterations to compose with existing and new expertises
- Organization: Define what to change in terms of power and influence from now on
- Skills: Confront yourself to the talent competition reality, defining an action plan
An online document containing the listed practices, associated resources and checklist are available at the end of this article. Let’s start by working in the right order and on the right priorities with methods.
Follow the QE Unit for more exclusive and regular content of Quality Engineering.
Systematically keep the focus on end-to-end business outcomes
Delivering Quality at Speed requires working on the most valuable priorities. There is no time to waste on negative local optimizations. In a world of fierce competition and uncertainty (a.k.a. VUCA), economies of speed surpass economies of scale to deliver valuable outcomes on time.
Like in Toyota’s way of dealing with problems on the factory site, our software outcomes are best measured by living the life of our users. We can achieve this without specific tooling by living the CRM and help-desk life. The NPS and usual KPIs can be helpful, but the reality is in your users’ feedback. If we are not resolving their pain points, there will be a happier competitor.
Therefore, this organizational focus on the end-to-end business outcomes can start there, with a continuous reinforcement with social media threads, verbatim and sharing within the company. Every actor must be clear that you are delivering customer solutions, not technology products. A good image from Yves Caseau’s talk at the JFTL was “From Customer to Code, from Code to Customer”.
The direct implication of end-to-end outcomes for different actors is stakeholders management. Each organizational element has a tendency for self-optimization inherited from survival mechanisms. Our goal in Quality is to counterbalance this force by aligning a shared value and mission. This effort is not only for operational or COMEX members; as the report states, there is a need for “management buy-in”. We can balance our effort reaching the Tipping Point.
Focus is the necessary element to achieve these end-to-end outcomes. Time is one of the most valuable resources we need to manage it like gold. Time flies, especially in this accelerated world, where a day of 10 hours is a consecutive rush of activities. Our capability of focus results from two key actions: selecting fewer priorities and removing waste. Waste is like extra-kilos, an extra-weight that is painful at every step, limiting us from achieving this lasting marathon.
We can concretely start by only keeping the initiatives where the “Why” and the “What” are clear from an outcome perspective. Leveraging our management buy-in and focus on value, it should be easier to negotiate. It seems hard to stop ongoing priorities, but the reality is that only a few ones are worth pursuing. Do the cleaning. Additionally, we can add an epic to address debt subjects regularly within our agile rituals like the Release Planning Day (RPD), sprint planning and retrospective.
We can apply a similar cleaning to our architecture domain.
Only retain the solutions that will survive for Quality at Speed
Quality at Speed constraints to rapidly deploy successful technology solutions while keeping flexibility for future evolutions. We cannot afford to be locked with a vendor unable to update its solution using proprietary or closed standards. Therefore, we must consider essential criteria in our architectural choices.
The first element is interoperability. The ability to switch pieces with relative ease supports speed of implementation and evolution. Maximizing flexibility is essential in our ecosystem to successfully orchestrate today and tomorrow. We are today like at the emergence of coffee capsules: a lot of vendor choices on pay-as-you-go but with a relatively high lock-in. In technology, relying on APIs that are standards, open and understandable is the first mandatory step. If your vendor only offers proprietary files, that is a first warning sign. The second step is to assess the Cloud’s maturity to ensure the platform can provide the desired high availability, elasticity, among other criteria accessible on the public Cloud.
These points lead us to the second one of automation. Our job is to automate successful business processes for users leveraging technology. The technology must provide reliable execution mechanisms and an easy configuration to support fast feedback loops. A good test is implementing a CI/CD pipeline with Quality gates providing visibility, environments and data to perform fast feedback loops. If you end up in a nightmare of complex code configuration, seriously assess your current solutions. Even if that seems like a standard practice, only 47% of the respondents share having quality gates in their CI/CD pipeline.
The combination of interoperability, Cloud and automation creates more intelligent systems capable of “headless collaboration”; meaning they autonomously collaborate. As a counter-force, this complexity of automated interactions increases the need for observability, especially to support human decisions. Observability is hard with progressive standards emerging in a dynamic ecosystem. When we were starting to be comfortable with Cloud, the Edge is coming for decentralized intelligent machines.
Hence the last point is to be prepared for the next generation of advanced automation based on Artificial Intelligence (AI). The machine-learning and deep learning algorithms are slowly expanding in narrowed contexts and use-cases. We can start by systematically preparing ourselves for future AI deployments by collecting and storing the data we are producing in our day-to-day. These data assets will be of great value and use as having the ownership for improved and personalized models. For concrete implementations, we need to carefully select the most valuable use-cases and matured technology. There is no time to lose in endless POCs.
A first approach is to leverage off-the-shelf solutions with ready-to-use models, leaving us with the already complex task of using AI products. As a final note, really assess the reality of the “AI” solutions from vendors. I am skeptical seeing 47% of respondents using AI to optimize past test cases within the current ecosystem. This report clearly pushes to adopt AI, hence the push for vendors to announce they do it. The behind the scene reality of some products is just old-school statistical algorithms to detect in the assessment phase.
The next area of Management is a key pillar of adjusting to a changing reality.
Keep transversal iterations to compose with existing and new expertise
QA Orchestration, Cloud environments, AI – the evolution of these practices requires the composition of expertise. The report shares at 59% the need for better communication and collaboration across the lifecycle. It is not enough to have good talent for specific positions. The role of Management is to compose end-to-end expertise to deliver valuable software in fast iterations, to deliver Quality at Speed.
The basis for an extended collaboration is the one of a shared mission. The manager must act like an architect coordinating the building of a monastery: every actor must be clear that their contribution is a piece of the same larger edifice. We can rely on similar techniques of vision, mission and values alignment in software. Additional complexities lie in software with direct contact with users through analytics, plus an accelerated competition and evolution pushing for constant product changes. An attractive purpose for companies is becoming the de-facto standard to keep the customer perspective, attract and retain the best talent.
One management action from the report is to accelerate end-to-end cycles. In the context of QA and test automation, continuous testing loops are a priority. A concrete way is to start by the most valuable software product lines, clarifying their existing state and performance. The goal for the team is to meet the Elite level of the Accelerate report: fast lead-time, stable deployment, high availability and recovery. The observed performance gap will give clues about the limiting factors to address: a missing quality gate to improve non-regression, a lack of gradual deployments with sanity checks, an unstable environment creating slow feedback loops.
This example of QA orchestration illustrates the management role of connecting end-to-end diverse expertise. The manager must therefore implement supporting practices to reinforce the transversal collaboration. Simple yet effective practices – if done correctly – must be systematically implemented like a daily stand-up and a retrospective. The common mission and purpose are the foundations to achieve that extended collaboration, favoring the creation of a shared culture.
Organizational Design remains key for end-to-end collaboration. A powerful silo can limit the organization to meet the high standard.
Define what to change in terms of power and influence from now on
Organizational design is about where money, investment and power go. Even in a non-political context or a startup, choices are necessary. Quality at Speed has no time for egos and self-interest limiting the value delivery. In this budgeting period, reflecting on the existing and target organization can change the next year’s performance.
Similarly to building software or solving a problem, start by clarifying the existing situation. What is the most valuable organizational piece today? Which perimeter has the most power and investment? What do you need for tomorrow’s competition and a better value proposition? While these questions can require more analysis, it is the evolution towards Continuous Architecture. A good way to identify the changes is also to list your main fears and gut feelings about changes to be made. You will usually find the most essential evolution to do and limiting factors to remove in the organization.
Once you identify the most critical changes, you can define their transition. In the bottom line, investments will change from one team to another, impacting the level of power and influence in the mid-term of the various actors. You can also rely on strategy and agile principles to manage these transitions; you don’t need to decide everything from the start. A first step can be to apply Scenarios Planning and Game Theory to define the possible moves that could happen. From that point, you can define which changes to perform on a trimestrial basis, leaving you room to adjust future changes from the upcoming and non-forecasted reality.
The last part of your investment also comes down to the technology choices you will make, favoring “economies of speed” rather than “economies of scale”. They can see merely technical options but this is not true. If you decide to move to a SaaS solution for a whole CI/CD or Observability platform, a team of 5 engineers will have other tasks to do post-migration. It does not necessarily mean they will leave; you probably have more valuable tasks for this talent. But it’s your responsibility to articulate these changes and plan well in advance the transition as technology evolves more rapidly than humans.
That leads us to the last structuring impact of skills to support your overall transformation.
Confront yourself to the talent competition reality, defining an action plan
The talent war is a reality in the context of digitalization. A company lacking the right talent is most likely to miss delivering Quality at Speed. Like in the simple example of a manufacturing line, the lack of an actor performing a specific task will impact the final product delivered; that’s the same with software with the concept of chain-link. This state is hard to reach, requiring the collaboration of expertise at the right level over the value chain.
Similarly to the organizational piece, we must first clarify our most valuable skills of today and tomorrow. Looking at the World Quality Report, the first criteria for better quality is of skills rising at 65%. I am convinced that Cloud Architect, Data Analyst and Quality Engineer are good bets in the current context. If tomorrow you mainly compose SaaS solutions orchestrating QA to drive performance, the competencies of these three roles seem more than necessary. For other skills that are emerging like no-code, it is best to keep flexibility watching the evolution, starting some training and identifying potential partners. The last work is to identify what we won’t need anymore in the future, preparing for the skills evolution.
The next step is to perform our skills gap analysis aligned with our talent pool. They will surely be holes to cover, forcing us to find more effective solutions. We have three choices for the type of resources: internal, external or gig. The main work for internal resources is to identify the evolution plan and how to address the skills evolution. You can use formal training, on-demand platforms and also rely on upskilling programs. This is not a one-person work or a sole HR subject. The best way is to lead these changes involving the stakeholders in the “Why” and the “What”, animating a pro-active “How”. Good talent finds ways to reach challenging goals.
Like delivering software, the best trajectory is incremental, reaching milestones and learning from it to adapt from the on-going reality.
You are ready for tomorrow only by acting right now
This edition of the World Quality Report was a good opportunity to step back on the on-going changes in the ecosystem. Mainstream ones remain constant while the Quality at Speed imperative translates for effective QA orchestration and AI deployment.
An article is a good way to share key messages and pointers for implementation. I thought it would be interesting to share a more actionable format based on an excel document with you. Each domain of the framework has practices listed with a first link to resources, where you can start to prioritize in your context. Access the document here.
Feel free to leave feedback to collaboratively improve it through the composition of expertises. That’s how we can start by acting as an example ourselves.