I was able to interview Lionel Ducrocq about an automation project experience full of twists and turns.
I wanted to share in more detail the practices that we identified in retrospect from our exchange.
- Remaining in control of your test, specifications and inputs repository is an asset
- Quickly test structuring and critical use cases including non-functional requirements
- Anticipate direct and indirect automation activities to build a realistic plan
- Use your instincts as a complementary guide to risk management
- Understanding the origin and the contours of a solution enables to evaluate its natural fit and integration
The following sections detail the points with actionable recommendations.
Staying in control of your test repository, specifications and inputs is key at all times
Would you outsource core functions of your business?
I imagine not and I am convinced that the reasoning must be similar for its repository of requirements.
Knowledge of tests is largely a reflection of business needs, which evolve, and reflect the company’s operating methods.
This is where the differentiating business value is, which proves to be a real asset of the company if mastered.
A controlled repository is a real asset of the company: a reflection of the requirements providing flexibility of implementation.
Antoine Craske
Its mastery ensures the relevance of test cases, whether manual or automated, performed internally or externally.
The control of our own requirements is a real strength through the flexibility it brings in an increasingly dynamic environment.
From shared experience, this allowed them to pivot quickly by changing tools, teams and partners.
Exceptionally, this also allows to perform manual testing if a problem is encountered on the automated tests.
Quickly test structuring and critical use cases with non-functional requirements
In this specific case, hundreds of test cases were identified within the scope of automation.
The feasibility study focused on four tests to validate different typologies.
However, the parallelization of the tests and their stability was not assessed in the first part of the project.
Testing non-functional requirements is also fundamental in an automation project.
Antoine Craske
In view of the structural infrastructure problems encountered subsequently, testing these non-functional requirements could have detected the stability issue.
We note that non-functional requirements are also structuring in an automation project.
It is therefore necessary to make these requirements explicit early in a project, and to define an adapted test plan.
Less visible and naturally evoked, they are easily put aside, at a price that can turn out to be expensive later.
Anticipate direct and indirect automation costs to build a realistic plan
In the experience sharing, we realize that the review and validation load was not foreseen.
Delay is created and accumulated, accentuated by the various round trips, the remote working context and different priorities.
A parallel can be drawn with IT development projects, where indirect costs to production tasks must be foreseen.
Automation tasks have associated overheads, such as specification, review, integration, and testing.
Antoine Craske
This is often an overlooked aspect in automation projects. We shorten the project only to automation tasks.
This load can be estimated upstream of the project in order to guarantee a better response to deliverables, inspired by the test life cycle (STLC).
A common practice is to allocate an indirect load by major category of complexity of the identified test cases.
This load is then to be adapted during the execution of the project when confronted with reality, and become subject to possible optimizations.
Much like software development and its peer review, dedicating time slots for test review and validation can facilitate direct feedback, collaboration, and mutual understanding.
Use your instinct as a complementary guide to risk management
Several people on the team noticed stable testing only in the partner environments and within scheduled demonstration.
In the project, the stability of the environment turned out to be a real limiting factor to iterate, which ultimately stood out as a blocking factor.
Other factors came into play, such as constraints on security and access to infrastructure.
Nevertheless, we feel that following our intuition and our instincts can sometimes put us on the right tracks.
Our intuition can guide us to the right tracks.
Antoine Craske
Not underestimating them and acting early can make the difference. An untreated problem tends to create others rather than resolve by itself.
Pragmatically, dedicating one hour per week in your diary for review and taking a step back enables you to identify corrective actions.
Personally, this is a practice I use to distance myself from operational subjects and gain perspective on the objectives and the evolution of the context.
In an aspect of collaboration and collective intelligence, organizing risk review sessions with the project team every 2 to 3 weeks is also relevant.
We speak of retrospective in the agile world, but remains less natural in traditional projects.
Risk management sometimes remains closed to the project manager, but it deserves to be open and shared for more value.
Understanding the origin and the contours of a solution enables a better understanding of its natural integration
Here I share a reflection that I read from the book The Software Architect Elevator, by Gregor Hohpe.
In the shared story, we realize that the technical platform deployed internally finally posed problems.
With hindsight, we wonder if the product was in fact suitable for on-premise deployment, we understand that it was rather made for another type of deployment.
It is here that understanding the origin of a product, identifying its core functions, origin, is very useful to delimit its contours.
Understanding the origin of a product allows you to define its contours and identify its limits.
Antoine Craske
In this specific case, if the solution was already unstable at first presentations, it was questionable to move forward with its wider deployment.
Concretely, building a list of questions for solution evaluations is a common practice.
Personally, I think we have to keep it simple and with a broad vision on a first approach, avoiding decision matrices with 100 criteria.
The questions should allow you to perform an 80/20 by identifying the key points of understanding and attention.
This approach makes it possible to limit a few relevant choices which will be explored in more detail.
An automation project remains a real challenge due to its complexity and transversality
We realize that a test automation project requires real organization, management and increased analysis of the context.
The fundamentals of project management, team management and software quality are key to the success of a large-scale project.
I hope this article has been useful to you in sharing, concrete and actionable, feel free to comment on reacting to continue the exchange.