For this story of Quality Engineering, I welcome Jean-Baptiste Crouigneau, Founder and Director of Prodigeo, specialist in software testing & quality, based in Morocco.
He is a consultant, trainer and auditor on all testing activities, with a predilection for aspects of test management, process improvement and test automation.
Jean-Baptiste is a founding member of the CMTL (Moroccan Software Testing Committee), and also active within the ISTQB (currently a member of the Exam Working Group) and the TMMi Foundation.
We share a project to define a test strategy in a high-stakes project, monitored by senior management, and with a fixed launch date to the public.
The project turned out to contain indirect objectives, far from a testing perspective, hence the title “The Story of a Hidden Agenda”.
A strong capacity for adaptation was necessary to balance the challenges of this project, complemented by enlightening elements!
We share here his feedback on the following points:
- Discover and follow leads to feel the context of intervention
- Identify the direct and indirect objectives of a project
- Understand the real expectations of the interlocutors
- Knowing how to stand on your convictions and values
- Adapt certain deliverables so that they are of higher value
Have a good listening, reading, or viewing at your convenience, feel free to comment.
Join the QE Unit community and get access to the exclusive content via the mailing list.
Antoine: Can you start by introducing yourself?
I am mainly an expert in software testing around three types of activities.
First, I do training to share experiences, in particular the ISTQB component, which is quite well known these days, with regard to testing. But I also work on requirements engineering.
Second, still in the field of testing, I perform audit with the TMMI model, less known but similar to CMMI for software development.
In addition, I also do fieldwork, I participate in projects, support teams together, and focus on results.
Besides, that’s why I have a lot of anecdotes and stories that have accumulated over time.
So I chose one of these adventures to share together.
The context of the project
Antoine: Can you tell us a little more about this story?
About four years ago I was doing an assignment at a bank based in Morocco, where I am currently living.
To contextualize the project, in Morocco as in many countries, Islamic banking offers have been launched.
It was a pretty long process that took a long time to come, and once structured it became a race between the banks.
The bank in question was Moroccan and wanted to create a new bank, completely separate from the existing one.
A late solicitation for a strategic project
We had been called to describe the testing strategy. As usual, at the last minute.
Jean-Baptiste Crouigneau
Antoine: So there was no budget or even tasks allocated to the test?
Not exactly, there were things planned, but developments were already started and well advanced, with a fixed date imposed by management.
Basically the schedule was not very realistic. We had 9 months left before the official launch of the project.
Antoine: What was the scope of implementation, the entire back office, based on an ERP?
Yes, it was a complete scope, based on a specialized Lebanese ERP, which provided a good base.
Still, there was a lot of work due to specific developments and the integration effort required.
We therefore started to meet the various interlocutors of the project, who reiterated their high expectations despite the strong constraints.
At the test team level, there were none!
There were four business leaders but they were not testers and had a lot of other work to do.
The planning and the availability of people were therefore two strong constraints.
Added to this were three phases of testing that had been planned and planned, without our being able to analyze the project.
Strong constraints force a prioritization exercise
Given those constraints, I figured I was going to estimate the load, divide by the number of days we have left, and see what could have been offered.
Antoine: In terms of time and resource constraints, did that imply prioritizing a perimeter?
Yes, indeed I will come to it!
We had received high level formalized requirements, between 500 and 600!
I made a little rule of three on the testing phase divided between design, implementation and execution.
We had identified between 500 and 600 requirements, for more than 1,500 tests “only”.
Jean-Baptiste Crouigneau
We also assumed that we were going to be able to get about ten testers to help us with the implementation.
On average, with 3 to 4 tests per requirement, we were able to perform about 1,500 tests.
The first version of the strategy is coldly received
Presented to the project team, it was a first shock for them, raising some non-negotiable requirements.
We therefore carried out a risk-driven approach.
The business has assigned a value and a risk to each requirement, to get a view of the possible optimization.
Following this first analysis, almost 100% of the requirements were still at a critical level, so without any real prioritization!
Several revisions are needed in a short time
We have tried with several iterations to get to a real prioritization, by ranking the requirements between them.
At this point, the project manager came back to us, escalating the delivery date of the test strategy, it should be ready within three days.
For our part, we wanted to be realistic and share the reality, not a plan we couldn’t stick to.
So we didn’t have much of a choice: to present something poor by the date set, or to ask for extra time.
As the general management was on hold on the project, we had little hope of obtaining any additional time.
So we said to ourselves that we were going to meet the deadlines, prioritize as best we could, and use lighter testing techniques when that was possible.
Antoine: So you tried to optimize certain techniques to achieve an 80/20?
Yes exactly, in addition we had planned to integrate experienced testers in the field of QA and in the banking sector.
Automation is inadvertently pushed by the customer
Coming back to our testing strategy, one of the four business managers offered to automate testing, for, in his view, save time.
For us this was not the solution given the additional investment required to set up the automation.
So I pushed for the automation not to be added to the project, to the constraint of the intuitive proposal that had been made.
The day of the official presentation of the strategy arrives
The three days passed and then comes the day to present the test strategy .
We therefore present it with the project director, the various impacted teams including the development team.
We share that given the constraints, prioritizing was mandatory.
Concretely, we had identified the services that were going to be necessary immediately upon launch, for example creating an account.
Less frequent and unnecessary operations at start-up were to be carried out later.
Despite a business focus, the presentation was not well received
Our proposal was not well received and our presentation turned out badly, to our surprise.
We can see that the project manager is not satisfied: it is too light, we cannot take this type of risk on a large-scale project, etc.
At the same time, we had exposed everything with the context, the calculation formulas.
Antoine: Did you also have to focus the prioritization on business value, taking into account the customer experience, functions and the dates communicated?
Yes completely, the proposal was aligned with the business context.
We are therefore a little uncomfortable and surprised by this type of reaction.
Key elements are obtained in a hallway discussion
At the end of the meeting, in a hallway discussion, we obtain additional information.
We understand indirectly that the project director, who asked us, actually wanted to demonstrate that the project could not be completed on time!
Antoine: So, because of the tests and the QA, the strategic project had to be postponed?
Absolutely, he wanted people to support his view and probably had more arguments to support it..
We, therefore, understood why our proposal was not suitable, we had to help make the project shiftable.
The deliverable is adapted to meet the real need
On the other hand, I had also developed a small tool based on an Excel to prioritize the tests.
By entering a few parameters such as the number of requirements, tests and testers, the project director could obtain load and schedule scenarios.
The idea was to think that the tool could be used in scenario building and as a possible supporting argument.
It was kind of like a way out for us, allowing us to go a bit in his way.
Antoine: A super interesting twist, we understand with hindsight what the real objective of the intervention was!
Exactly, everyone was aware that the project was not realistic.
Without participating in the project, the launch took place later than the scheduled date.
Understanding real expectations is a real challenge
Looking back, I recognize that you have to pay attention to the first warning signs, especially understanding people’s real expectations.
To deliver what is expected, we must be very attentive to the direct and indirect interlocutors who can put us on good or bad leads.
Pay attention to the first warning signs and the real expectations of the interlocutors.
Jean-Baptiste Crouigneau
Maintain your line of conduct despite external pressure
The other advice I can give, at least my course of action, is to stand firm on your convictions and ideas .
It’s easy to be influenced and adapt a plan to meet a more authoritarian demand, for example.
Stay firm on your convictions and ideas.
Jean-Baptiste Crouigneau
In the project we preferred to respond in a factual way by confronting the reality, to allow people to work in good conditions, rather than inside an illusory plan.
It’s not easy when the pressure is on. In our case, all the key and influential players were under great pressure.
Antoine: You could almost draw a parallel with defensive sports where you use your opponent’s strength. Is adaptation better than going against the tide?
Yes that’s a good image, our goal was not to end in a dead end.
Knowing how to adapt deliverables and have a way out
We were not able to deliver exactly what was expected, so we proposed an alternative allowing him to explore possibilities with realistic hypotheses.
Antoine: You almost want to ask what are the hidden objectives of the project when you receive a solicitation!
Exactly, it’s quite complicated when you don’t know the business context well.
You have to try to feel things, follow your instincts, act like an investigator.
This is also what makes the experience.
Antoine: Thank you very much Jean-Baptiste. I can hardly wait to chat again as I understand you have more stories and anecdotes to share!
Yes, I can tease a future story.
It is made up of two project managers, the top management wanting to separate from one of them, and has been the subject of some interesting moments and learning…
The full story in the next episode!