Sometimes during estimation, refinement or planning we already think about possible places it can be broken, where it can fail, what should we check. Then we might form some test cases or prepare some exploratory session. But in the end, especially when we face a complex part of the software, this evil whisper appears in our heads: Is there anything else we should check? Did we cover everything? Did we miss some broken part? And we try our best, but we are just a human being under restrains of time, knowledge and technical capabilities. Often we live in fear, that we missed something very, very important and it will fail miserably.
And You know what? It just happened to me and my team. We thought about many different cases we should check during our tests. We found out many issues and after few months of re-work and re-tests, we decided it was fully tested and ready. It was so painful piece of software that we wanted to open a champagne and celebrate it is finally done and closed. We moved to another piece of software and forgot about this pain. The time had passed and the release was coming closer – just by accident we used this already tested piece of software again. And it failed. Badly. Randomly we selected data, that we never considered that should be used during our tests. Our last hope was, that if we did not consider it should be used, maybe then it is not the valid business scenario? So we asked. It turned out it was a valid business scenario that will be used right after the release and we overlooked it.
It may happen You will miss something
Even if we would spend another month or two testing this piece of software we would not be able to find this issue. This case was simple – this data was not available for us at the time of our testing. On the other hand, we knew this data will come one day, but we did not think to leave some note or one little task to check it when it comes. We just assumed this new data will behave this same as the old one.
Never cover things up
I am very proud that when we found this issue we just exposed it to everyone. Team members, managers, client – everyone. Noone though even for a second about covering it up. Issues we find might affect many things we are not aware of – like even if we go with the release or postpone it. At first, it was scary – so close to release there are many emotions in motion and bad things can be said (like “Who tested it?”, “Why it was discovered so late”). But in the end, being transparent is the only possible scenario.
Solution focus instead of blame focus
As said previously if there are emotions and losses the search for a guilty party might start. Who tested it, who wrote it, who gathered requirements? I’ve seen some follow this scenario and dedicate all their forces into throwing the blame on teams and people. Instead of dividing it is very important to bring teams and people together and work out the solution. In our case the same day the issue was found a solution was prepared. Noone was found guilty 🙂
The Pareto Principle is a fact
As it goes to my story I believe that we made a huge mistake to forget about Pareto principle. This piece of code was very complicated and we found many defects within it. It was easy to guess that if there were any issues missed – it will be there. It should motivate us to be extra careful about it and spare even more time and effort with coming out with new ideas what can go wrong. Instead, we were just happy we can close it and forget about it. That is bitter, but necessary lesson.