We, at Quality House, strongly believe that at some point in their career, no matter of the position, if it is a QA, a Team Lead, a Test Manager, or a QA Manager, everyone has experienced a situation as the one described just below:
You and your team give the best of you to thoroughly test a software product and suddenly, in the end, when it goes live, and you believe that everything will be perfect, your expectations of great quality are actually not met? The product that is now live turns out to be a great failure, rather than a success. Your reputation is ruined, your team’s reputation is ruined, and the company’s reputation is ruined as well. Not only you now have bad reputation, but you also feel quite guilty for the result and you doubt your professional skills and knowledge.
So, you start to review the process you have followed, the test cases, the documentation (if there is any), then your actions, step by step, just trying to figure out what you did wrong or what you missed.
“The Release of a Lousy Product Is a Result of Not Noticing Some Very Typical Signs That Your Testing Is Not Enough.”
But, sometimes, it is not something that you did wrong to have a product that fails. More often even, you did your best, regardless of the time and budget constraints.
Quite the case, the release of a lousy product is a result of not noticing some very typical signs that your testing is not enough. Being focused on the testing process itself, it is quite possible to miss those little “hints” or “signs” showing you that there is something more to do, something to change quite drastically to have success. These hints are quite numerous and still, there are ten of them that usually happen.
Well, there is no reason to blame anybody for missing the hints. Even, if you have the gut that something is wrong, there is still that uncertainty as of how much testing is enough? If the exit criteria are already set, well, there is little to do about it and yet, you can try. It is quite difficult to set a clear criteria on enough testing – there are so many dependencies – deadlines, budget, team experience, complexity of the software and that is why you might get lost. One thing for sure, you are supposed to stick to is to assure the minimum acceptable level of quality. Otherwise, big fiascos are ahead of you.
So, we can advise, when you have the gut that there is something not quite well in the testing phase, make sure you analyze on an early stage if you have enough and the right resources – both human and technical, if you have the amount of money you need, the time you need and then be sure to set a clear line on whether you are about to face “not enough” testing or “not good enough” testing.
“Companies Spend Each Year More Than 310 Billion US Dollars on Debugging Their Software.”
Just to give you an example, according to a research from Cambridge University’s Business School, companies spend each year more than 310 billion US dollars on debugging their software. To that amount, we must add another 60 billion for fixing production bugs.
Even to be more precise, we would like to share with you the story of the Knight Capital. Only one defect in the trading algorithm for Knight Capital caused a loss of 440 million US dollars in only 45 minutes. That loss equaled 3 times the company’s annual earnings. And the shock that followed caused Knight Capital’s stock to lose 75% of its value in 1 business days.
In fact, Knight Capital shut their algorithm down after just three seconds of trading, but it had already lost more than 1 million US dollars. This is not the first failure in a software of stock companies. Numbers show that 4% is the average decline in the stock price following a newsworthy software failure in 2015 and averagely a decrease of 5.7% of the stock price for companies experiencing multiple software failures (according to Parasoft research). That equaled 2.7 billion US dollars total loss in market capital from these stock price dips.
“Management Misunderstanding or Under – Estimation Of The Need Of a Proper Testing?”
Then, you have the management misunderstanding or under – estimation of the need of a proper testing. It is indeed a good thing that you have hired the best professionals and that you are quite sure that there is no chance to have any bugs in the product. And we all know, that this is quite impossible. We are humans and we make mistakes, no matter what the managers think. Then, we have the phrase where testing is quite expensive and that it does not produce Return on Investment (ROI). Well, it’s not quite fair to expect direct and quite measurable ROI from the testing phase, but you will definitely see the pros in a long – term period. Ensuring proper testing builds reputation and trust and thus will attract more clients. There will be small or no benefit at all from it in a short – term. Never forget, that in its nature software testing is most of all a preventative approach.
In the minds of some managers, not only testing is expensive, but also testers are quite not needed. What is needed is developers to test their own code or best is we have the DevOps. We will go Agile and all team members will do both the development and the testing. What is not quite good here is that the mindset of the developers, the testers and the DevOps is quite different. The developers tend to perform only positive tests to prove their code works as asked. The problems quite usually are found through negative tests. The testers and the DevOps have also different skill sets and it is wrong to think that the DevOps can replace the Tester.
“The Testing Budget Is Actually Gradually Increasing Every Year.”The positive trend, despite the unwillingness of some managers to devote resources, money and time to testing is that the testing budget is actually gradually increasing every year.
And then, the priorities before the QA professionals in the year ahead are:
Keep an eye on our Knowledge Hub so you don't miss our next article devoted to the signs that strike us most that there is something not “quite enough”.
Quality House Team
Quality House Team