10 Signs Your Testing Is Not Enough (Part 1)

We, at Quality House, strongly believe that at some point in their career, no matter of the position, if it is a QA, a Team Lead, a Test Manager, or a QA Manager, everyone has experienced a situation as the one described just below:

You and your team give the best of you to thoroughly test a software product and suddenly, in the end, when it goes live, and you believe that everything will be perfect, your expectations of great quality are actually not met? The product that is now live turns out to be a great failure, rather than a success. Your reputation is ruined, your team’s reputation is ruined, and the company’s reputation is ruined as well. Not only you now have bad reputation, but you also feel quite guilty for the result and you doubt your professional skills and knowledge.

So, you start to review the process you have followed, the test cases, the documentation (if there is any), then your actions, step by step, just trying to figure out what you did wrong or what you missed.

“The Release of a Lousy Product Is a Result of Not Noticing Some Very Typical Signs That Your Testing Is Not Enough.”

But, sometimes, it is not something that you did wrong to have a product that fails. More often even, you did your best, regardless of the time and budget constraints.

Quite the case, the release of a lousy product is a result of not noticing some very typical signs that your testing is not enough. Being focused on the testing process itself, it is quite possible to miss those little “hints” or “signs” showing you that there is something more to do, something to change quite drastically to have success. These hints are quite numerous and still, there are ten of them that usually happen.


Well, there is no reason to blame anybody for missing the hints. Even, if you have the gut that something is wrong, there is still that uncertainty as of how much testing is enough? If the exit criteria are already set, well, there is little to do about it and yet, you can try. It is quite difficult to set a clear criteria on enough testing – there are so many dependencies – deadlines, budget, team experience, complexity of the software and that is why you might get lost. One thing for sure, you are supposed to stick to is to assure the minimum acceptable level of quality. Otherwise, big fiascos are ahead of you.

So, we can advise, when you have the gut that there is something not quite well in the testing phase, make sure you analyze on an early stage if you have enough and the right resources – both human and technical, if you have the amount of money you need, the time you need and then be sure to set a clear line on whether you are about to face “not enough” testing or “not good enough” testing.

“Companies Spend 61 Billion US Dollars on Debugging Their Software.”

Just to give you an example, according to research from Undo in collaboration with a Cambridge Judge Business School MBA, debugging efforts cost companies 61 billion US dollars annually. As software continues to become more complex in the years ahead, challenges associated with debugging applications will only continue to grow.

Even to be more precise, we would like to share with you the story of the Knight Capital. Only one defect in the trading algorithm for Knight Capital caused a loss of 440 million US dollars in only 45 minutes. That loss equaled 3 times the company’s annual earnings. And the shock that followed caused Knight Capital’s stock to lose 75% of its value in 1 business days.

In fact, Knight Capital shut their algorithm down after just three seconds of trading, but it had already lost more than 1 million US dollars. This is not the first failure in a software of stock companies. Numbers show that 4% is the average decline in the stock price following a newsworthy software failure in 2015 and averagely a decrease of 5.7% of the stock price for companies experiencing multiple software failures (according to Parasoft research). That equaled 2.7 billion US dollars total loss in market capital from these stock price dips.

“Management Misunderstanding or Under – Estimation Of The Need Of a Proper Testing?”

Then, you have the management misunderstanding or under – estimation of the need of a proper testing. It is indeed a good thing that you have hired the best professionals and that you are quite sure that there is no chance to have any bugs in the product. And we all know, that this is quite impossible. We are humans and we make mistakes, no matter what the managers think. Then, we have the phrase where testing is quite expensive and that it does not produce Return on Investment (ROI). Well, it’s not quite fair to expect direct and quite measurable ROI from the testing phase, but you will definitely see the pros in a long – term period. Ensuring proper testing builds reputation and trust and thus will attract more clients. There will be small or no benefit at all from it in a short – term. Never forget, that in its nature software testing is most of all a preventative approach.

In the minds of some managers, not only testing is expensive, but also testers are quite not needed. What is needed is developers to test their own code or best is we have the DevOps. We will go Agile and all team members will do both the development and the testing. What is not quite good here is that the mindset of the developers, the testers and the DevOps is quite different. The developers tend to perform only positive tests to prove their code works as asked. The problems quite usually are found through negative tests. The testers and the DevOps have also different skill sets and it is wrong to think that the DevOps can replace the Tester.

“While there are currently many challenges, organizations are taking advantage of the upheaval to improve their entire approach to QA.”

According to the World Quality Report 2020-21, as a result of the pandemic, 47% of respondents said there would be a significantly greater focus on customer experience validation and on usability testing. Meanwhile, 43% of respondents said that the focus on more and better collaboration tools will go up, and 34% said they will need more remote access to test systems and test environments. There have also been COVID-19-related QA implications around security: 83% of CIOs and IT directors said their application security concerns have increased over the past 12 months, likely a result of the move to a remote working world.

“Software Testing Market size exceeded USD 40 billion in 2020 and is estimated to grow at over 7% CAGR from 2021 to 2027.”

Just to give you an example, according to a research from Global Market Insights, Software Testing Market size exceeded USD 40 billion in 2020 and is estimated to grow at over 7% CAGR from 2021 to 2027.

Keep an eye on our Knowledge Hub so you don't miss our next article devoted to the signs that strike us most that there is something not “quite enough”.

Quality House Team