Resolving software failures with continuous testing

More than three billion people were affected by software failures in one way or another last year. No industry was spared. Every major sector had several dozen software horror stories. The impact ranged from ‘best case’ scenarios of user dissatisfaction, poor usage rates, and customer loss – to worst case scenarios of serious security breaches, shattered brand reputations, severe revenue loss, expensive legal complications, and crashing stock prices.

Tricentis unearthed the scope of these failures in its annual Software Fail Watch report, which identified 606 recorded software failures, impacting half of the world’s population (3.7 billion people), £1.35 trillion in assets, and 314 companies. And that’s just the incidents that made their way into the news.

Why does this happen?

A constant stream of new digital offerings from upstarts, and improved offerings and upgrades from incumbents threaten companies’ competitive position and market share every day.

The ability to get new functionality to market ahead of the competition – like Bluetooth connected devices, Apple Pay, and facial recognition – can often be the difference between winning and losing. But in the race to meet customer demand, many companies are unable to assure optimal performance, functionality and quality.

Many organisations simply are not as evolved in their software testing processes as they know they should be. An inability to put the latest best practices into action, a lack of expertise with new technologies and tools, and difficulties putting the right team and skill sets together are all obstacles. And the faster pace being driven by DevOps and Agile initiatives only exposes those shortcomings further.

In a recent Infostretch survey of test/QA professionals and executives in the US and UK, participants reported their teams being very challenged by the increasing complexity of software testing and the growing pressure to ensure seamless processes and integrations.

Half of the respondents felt that the high degree of complexity caused by increased technological combinations was a major challenge. An equal number said that they did not have sufficient time to test new apps and services before pushing them into production.

While there was a strong understanding that automation can help speed test cycle time and was critical to advancing their DevOps and Agile goals, the majority of organisations had automated less than 25% of their testing. In the latest World Quality Report, the average level of automation was only 16%.

Both surveys highlight a huge opportunity for improvement. Reinforcing that point further – most of these organisations report a huge backlog of manual test cases that need to be automated before they can move forward on automating current initiatives.

If real progress is to be made, and higher levels of automation are to be achieved in a material way, enterprises must find ways to optimise complete testing cycles. Otherwise, they simply won’t succeed.

Standing still is not an option

Without continuous improvement in software delivery, digital projects are destined to fall short of consumer expectations. This can happen in user experience, functionality, performance – or a combination of them all.

And when it does, it’s inevitable that declines in market share, revenue and customer loyalty will follow. It might be a slow slide, or a sharp decline, but it will happen, especially where a high-profile software error occurs. Remember Equifax?

The point is you have to continue to innovate in software testing just as you do in software development. To deliver better products, you need to be able to test better. Most of us are entirely oblivious to how many versions and updates of our most-used apps hit the market weekly. Many auto-update daily without any of us knowing it.

This is just one example of how test and QA organisations need to keep pace. They must continuously improve their processes for faster cycle time and greater operational efficiency. Simply speeding up and automating your regression cycle alone is not enough to get the desired outcome.

Continuous improvement requires continuous testing

Continuous testing is the process of automating the testing of software on a broader level to accelerate the feedback cycle on potential defects and business risks associated with any given release. This requires a reassessment of the traditional test automation process; addressing the entire process of testing is the only way to effectively accommodate the faster pace and increased complexity of current application development and delivery.

Essentially, continuous testing demands holistic testing optimisation for higher rates of effective test automation. This can only be enabled by making some fundamental changes in the test and QA process, including:

  • using RPA to eliminate duplicates and enhance efficiencies of existing test cases
  • implementing strict automation guidelines with a self-healing bot to confirm adherence
  • leveraging data to prioritise test execution by risk, objective, and pass/fail prediction
  • using test data management and service virtualisation to conduct valid end-to-end tests in representative user and operating environments
  • aligning testing with quality milestones and risk analysis that can be factored into release decisions.

Many businesses focus on automating just one or two aspects of the testing ecosystem. There are a lot of tools on the market – many good ones – that do just that. But these tools can only do so much on their own. Continuous testing requires tight integration and coordination between every stage of the test/QA process starting with the definition of requirements & expectations and repeating with new insights, analytics and user feedback.

Continuous testing

As a result of this broader, more integrated view, continuous testing enables organisations to understand where they can and should automate for the best results, as well as where they shouldn’t. That translates into big cycle-time efficiencies while also truly enhancing the end product.

AI to the rescue

But there is still the issue of automation. Continuous testing can’t happen without a high percentage of your test footprint being automated.

Artificial intelligence (AI), including machine learning and BOTs, has emerged as a major disruptive force in the current digital landscape – 43% of survey respondents mentioned above see the increased use of AI as inevitable in new applications and devices, as well as software testing processes. These AI-powered test automation solutions, also known as intelligent testing, can help accelerate test automation in multiple ways, including:

  • optimising test case management
  • using adaptive learning to identify high risk defects and vulnerabilities
  • automating test result analysis to identify, categorise, and prioritise defect patterns for regression engineers.

By incorporating AI in their test automation efforts, businesses can increase their ability to leverage continuous testing to overcome time to market pressures while also meeting the stringent quality standards needed to achieve desired business outcomes.

Real business outcomes

Many believe AI will be the core enabler for organisations trying to achieve widespread test automation to fuel continuous testing. Nearly 60% of respondents to the survey see AI and self-learning technologies as a priority in their QA efforts, and there is a high demand for talent with intelligence-driven QA skills.

Continuous testing enables organisations to put in place the systems, processes, engineering, and automation needed to truly accelerate cycle time and deliver real business outcomes – higher quality products and services, enhanced customer experience, competitive differentiation, and operational efficiency.

Furthermore, it increases the flexibility of their delivery organisation to adjust to changes in the business, an increasingly important capability – no matter what business you are in.

Rutesh Shah, CEO, Infostretch

Related Posts