Today’s QA professionals must work on mastering new technologies such as AI, IoT, and blockchain in order to ensure that users will have the best experience possible.
For the first time, end-user satisfaction ranks at the top of surveyed professionals’ testing strategy goals. According to the recently-released World Quality Report 2018-19 (WQR), organisations are placing the spotlight on customer experience and experimenting more with artificial intelligence (AI) and machine learning (ML) to help them optimise their testing efforts.
Significantly, the report found that 99% of respondents are using DevOps practices. DevOps aims to deliver value to the user as quickly as possible, while keeping quality and security high. This requires more automation, which is essential to speeding up the testing process. However, the WQR notes that automation is the biggest bottleneck that is holding back QA and testing today.
This article will take a look at the most prominent findings of this year’s WQR, including a deep dive into the healthcare industry, and will suggest some steps that organisations can take to ensure they focus on developing the skills of testing teams while also keeping users satisfied.
Automate more but only with strategy
Automation is essential to speeding up the testing process, but the level of automation is surprisingly low, at between 14-18% for certain activities.
This can be explained by a number of factors – the complexity of building a flexible test automation ecosystem that is robust in the face of the ever- changing functionality of applications; an inability to provision reliable test data and testing environments; and difficulties in developing the right skills and experience.
Organisations need to focus more on end-to-end test automation; however, ensuring that you are testing the right things consistently does not happen overnight. You might already be doing some automation, so what should you automate next?
Jez Humble and David Farley state in their book, Continuous Delivery, that you don’t need to automate everything at once. The first step is to define your goals and identify the bottlenecks that are preventing you from getting there. Work out what you need to do to resolve them and make that a part of your backlog so it can be prioritised appropriately.
As you continuously prioritise and re-evaluate these items, you’ll introduce automation gradually and effectively over time.
To maximise the chance of automation success, the WQR recommends a staged approach coupled with test automation solutions augmented with smarter, more intelligent capabilities.
Step 1: Optimise testing
Before automating more test sets, organisations should first understand what their tests should be covering to maximise the amount of coverage and reduce risk. Teams today are using analytics from project data, production data and model- based testing, and they are adopting AI techniques to guide the direction of testing efforts.
By understanding what parts of applications are most important to their customers, organisations can feed this information into their testing efforts to ensure they are focused on reducing risk to the features that are most valued by their users. This can be done by prioritising testing to favour the most sensitive and critical areas.
Step 2: Improve automation
Test automation involves many moving parts, not least of which is the test environment. Manually deploying test environments is a slow process that tends to be error-prone. By automating test environment deployment and configurations, organisations can achieve predictable and repeatable conditions for their testing efforts. Building this into the continuous integration pipeline reduces the risk of test failure due to environment issues, and reduces the amount of time to determine the root cause of defects.
While manual testing is unlikely to disappear completely, teams must look for opportunities to reduce the amount of manual testing performed. Replacing manual testing with automated testing will expose errors earlier and reduce release cycles, but the frequency of change means that automated tests are fragile.
In addition to automating end-to-end tests, organisations must shift left and automate unit tests and regression tests at the API level. These tests should be run upon each check-in, and the results should be made visible to the team by displaying them on a dashboard to ensure timely feedback and resolution in case a test fails.
Step 3: Make automation smarter
In the first step, I mentioned using analytics to make critical decisions about which tests to run. But analytics and AI also play a key part in creating a smarter testing architecture that helps in test case design, the identification of risk factors, as well as analysing failures to recommend possible solutions.
Self-monitoring and self-healing architectures will also become more prevalent over the next few years. You don’t need to do this yourself. Testing tools today already incorporate many AI, ML and analytics features. In fact, the lifecycle management tools and test automation tools you’re using today are likely to include features such as:
- Pipeline failure analysis, to understand why a build failed, and to provide actionable insights to get the pipeline working again
- Hot-spot analysis, to identify areas of code that tend to be prone to defects
- Cognitive analysis, to identify text and data from applications and graphical images
- Anomaly detection, to identify abnormal behaviour in performance tests.
Leverage these features to help your organisation implement better and smarter automation throughout your DevOps pipeline.
We already saw that artificial intelligence refers to the application of AI to QA and testing, but it also refers to the testing of AI algorithms and applications. It should be noted that it is an evolving technology that is highly technical in nature, and consequently, requires a certain amount of expertise to leverage correctly.
Nevertheless, the results of the survey indicate that many respondents are either already adopting AI, or are planning to apply it over the next year to internal processes (62%), QA purposes (57%), and customer processes (64%).
However, it’s not that easy. Although they understand the importance of AI and want to apply it, more than half of the respondents are struggling with identifying where to apply AI, and many have experienced difficulty integrating AI with their existing applications.
This indicates that there is an increasing demand for people who have mastered an additional set of highly technical and mathematical skills like mathematical optimisation, neurolinguistic programming, and algorithmic knowledge.
The rise of new roles
Finding professionals with these skills today is not easy and it may become even more difficult as more organisations start adopting AI techniques. The World Quality Report discusses the rise of new roles in QA and testing:
- AI QA strategists, to understand how to apply AI to business. They will need to master both business and technical knowledge
- Data scientists, to sift through test data and use predictive analytics, mathematics, and statistics to build models. They must have a deep understanding and experience of data analysis techniques
- AI test experts, to be involved in the testing of AI applications. In addition to traditional testing expertise, they will need to understand machine-learning algorithms and natural-language processing techniques.
While AI requires a new skill set for testers and QA professionals, it is by no means the only technological challenge for this group of staffers.
Focus on other new QA skills
The report notes that trends such as AI, Internet of Things (IoT), and blockchain all require new skills and roles for QA and testing professionals. The adoption of IoT technology is rising to the point where 97% of the respondents have some kind of IoT presence in their products.
IoT devices can capture very large amounts of data, which might be sent to the cloud for processing, or processed in part or entirely on the device itself (edge computing). These devices also communicate using a variety of protocols, some standard and some proprietary, and implement various sensors in hardware.
Some 66% of respondents also say they are either already using blockchain technology or have plans to do so over the coming year. No longer exclusively associated with the cryptocurrency Bitcoin, blockchain is being implemented in many systems that require a secure, distributed ledger to record transactions.
Implementations must take into account security and data risks and mitigate dangers associated with integration into other systems.
Hiring personnel with the necessary skills can be a challenge, so the report recommends building up the skills within the current workforce. It presents a four-stage approach:
- Since automation is now an essential skill for QA today, find ways to bring in new agile test specialists with automation skills, or train your existing people
- Deepen the automation skillset in the team by adding software development engineers in test. They will be able to introduce and maintain automation throughout the development pipeline, and apply advanced skills and development techniques to make testing more efficient and maintainable
- Ensure the team has specialised skills in security, non-functional testing, test environments, and data management
- Bring in QA experts with AI skills to optimise the test focus and testing effort.
Taking the pulse of healthcare
The healthcare and life sciences sector has traditionally lagged behind when it comes to adopting the latest technologies and development methodologies. It is risk-averse, being highly regulated, but at the same time, patients are being put at the forefront.
In fact, 43% of respondents in the healthcare sector indicated that ‘ensuring end-user satisfaction’ is top priority, compared with 42% across all sectors. This is a very encouraging finding, and is what is pushing the sector towards a more holistic approach to healthcare and health management.
While the sector is adopting digital health solutions, such as apps to track and act on changes in health metrics, regulatory requirements must be met.
This means that many organisations can’t break away from requirements-driven development to agile development, and are still testing as a distinct phase in the software delivery lifecycle. Organisations that are looking to improve on their testing are reporting that the biggest obstacles are a lack of end-to-end automation from build to deployment, slow testing processes, and a lack of skills for QA and testing staff.
Nevertheless, the sector is seeing adoption of cutting-edge technologies such as blockchain and IoT-enabled devices, and has even recognised software itself as a medical device. Such software used to be called ‘medical device software’ or ‘health software’, but today it has the name Software-as-a-Medical- Device (SaMD).
The software has its own regulation requirements to protect patients and promote safe innovation, requiring the sector to adapt to a different type of lifecycle testing.
Unlike most of the survey respondents, those in the healthcare sector indicate a certain reluctance to move to the cloud.
Security continues to be the most important objective, with 47% putting it at the top of their IT strategy.
AI is beginning to make inroads into the Healthcare sector. Organisations will be looking at it seriously to facilitate smart testing and automation to help make sense of the thousands of combinations of scenarios that must be tested to reduce time to market while minimising the potential for human error.
The prognosis for testing and QA
In general, the outlook is excellent. The user is top of mind for QA professionals today, although they must work on mastering new technologies such as AI, IoT, and blockchain in order to ensure that users will have the best experience possible.
This can be seen across all of the sectors that were analysed in the survey. The healthcare sector in particular will need to focus on automating test data and test environments, because of the huge number of scenarios that must be validated.
With back-end systems that are largely using waterfall methodologies, they will be looking towards a digital transformation over the next few years that is enabled through smart automation.
Their major challenges will be putting together a testing strategy and roadmap, and acquiring the skillset required to master the move to an agile and DevOps approach.
While the World Quality Report includes findings and implementation recommendations for addressing emerging trends in the QA and testing space, it also highlights a quickly changing environment that will need the right people and tools to move forward successfully.
Whether we choose to concentrate on the healthcare sector or other industries covered by the WQR, growing areas like automation and AI when paired with the right QA skills on a team, will ultimately have a positive effect on the experience of the end user.
Raffi Margaliot is Senior Vp & GM at MicroFocus