In the world of software testing and the development industry, QAOps has started to play a rather important role. QAOps is a combined process, which integrates Dev, Ops, and QA (Development, Operations, and Quality Assurance).
QAOps can bring a lot to businesses’ practices as the QA team works closely with the software development team and operations team. This allows a quicker software delivery process as well as provides a quality web application along with it.
To have a better insight into the topic, we asked experts in the industry to share their inputs on what is the role and future of QAOps.
What is QAOps?
QAOps brings together Development, Operations, and Quality Assurance in order to improve the quality and development lifecycle of an application within an organization.
Girish Vasisht, Senior Portfolio Test Manager at EE, defines QAOps as being similar to DevOps but more focused on QA aspects of the system development lifecycle (SDLC). Indeed, QAOps is a set of QA activities (and not just testing) that are performed by engineers alongside planning, development, and operations.
Aman Dhudwarr, Senior Test Consultant at First Derivatives, continues on that point by saying that QA is already an integral part of the DevOps process and plays a key role in achieving this higher-quality software. Therefore it is introducing a new angle to DevOps where QA and operations similarly work together, and the importance of QA testing within DevOps is emphasized.
Moreover, according to Lewis Prescott, QA Automation Lead at Cancer Research UK, QAOps is the process of applying QA practices to all parts of the deployment process: from development through test environments to production, usually through CI/CD tools.
Ravi Shankar, Test Automation Engineer at PubAudit (For Chase Farm Hospital), adds that we need to integrate QA into the CI/CD pipeline as well. The software testing process should be integrated into the CI/CD pipeline rather than being an isolated process. Although the silos between QA and development teams have been removed, the QA team needs to work closely with the operations team.
‘QAOps takes the idea behind DevOps to develop continuous delivery pipeline integrating development, QA, and Ops into one platform.’ Ravi highlights.
Likewise, Nathan Ballantyne, Senior QA Engineer at Babylon Health, emphasizes this point. According to him, QAOps is about integrating quality into the CI/CD process. This process involves working effectively with developers (and others involved in the CI/CD process) to make sure the right automated tests are being run at the right time, and in the right way.
Thus, he says, you automate as much as possible by using the testing pyramid as a guide, i.e., more unit tests and fewer e2e/UI tests, and these tests are executed at the right point in the CI/CD process. For instance, you should be running all of your unit tests whenever merging to your master branch, so you at least get alerted if you have introduced a breaking change.
Furthermore, Nathan adds, testers shouldn’t be limited to getting involved in the deployment phase, they can get involved as far back as requirements gathering and user story mapping by “shifting in all directions”
Knowing whenever a business should use a QA framework is important and according to Lewis, it should be used for any event where feedback is needed, any bottleneck, any blind stop you want to learn more from.
Lewis adds that it ultimately comes from the customer, wherever there is a point in your process that could impact the customer through delivery or quality.
Girish also highlights that a QA framework can be used at every stage, whether it is in UX/CX, design, development, and production post-go-live during day-to-day operational/content activities.
Aman points out that a QAOps framework could be applied in the following high-level areas:
- Automated Testing – automate as much as you can!
- Regression Testing – reduce the effort when updating and re-releasing a product.
- Parallel Testing – run multiple tests at once.
- Functional Exploratory Testing – unique manual test cases.
- Test scalability – scalable infrastructure for hosting tests.
- Integrate Dev and Ops with QA – integrate QA process in the CI/CD pipeline.
She also notes that whilst a QA framework is more powerful with the support of automated testing, manual testing should not be ruled out when evaluating critical applications.
Nathan highlights that you can drive quality throughout each stage of the SDLC (Software Development Lifecycle), you don’t need to limit testing activities to just a dedicated testing phase.
For him, assuring quality isn’t limited to just identifying bugs in code but actually about delivering the best possible experience to our end users which could mean suggesting tweaks to the behavior of a feature or the UI. It could even mean suggesting improvements to how your data is being returned via an API and how it’s being consumed by the client.
One of the easiest ways to encourage this behavior, Nathan continues, is ensuring that 3 amigos sessions are involving the tester(s) or you spend some time pairing with developers in order to help encourage the quality mindset. You can also focus on QA activities postproduction via monitoring, testing in production, and A/B testing, among others.
In order to understand the need for a QA framework, Ravi tells us that there are a few questions you should ask such as:
- How big is the application?
- Are you having a number of applications?
- How technically fit is the team to use a framework kind of thing?
- What is the ROI?
- How many different teams you have who need to use the same framework?
So, he continues, by asking these questions, you need to understand if our applications have many reusable elements that can be used across different application development, if the answer is yes, then the framework is necessary.
Besides, if you are developing a huge application running into a number of pages where locating elements and maintain the code base becomes difficult, a QA framework is the right approach.
How to integrate QA in development and operations?
Many organizations are already using the Agile process, which is a more than 15 years old concept now.
Hence, with Agile, it is very important to break silos and integrate QA into the development process. Ravi then tells me that the earlier the QA gets involved in the development process, the less there is the risk of getting a bug in later stages of development. Indeed, the later the bug is identified, the more costly it is to fix. It works similarly to integrate QA with the Ops, as QA already plays a major role in the release.
This is about giving more responsibilities to QA, which is good for the industry.
Moreover, Girish points out that QA is integrated by developing a framework that provides fast and early feedback to developers during the CI/CD process and triggers automated tests in production for operations.
Lewis also highlights that, in order to integrate QA in development and operations, it is vital to ask the question “why” & “how”, build the right thing and measure how you built the right thing. Thus, you need to use the data from operations to reflect the validity and reliability.
Nathan notes that, during the 3 story kickoffs, automated tests can be discussed at that point so before a developer even starts writing code, they have already agreed what test automation will be covered and at what layer; and the testers can be paired with the developers to write them.
Hence, he says, this implies a number of things: it encourages collaboration between testers and developers, as well as gives insight into how testers test to ultimately impact the quality of the application. Finally, it encourages shared ownership of quality across the team.
Aman also adds that continuous Integration and continuous delivery are the fundamentals of DevOps and QA teams must work closely with the development and operations teams to integrate the software testing process into the CI/CD pipeline as opposed to being an isolated process.
This can be achieved through the likes of developers contributing to the creation of test cases and Ops engineers reporting user experience problems with live apps. She emphasizes that closer collaboration with other teams will provide a clearer perspective of the overall QA process and increase the turnaround of bug-finding and bug-fixing.
‘The fundamentals, as part of the QAOps process, therefore become Continuous Integration, Continuous Testing, and Continuous Delivery.’
The benefits of QAOps
QAOps comes with many benefits for the testing teams but also the business in itself.
Nathan emphasizes the fact that QAs won’t be working in a silo and thus, can get involved earlier on in the SDLC to help drive quality activities. This then encourages pairing between testers and developers. It helps with a shift toward an approach where testers get involved earlier if they aren’t already and should encourage developers to write more/better-automated tests that will be run as part of the CI/CD process.
Besides, he adds, for organizations that don’t have a dedicated testing resource, it’s a great opportunity to get conversations about the quality going and to start introducing (more) automated tests.
For Lewis, the main benefits of QAOps are having testability combined with observability. Indeed, mirroring your metrics from production to your test environments, means you can target your testing with more real-life scenarios.
Ravi points out two vital advantages of QAOps: Increased Quality and Productivity.
Indeed, to him, the main reason behind integrating QA with the development process is Quality. If you can integrate QA activities with the CI/CD pipeline by using automation, the product is likely to have high quality.
Moreover, removing silos is so important as it reduces the feedback cycle and you can get instant feedback, which is very important to boost productivity. Hence, by integrating everything, you can achieve frequent releases and a quality product. Besides, Ravi adds, continuous interaction with other team members helps the QA team in understanding the requirements more clearly and performing the testing activities in a better way.
Aman highlights that besides the obvious increase in quality of software, QAOps also improves the software delivery process making it faster and more stable without compromising the quality of the product.
Indeed, she continues, the increased collaboration leads to bug fixing at earlier stages rather than later, which is what often causes time delays and increased costs in more traditional approaches, as well as preventing any lag between releasing and testing new applications. And, of course, a better quality product means happier customers!
According to Girish, here are the principal benefits of QAOps:
- Defect prevention
- Fast feedback to developer,
- Fail fast & learn from failures,
- Proactive monitoring,
- Alerts and ability to run tests on demand anytime anywhere in any environment with consistency
The challenges of QAOps
With every update, there will always be some drawbacks.
Indeed, Ravi points out that one challenge may be the QA team losing focus on QA activities or having too high a learning curve that can slow them down. There might also be the fact that some senior testers may feel isolated because they may find it difficult to learn new technologies.
Hence, the process could become so fast-paced that everyone may not obviously feel comfortable. It is then very important to strike the right balance and make sure no one in the team is left behind.
Lewis also highlights that data overload, getting side-tracked debugging a low impact issue in production, or even spending months setting up the local environment can all become challenges.
‘It can be overwhelming realizing that QA now includes everything within development and operations.’
Aman notes that there’ll always be the short-term pain and financial costs of implementing a new process before realizing the long-term gain. Indeed, there is a lot of debate around whether ‘QAOps’ should exist or not and may take some convincing key decision-makers of a business, however continuous testing when implemented correctly, undeniably leads to faster delivery and time-to-market without any quality compromises. No pain, no gain!
Following on this, Girish tells us the only drawback that matters is that it is a costly and time-consuming exercise. Indeed, developers and testers need to invest in the right resources/tooling to cover various aspects of the product quality lifecycle, such as accessibility, security, compatibility (devices, browsers, etc.), and non-functional aspects.
Nathan, on the contrary, doesn’t see any particular drawbacks in considering quality in the deployment phase. For him, ‘any opportunity to talk about testing and implement better test automation is great!’
However, there could be some challenges, such as a shift in mindset for QA, as they would have to work closely with developers. Besides, the CI/CD process may not be set up for test automation, which could an additional overhead and become difficult due to organizational restraints.
Why should organizations use more QAOps practices?
Breaking down the silos between development and operations has always had positive results, Lewis tells us.
To him, more organizations should adopt QAOps practices as it is a new challenge, especially now when there is a lot of skills and experience where developers can help operations teams. Besides, this makes a change from the one-sided relationship that used to exist, with developers constantly asking operations to deploy their new feature.
For Aman, developers and IT Ops teams are not the only groups of people involved in creating a successful software product. The introduction of QAOps reinforces the crucial role of QA and emphasizes the importance of efficient, automated software testing and QA in delivering quality applications to reap the benefits as mentioned earlier.
According to Girish, if organizations value product experience across different channels they should adapt QAOps. Of course, as he points out, digital and IoT are driving a lot of transformation but the key is not to forget about QA, which is a culture change to the common processes/practices followed to deliver products.
Moreover, Ravi adds that it is completely dependent on the type of application businesses are developing and the kind of organization they are. It is very hard to adopt new changes and if it is something of a cultural thing, it is even more difficult.
Hence, this should be completely up to the organization to ask these difficult questions before they change their approach because; one, they need time, and two, they need patience and resilience to adapt to the changes.
Adopting QAOps practices just because everyone is doing so is a bad approach. They need to ask themselves if they do really need it and then make a decision based on the answer to this question.
Nathan thinks that if organizations have adopted DevOps, then QAOps should be inherited. If it isn’t, it’s a great opportunity for organizations to focus on their QA operations and how quality can be improved as part of the CI/CD process
What is the future of QAOps?
Lewis points out that this trend has been happening for years within tech teams and that, in the future, we will only see more of these roles.
Ultimately, he continues, the responsibility of QA is a team problem: the more we can integrate that into all parts of the software process we have succeeded in our jobs. You should always be looking to improve the quality around you, to finally be able to remove the need to have QAOps and just call it Ops again.
Nathan also thinks that QAOps is only a word that will help conversations about quality to get started for those that need it.
‘“QAOps” is just a part of shifting testing in all directions, there is so much more that testers can do for their organizations and teams.’
Aman believes that QAOps doesn’t require a complete cultural change like DevOps does. A QAOps framework can exist without DevOps and be used as an aid for developers and testers to mutually collaborate in an agile setting, thus achieving faster results and testing new features more thoroughly before release.
With no official definition of QAOps yet, she says that a level of QAOps has almost already been achieved in many businesses and she thinks that it will continue to be expanded and improved upon as QA continues to drive digital transformations within testing.
For Girish, QA practices have evolved over the years to adapt to ever-changing ways of product delivery so QAOps isn’t going anywhere. On the contrary, it will only evolve further especially if businesses start investing in AI, RPA, and machine learning models.
On the other hand, Ravi believes that it will take some time before people will really fully understand QAOps and adopt it completely.
Special thanks to Girish Vasisht, Ravi Shankar, Nathan Ballantyne, Aman Dhudwarr, and Lewis Prescott for their insights on the topic!