How test automation can avoid 70,000 angry passengers

There is a growing need for test automation platforms in the transportation industry and companies should turn their manual testing processes into modern ones.

The following news is the worst nightmare for QA managers, especially ones working in the airline industry.

Around a year ago, a leading software system supplier to British Airways, Air France, and Lufthansa experienced an awful glitch in their system (The Telegraph, September 28, 2017). Their check-in system was down for 15 minutes during the holiday season and experienced a power failure to top it off. This created absolute mayhem. Thousands of customers filed complaints, six hundred flights were cancelled and over 70,000 angry passengers were created.

However, the software glitch didn’t just create upset customers – this failure ultimately led to an international news frenzy. The airline brands were negatively affected and lead to a loss in profit.

There’s no question of how critical customer loyalty is these days. Throughout the past decade there have been many instances where a small software glitch has turned a company upside down – cases where one small software bug creates one big fiasco. So, how can big airline companies avoid this?

Competitive edge – continuous delivery

Nowadays, a bumpy digital interaction is a deal breaker for both the business and the customer. Brand loyalty can weaken in an instant when a customer becomes unsatisfied. Marketplace competition is much stronger, so there is a greater chance these customers will move on to a competitor.

If I enter Expedia to purchase a plane ticket, for example, and experience even a minor bug, I would then instantly move to Skyscanner or any other comparison website to move forward with my purchase. This not only applies to travel-fair websites, but to all transportation companies where competition relies not only on product quality, but on customer service and user experience.

The digital revolution, thus, turns customers into omni-powered beings who expect nothing less than a seamless and effortless experience. Software development, inevitably, moves hand-in- hand with a customer’s expectation, and the industry has seen huge developments towards a smart and automated environment that provides continuous delivery.

Nonetheless, software testing is still heavily manual and is lagging behind the expeditious agile dev process. Previous QA efforts are no longer effective in a fast-paced industry, which brings us to a new era. The era of the modern manual automation tester in the agile world.

When did agile break through?

Companies have started to adopt agile through the dev process over the past decade. However, the agile framework has picked up momentum over the past few years. The motivations from shifting from waterfall to agile were pretty clear from the get-go: bring software faster to the market by dividing the production process into short stacks, thus, increasing both the level of software quality and customer satisfaction in organisations.

Is agile the norm today?

Definitely yes. A recent HPE survey shows that 67% of companies are either pure agile or leaning towards it.

With agile taking over the software development process, many challenges rose to the surface that companies were not yet aware of. Teams that had a lack of communication typically failed to pursue deadlines and deliveries for new features. Plus, the teams weren’t appropriately equipped with the right tools.

It is important for any company or team to have tools made especially for agile delivery. Agile-based tools may include types for continuous integration that allow teams to continuously develop and deploy robust test code. Still early on in agile, all the focus was on the dev process and the tools behind it, while the testing part was not dealt with.

Fortunately, the DevOps movement was introduced to better align and integrate QA with every aspect of the development process. I won’t go deep into it, however, it is important to highlight the realisation that, from ten years ago to recent days, there has been a latent need for better integration of teams and automated processes.

Which brings us to question: why are companies still relying on old-fashioned manual testing, even 10 years after the introduction of DevOps and automated testing?

Software testing & development gaps

Regardless of major upgrades in the development process, testing was still left behind. The focus was on the dev team, the processes of the dev team, and the tools to enhance their productivity. QA, in contrast, could not catch up and testing soon became a bottleneck.

The challenges are ever growing – software release cycles are getting shorter and more frequent. From releasing once a year in the waterfall method, to even a few times a week now, means that regressions are being rapidly built up. Plus, the introduction of many different platforms multiplied by several test cases have created huge matrices for companies to cope with.

With all of that to consider, everybody would agree that test automation is a must.

Test automation: industry standard Selenium

The need for test automation led to the quick adoption of Selenium, which started simultaneously with agile and has become the de-facto standard in the industry. It caught momentum by being an open-source platform with a vibrant community, by having integrations with different APIs and plugins, supporting several devices and browsers, and other reasons most of us are already familiar with.

However, Selenium has some considerable hidden costs.

For example, the Selenium integrated development environment (IDE) stores element locators in each step of the test, so testers need to interact with each element every time the locator is defined in a different place – if it’s 20, that’s 20 locations. This produces serious maintainability issues because of the duplications involved. In addition to that, Selenium requires substantial technical expertise and the support of third-party tools to get completely functional. Plus, manual testers are not able to easily familiarise themselves with the tool as it requires knowledge in coding.

Bringing the manual tester into DevOps

Manual testers bring significant value to the company – they deeply understand business processes and the applications that they test. In many cases, they come from testing related disciplines, so they know how to structure testing and what to test.

In fact, the testing world still heavily depends on manual testing – 70% of testing is still done manually, according to Capgemini.

If manual testers are already present in the business, then it might be easier to keep the team, rather than hiring a whole new team of test engineers. Manual testers bring significant value, but manual testing as the core practice is inefficient. Of course, all companies should find a way to automate their tests and to work at a continuous and fast pace.

For a transportation company, time effectiveness is extremely important. This means that every second a software application is delayed, it’s a second longer to take one item (or person) from X to Y, and a second later for check-in or worse, another customer lost to the competition.

Transportation companies, such as airlines, online ticket resellers, deliveries, etc., should look for ways to bridge the gap between the clear need for test automation, and to support digital transformation and the value brought by manual testers. The way to do it is to find a way for manual testers to start creating test automation, without having to learn how to code, thus being connected with the DevOps and agile methodology.

Automated tests should be intended to serve manual testers, therefore should be easy to create and, even more importantly, easy to maintain. With this in mind, QA managers should look for codeless test automation that will enable manual testers to turn their manual test scenarios into automated ones, without coding knowledge.

Even further, managers can choose codeless Selenium test automation platforms, that will enable their team to enjoy the advantages of Selenium without the hidden costs. Codeless platforms often have a mechanism to reuse test elements or tests so that when a change is made, it will be automatically applied to all other tests that use the same element.

Additionally, you can choose platforms that use artificial intelligence or machine learning to automatically overcome changes in the app and adapt the tests accordingly.

This will allow a smooth transition to continuous testing (CT). The need for CT is quite clear when it comes to the transportation industry. This industry can have massive financial losses just from single software bug instances. If your team correctly bridges this gap it can lead to fewer glitches and bugs, which in turn leads to less money lost, no negative media coverage and an edge over the competition.

Manual training for test automation

The next step to achieving a more efficient QA automation operation is to better train your team. The training implementation has been far too tool-centric. Testers are often incorrectly taught how to use a specific automation tool rather than learning about its underlying features and patterns.

Testing experts believe the right automation training should be focused on how to build an automation strategy and how to manage certain expectations around automation. Previously, QA managers and testers were stuck with legacy tools and had to adapt themselves to them.

Now, tools such as codeless test automation platforms are created and adapted for the tester. Another focus of efficient automation training is to better understand when and what to automate:

  • When: automation should be done at the beginning of the dev lifecycle. It is paramount to test early and to test often. Part of CT is to allow testers to begin creating tests early in the release cycle (‘shift left’) – an issue already addressed by some of the new-age automation tools
  • What: when it comes to which tests to automate you will gain a lot of benefit from automating repetitive tests, multi-data sets, functionality tests, etc., but you definitely don’t want to automate everything or to reach 100% test coverage. Teams should have all of this in mind when training and learning to be successful automation testers.

Comparing manual & automated testing

Testing execution at scale is much faster when done by a machine. Software development is a repetitive process by nature, thus being able to execute new test code alongside regression is an integral part of the release criteria.

Performing such activities manually is completely inefficient and hinders the entire release process. In fact, most cases of tech disasters such as the one described in this article are consequences of unidentified software bugs and inadequate QA.

Can companies really blame manual testers for it, or should they take it as a lesson and introduce tools to automate and improve the reliability of the process?

Manual testing alone is, by definition, error-prone and indeterministic compared to consistent machine-based test automation. Companies need the ability to consistently get the same results when the expected criteria are met (or not) to release high-quality products. Such reliability at scale, as the product grows, is a critical enabler for continuous testing.

Most organisations of the digital age are exposed to fast market releases of new operating systems, devices, gadgets and more. Therefore, they are expected to readjust their product and try new features and operations at the same pace. Fortunately, companies that work with agile and DevOps have this mentality as a core practice. Dealing with these changes in an efficient way is only possible by having a robust test automation foundation that can be adjusted and triggered on-demand as changes occur.

The testing framework and network for transportation companies can be a bit more complex than any other industry’s network. A transportation company’s network needs to be adaptable and able to perform multiple different functions like monitoring vehicles and cargo, finalising delivery payments, checking supplier operations and last-mile deliveries, tickets purchase, check-ins, delays notifications, reminders, etc.

Those functions should always be available, otherwise there may be consequences throughout the entire supply chain.

Audit trail and evidence

Test automation leaves trails and can be retained for as long as needed, while manual testing is by nature harder to record and store. Specifically, having the test data for ad-hoc audits and compliance processes is a clear business requirement for verticals like transportation, logistics, and airlines.

The case for having test automation platforms as key enablers for the entire DevOps and business’ success was proven years ago and, in reality, this case has grown in significance due to advancing technologies.

It is now all about expanding test automation capabilities through best practices, increased value of the QA tester, robust test labs, and high- value test frameworks to support the digital changes.

Nevertheless, it is not sufficient to simply have a great test automation tool. QA managers also need to push themselves more thoroughly to cope with testing developments. DevOps methodology is a powerful enabler for testing teams to be integrated with the development process.

The benefits of DevOps and test automation are clear, but how should the modern manual tester change to fit with them?

Shaping QA managers & their team

To enable DevOps, teams are focusing on rapidly delivering high value and quality. This translates into efficient continuous integration (CI) that builds new software, and continuous testing (CT) that validates its quality in each step of the development process.

Only by having reliable CI and CT, teams can effectively release software as part of their continuous delivery/deployment (CD). The QA manager’s role in assuring the fluent process of these three pillars is fundamental. QA management should have on-demand data in each stage starting from design, implementation, build acceptance, integration testing, and production testing.

Continuous test measurement

DevOps and software development are continuous processes. Innovation does not stop at the end of a release, nor does test management and measurement. Hence, QA managers need to define metrics to help them measure efficiency from one to the other. Such metrics would include:

  • efficiency of test scenarios
  • pass/fail rate between cycles
  • amounts of defects on the defect backlog
  • progress of the CI cycle
  • test coverage alignment with market and business requirements & product analytics/usage
  • ratio of test automation/manual.

Team productivity & skill sets

Continuous improvement in the software quality process (as mentioned earlier) is great, but we must not forget that the process is led by people. The team must continue to sharpen their skills, to learn new tools, techniques, and to measure their efficiency against the management objectives.

For example, if issues like a test stability problem due to flakiness, or too many escaped defects to production appear, it is time for the management and the team to come together and build a plan to evaluate what needs to happen to overcome these issues.

Fortunately, the world of testing is evolving fast, with new open-source frameworks, machine learning, and deep-learning based tools. Such innovations open doors for practitioners and managers to improve and develop to a more mature state.

QA managers can be the steering wheel for a mature DevOps practice. It involves leadership, cross-team communication, measurements and improvement based on metrics, and a matching set of technologies to support their agenda.

If, in the past, QA was about how many tests one can develop in a given time, these days the reality is not about amounts of tests, rather about the reliability, value, stability, and the speed at which tests can run.


The transportation industry is going through a massive digital transformation. Customers are expecting nothing less than complete digitisation from businesses, hence, if a company wants to survive or even beat the competition, its internal processes have to keep up.

Experience has become pivotal in customers’ journeys through brands, meaning if interactions are not smooth and fluid, they will immediately shift to a competitor. The software industry has come a long way since the introduction of waterfall.

We have seen major changes and new practices in software development, but still, QA testing as a manual practice is still heavily used. Modern manual testing, as a combination of test automation tools and an integrated team, are the keys to keeping companies en route on their digital roadmap.

The implementation of test automation tools serves as a catalyst to bring testers into the DevOps era – an era where we will no longer see worldwide chaos in airports caused by system failures. The question here is not why corporations should work with automated software testing platforms, but how soon will all of them use them?

Yarin Podoloer, co-founder & chief product officer, Testcraft

Related Posts