Head of Test at YouView, Riel Carol, explains how good quality code is shipped to all required automated tests and frameworks
Carol’s main responsibility at YouView is to ensure he ships good quality code regularly to the firm’s 3 million set-top boxes (STB). In order to achieve this, he heads up a “good-sized team” of around 30 people, while looking at all testing phases and developments of the required automated tests and frameworks.
He believes that the code used for testing is as important as the one built to put into customers STBs. In essence, he ensures the firm delivers quality code as well as maintains delivery cadence.
“In order to be able to respond to day-to-day demands of the job, you need to have a strong technical understanding of the technologies used and the ability to juggle many projects at the same time,” said Carol.
“You also need to have good people management skills in order to develop the team, and above all, a long-term vision that will guide you. In YouView’s case, we are trying to automate as much testing as possible and reduce delivery cycles while aiming to get to a continuous testing and delivery model.”
Developing new tools
The hybrid television platform is continuously trying to develop new tools, using a mixture of tools that are built either internally or with the help of partners.
Carol continued: “When we have got a need for automation, we will look at what we can do ourselves, how much sense it makes to do it ourselves, what is available out there, and how it fits in our continuous pipeline.
“We aim to make our tools usable for as many people as possible and embed the automation and maintenance of test cases inside agile or project teams. The teams then run the tests on a per-commit or nightly basis, giving us quick feedback on the code and test quality. This avoids tests being out of sync or being broken.”
YouView uses a lot of testing on its live environment, which, according to Carol, is very expensive to create; especially an environment replicating the live TV service. The firm uses specialised test environments for limited testing that they maintain themselves, which is not something that requires a lot of effort, as the cost is more of a setup cost than an operating cost.
Functional and non-functional requirements
Carol revealed: “At YouView we aim to test as soon and as automated as possible and the test team gets involved in development at the same time as the developers. From the moment the story gets placed into the product backlog, the tester assigned to the agile team sizes it for the work required in testing and starts working on it once it is in its sprint.
“The tester will then write test cases to test the story and starts validating the development by testing early on the development branch, even before the code is finished. This way we can assess functional and non-functional requirements with early exposure to the product owners. There are then a few iterations of testing on the branch until the code is ready to merge. At this point, more formal testing takes place.
“Automated tests then run against the branch before merging, and if the results are positive, are merged with the code. This then runs with all the other components through our overnight and weekly functional and non-functional tests before it can be deemed good enough to go to different levels of trials.”
In summary, manual testers are responsible for creating all the required test cases and test materials, as well as performing the first iterations of testing. Once the code is ready, YouView puts it in their continuous pipeline to ensure that integration is working.
YouView testers follow different software development lifecycle methodologies depending on a products/projects needs and requirements. As a result, they have implemented full agile for some of its deliverables, with an agile methodology used by several agile teams (cross-functional teams), continuously monitoring processes and trying to improve them. They began this transformation two years ago to ensure they have plenty of optimisations.
On the DevOps side, the firm has implemented where it made sense, especially in parts of systems that are cloud-based. According to Carol, the DevOps model is not suitable for parts of its system, particularly where SDLC is not controlled.
He found that the biggest of benefits implementing DevOps and agile have been to allow delivery functionality in a faster and flexible capacity, bringing innovation and value to the firm’s customers.
Carol added: “At the beginning, there was waterfall, a bit like a production chain, where each team does its job and then hands over to the next team, so you have a big design phase before the development phase and testing phase.
DevOps and agile
“Then came agile, with the idea of making the whole process shorter, instead of trying to imagine the product in all its detail in perfection from the design. The agile process is much more iterative and focuses on a short delivery cadence of code to the customer by an agile team made of all the required functional roles, with the aim of getting value and feedback of the customer as quickly and often as possible.
“Finally, with DevOps, you push the agile model further by decoupling all deliverables and reducing their size, helping deliver very small units of value as soon as they are ready, where each member of the team is able to perform all the different functions and you reach a place where changes are continuously happening and with very quick quality and feedback loops.”
Carol believes there are many aspects that make working in agile way more pleasant. For him, the ability to see results quickly and get the customer to give feedback is great. Agile also allows him to try new things knowing that if they don’t get a good response he is able to correct it quickly.
“The key agile principles that I always have in mind (and I am pretty sure they are in the agile manifesto) is to release small and often, to give value to your customer, and to make changes as part of your process,” he added.
Automating manual tests
The test methods he is most familiar with is functional and non-functional testing methods, as he has more personal experience within this domain. He finds non-functional testing, and performance testing, a very interesting area.
“It is complex to get right as a lot of work goes into trying to reproduce real conditions, then you need to have the tools to load the system and finally a good set of performance monitors,” he revealed.
“Once you have all of this in place, then you need to be able to understand where the bottlenecks are the causes of problems. Some of the issues that are found in load and performance testing can be very difficult to track and fix, which he finds very interesting.”
Nowadays, automation is the basis of any test strategy, admitted Carol. According to him, it is extremely difficult, if not impossible, to implement models like agile and DevOps without automation.
He also noted that the challenge in automating manual tests is that manual testers tend to operate at a fully integrated level, where test automation is the most expensive. So, to be more efficient, testers should look at what is tested at that level and try to replace those tests with lower level tests while keeping some forms of high-level tests which can be automated.
Written by Leah Alger