Test Automation: Here's Why its the Future of Quality

by Dov Rosenberg – 

Our future is filled with almost unimaginable forms of new and improved forms of transportation. 

The reality is that we are on the road to autonomy- with fully automated cars, trucks, trains, and possibly airplanes that will have the task of transporting us instead of us driving or piloting them. It will not only be driving-assisted technology but a completely capable self-driving vehicle that is in full control. 

I would love to be a passenger in one of these cars and be able to see all the capabilities and high-tech features. This advancement in technology disrupts everything that we are used to and know about cars. Fully autonomous cars are the future. The software and automation within these cars will have the ability to identify safety risks and handle the task of driving under all circumstances with routes that are updated in real time according to traffic patterns and delays. 

There will be so many new capabilities that will come with a fully autonomous car and it is hard to grasp the concept that this will be just another technological advancement to our current lifestyle. Overall, the modern software within the delivery processes are extremely complex and include a lot of moving parts. The software to create these cars already exists, but in the future, developers need to create more precise maps and capabilities to fit within the small computers in these cars for a more reliable and effective car. But how is this goal achieved? 

Consumers want the best functionality, operational quality, and user experience when we buy a product or service. But, as a developer when it comes to speeding the delivery without sacrificing the quality while also reducing overall costs of delivery, there are so many challenges when it comes to meeting those requirements. 

Ensuring Quality with Test Automation 

Daily, we use products and services that are composed of different applications that make them run properly and ensure the highest quality. Even before we make those purchases, everything is quality checked. But – how does the company guarantee that those manual tests are effective and efficient overall?  

I think a good answer to change the perception of how effective and efficient testing actually is when going forward is test automation.  

Test automation provides the framework and methodology that maximizes the use of automated testing tools and validates complex applications.  It involves selecting tools, designing and writing detailed test scripts, automating those scripts using automated tools, executing the automated scripts, and reporting the results. These tests are run throughout the agile development lifecycle and as part of an operational readiness check before the software is in the final production. Automated tests can also be run in a production environment as part of a continuous monitoring program. 

In general, there is a fine line in knowing what to automate and what NOT to automate. Even if we stray away from the example of autonomous cars, think about all the products and services that we use daily – all of those are made up of a variety of components and applications that were quality tested before production. As a company that produces software solutions, it makes sense to have components testing continuously rather than manually or occasionally so that bugs can be caught and fixed rather than having to fix code for the application after it is in production.  

In addition, one of the main and most significant reasons to automate testing is repeatability! Automation not only saves time and money but repeatedly running the tests consistently checks the processes and systems to ensure that the latest updates function and perform as expected. It is extremely important to remember that software testing is crucial prior to production releases because if the application does not function as needed or expected, it affects the user experience. 

Automated Regression Testing 

Next, it is important to consider what kind of testing should be put into action when implementing autonomous testing.  

In my opinion, regression testing is one of the best options. Regression testing is the process of making sure that functionality previously tested and implemented is not negatively impacted by the changes currently being tested.. 

When it comes to regression testing, there are two main approaches:  

1. 1. Include every new feature into the regression suite  
   2. Limit the scope to that which represents the user path and priority transactions through your system. 

Wouldn’t you think that the latter is the better option because eventually, your regression test suite becomes too big that it becomes unreliable and fragile? When the regression suite is rock solid and actively maintained – it ends up being a much better investment than an enormous regression that never works when needed. 

So, let’s look at regression testing and functionality testing – the main difference would be that regression testing provides the answer to “IF” this new functionality should be included or not in an upgrade rather than if the upgrade works.  

If we think about regression testing in relation to an autonomous car- an example would be adding a new functionality or button. Once this button/function is added to the dashboard or application, the new addition would be tested to see if everything is working properly. Therefore, with regression testing, it becomes known whether a feature affects the functionality of other components in the dashboard. The new feature, as well as all the initial ones, are tested together to make sure that there aren’t any unforeseen issues that arise from implementing new functionality. 

Testing & Quality Assurance

A big question that comes to mind about test automation is, “Do we need to automate every possible scenario?” The answer is that not every scenario will need to be tested, but rather, specific tests can run when needed or wanted.

This is when it would make more sense for the QA team to work with the development team to set up unit tests within the code so the developers could run them at any time. The benefit of building and maintaining code correctly suited to the application and regression testing is to be certain tests are passed BEFORE the code is moved to the next step in production.

Within my years of experience, I have noticed that the actual coding of the application is the most straight-forward part of the application lifecycle, but the primary causes of poor software quality stem from poor or ambiguous requirements. This challenge stems from the fact that people struggle to document everything that they know.  So when it comes to writing software requirements, it’s difficult to get the right level of detail while not expanding the scope beyond the capacity of the defined duration. User stories need to be able to be coded and tested during a sprint (usually 2 or 3 weeks). 

If the user story is well defined – it should include everything the developers AND testers need to complete the test story, which eliminates any grey areas or confusion. Therefore, the sprint is the WRONG place to do story development. 

In addition to knowing when to run the tests, I have noticed that most of the time the QA group is the only group of people that are able to run tests at all. As a result, the developers push their code and applications that are half-baked rather than making sure everything is working properly first. In that case, the QA team finds themselves having to make sure that every new functionality is tested during every sprint. At the end of the day, it causes you to question how much test automation is enough? When all the components of the code are done the right way, you will know what needs to be tested at any point that you want.

It is also extremely important to ensure that the test data represents valid and positive test cases as well as being updated as new functionalities are added. The value of building a stable test automation library ensures the process includes the appropriate data to go along with those tests as well as ensures the data represents all components through the cases.

So, when there is only a specific group of people that can run the tests, the user story development is affected because the focus tends to be on the “just in time” user stories. In reality, it takes just as long to write a quality user story as it does to code and test it. Writing each user story in isolation is usually inefficient overall because it leads to defects due to the incompatibility between stories.

So, in an ideal world, any developer could pick up any user story and create the necessary code to implement the user story without needing any additional input.

The Value of Test Automation

As stated previously, test automation provides stability and ensures quality. But what are the other valuable components of it? 

• • Reduces time to market through automation of tests that require long setups or large amounts of manual input

• • Validates and optimizes an applications’ accuracy and performance 

• • Provides repeatability of test scenarios in every product release

• • Allows for increased depth and scope of tests 
  • Catches problems for developers and testers before the end users find them and provides the ability to gather metrics that produce the baseline for quality improvement

A user story represents a single point of view on a process, for example, a customer wants to be able to do X- but a typical feature has multiple points of view that need to be implemented to represent the WHOLE picture. This causes confusion on what needs to be done or what needs to be eliminated so that the stories are compatible at the end. Companies that spend the time to improve the software development practice overall not only improve the code quality but also have a consistent process in which the QA team validates that the code is ready for production.

This provides the next step of knowing what to do when a bug is found, simply make sure nothing additional arises to cause another problem. When there is an initial plan and the test strategy is clear, you can help prevent unexpected issues from damaging aspects within your company’s products or bottom line. 

The QA team will have better outcomes if the inputs are at their best. To get the most out of test automation, however, requires some strategic thinking of how to build a cost-effective automation strategy that fulfills the promises that the tool vendors continuously tout.  Having good code, well-defined user stories, efficient processes, and stable test automation allow a company to reliably accelerate the speed of production without sacrificing overall quality.