What is Regression Testing and How it Affects the Company’s Bottom-Line?

There’s a funny joke about how when you fix one bug in your application, you end up creating two more. The fixes we make in response to bugs can cause other things to break.

So how do we make sure that our fixes don’t end up causing more problems? How do we make sure that bugs that we have already fixed don’t come back to haunt us?

It’s easy to find a problem when you’re actively looking for it. What’s harder is to catch all the problems that you aren’t looking for.

That’s where regression testing can come in handy.

What is Regression Testing?

Regression testing is a method of testing your code in a way that you can confirm that your recent code changes don’t break your existing features.

This testing is done in order to make sure that any of your new code changes don’t have side effects. For example, say that you make a fix for your login form where the submit button was not working. With regression testing, you would make sure that your fix for the submit button did not accidentally break something else in the application.

So ultimately, its purpose is to catch bugs that may have been accidentally introduced in a new build.

The Business Value of Regression Testing

Developers have their own technical reasons for using regression testing, however, there is business value in incorporating regression testing into your process.

First of all, engineering hours are expensive. Not only money-wise but time-wise and energy-wise as well. It takes a lot of time, energy and money into engineering to make a high-quality product. If developers are spending time having to revisit old bugs over and over again, it’s not the best use of resources.

Next, you’re going to ensure product quality stays high. The worst thing that can happen is telling a client that something is fixed just for another feature to break. It looks bad and the more it happens, the perception of product quality drops and may never recover. So by catching issues before they get out, you make sure that you build a solid product that clients can rely on.

Finally, it allows developers to be confident in their work. It removes the constant lingering stress of ensuring that your fix didn’t break something else. This leads to happier and more productive developers, helping to retain talent for the company.

Performing Regression Testing

One of the challenges of regression testing is that these tests can be manually handled on smaller projects, however, as the project grows, repeating a large suite of tests for each update can be time-consuming, so typically an automated testing tool is required (like Cypress for Javascript).

Every time that you change your code and your app undergoes significant change, you’ll want to follow these steps:

1. Determine what kind of changes you have made to your app.
2. Based on the changes made, what major features of your app will be affected by each change?
3. Depending on how many features will be affected by the changes, determine if you are going to need to do a full, partial or unit regression.
4. Schedule a time to run the tests since they can be time-consuming. It’s best to do it when you may have to go do something else and you can let them run in the background.

This process makes sure that you are performing your regression testing in an efficient manner. Since the number of tests can balloon as a project grows, you won’t want to run the entire test suite.

The Types of Regression Testing

Step 3 talks about determining if you need a full, partial or unit regression. Let’s look more into what each one means:

• Full regression — This should be done when your changes are touching multiple features on your application. It will be the most time-consuming, however, it will check the product as a whole and make sure nothing has broken.
• Partial regression — This should be done when your changes touch only one feature on your application. You can run your tests much quicker since you do not have to run the entire suite.
• Unit regression — This should be done when your code touches a unit of code. A unit of code generally represents a function. So essentially, you are making sure that your code changes do not break the expected behavior of the unit/function.

Regression Testing Techniques

One of the biggest challenges is how resource-intensive it can be to run the tests. So there are techniques to help guide you to make sure that you run the correct set of tests.

1. Retest Everything

In this scenario, you run the entire regression test suite. It will take time, however, it will make sure that everything is working as expected.

This should only be done if you’ve made a fundamental change in your code and you really want to make sure that your product behaves as expected in many different areas.

2. Carefully Select Your Tests

In this scenario, you will select the tests that you want to run. By manually selecting which tests to run, you can significantly cut down on the number of tests that you have to run, ultimately taking less time.

You cannot be sure that the entire product works as expected since you have only selected a subset of tests, however, if you pick your subset well, this is a great way to speed up testing cycles.

3. Prioritize Your Test Cases

In this scenario, you run the tests with the highest priority first. What determines a test as a higher priority can be based on multiple factors like how critical a feature is to the company, its impact on the product and the company, and how much a feature is used.

This makes sure that you didn’t break something that is critical to the business. Breaking something that’s a low priority is not as bad as breaking something that is a high priority.

4. Hybrid Approach

In this scenario, you both select a list of tests to run and then run them in a prioritized fashion.

By using these techniques, you can make sure that you’re not being inefficient with your testing as it can take time.

Regression with Agile

You can incorporate a regression testing schedule into your Agile processes. Since products are generally built in two to four-week sprints, you can perform regression testing at certain times to make sure that releases go smooth.

In order to allow for some margin in case you do find errors elsewhere in the application, make sure to run an Agile regression check a day or two before the sprint is going to end. This gives time to respond to new bugs that have been created before they reach production.

These Agile regression “checks” can be put into one of these two categories:

1. Sprint Level Regression — Whenever new functionality or enhancements are done within a sprint, select the appropriate tests near the end of the sprint and run them. Make sure that the changes made over the sprint have not negatively impacted something else.
2. End-to-End Regression — You simply run the entire test suite to make sure that the entire product is working as expected.

Conclusion

It’s important to keep in mind that even the smallest of changes in our code can lead to ripple effects outwards in surprising and negative ways. A small one-line change could potentially lead to it breaking a major feature elsewhere on the application.

Regression testing helps with catching these newly created bugs and helps developers from eradicating them before they reach production.

Not only is there value for developers to adopt regression testing, but also many valid business reasons as well. It helps to make the best use of engineering time, ensures high product quality and can help to retain talent as it can reduce developer stress.

Now that you know what regression testing is, the next step would be to choose the appropriate automated testing tool that fits your needs. Generally, you can just Google “automated regression testing tool for [programming language/framework]” and you will find them.

Hope that you are able to implement regression testing in your projects and make sure that the bugs you’ve killed before stay dead!