What are automated tests & why do we use them?
What are automated tests?
The words “automated tests” are commonly used in the software engineering industry, but frequently with different context and understanding. This blog post is referring to Unit, Feature and Integration tests which are one of the primary methods of testing that are used at Evoluted. Automated tests are code written "outside" of the functionality of a site, and run small subsections of site code to check it is behaving as expected.
An example of a feature test might be a test which calls the site code responsible for returning a specific page for a URL, and checks that the content returned is correct for that page.
What Are the Benefits of Automated Tests?
Earlier Bug Detection
Automated tests are excellent at catching bugs early, and increase the robustness of the site. They provide additional assurance that your site code will run correctly and consistently. The number of bugs that make it to the live site is drastically reduced, and bugs tend to be much easier to locate and resolve when they do occur.
Example
When requesting a URL the response includes a “status code” which indicates if it has been successful or not. A simple Feature test would be running the code responsible for rendering the homepage and checking for the success “status code”. The addition of this test would greatly reduce the chances that the site will be deployed with a homepage that is inaccessible.
Improved Internal Documentation
The tests act as live documentation for engineers who are maintaining the site, outlining and enforcing feature requirements which may not be obvious when looking at the site code later down the line. Particularly when a site has been running for a while or has multiple engineers working on it, automated tests as documentation and acceptance criteria enforcement reduce feature regression. This, in turn, provides engineers with confidence when writing new features and deploying.
Example
JavaScript functionality will often depend on HTML elements having specific identifiers. In a HTML template it is not always evident what these identifiers are used for, and it is relatively easy for them to be removed accidentally (potentially impacting or breaking JavaScript functionality). Having JavaScript automated tests increases the likelihood that if a functionality dependent ID is removed that the bug will be noticed before the code is deployed.
Greater Consistency
Writing automated tests for code also improves code consistency and quality, as the act of writing the tests can often highlight where behaviour is inconsistent or where possible use cases haven't been accounted for. Writing tests encourages engineers to write code in individually contained “blocks” which improves organisation and maintainability of the code base.
The Main Considerations of Automated Testing
Time & Resource
No matter what feature or task automated tests are being written for, it will increase development and maintenance time. This is because the tests need to be designed, written and debugged as code alongside the code which handles the site functionality.
Automated tests are purposefully run in an enclosed environment to effectively help engineers locate issues. A result of this is that automated tests will always run slightly differently to how your production site code runs (as production code by definition needs to be accessible and therefore is not in an enclosed environment).
Different Pages Have Different Testing Requirements
The more complex your site is, the more tests that are required to "cover" all of the different use cases, and the more likely aspects of the functionality will not be enforced by automated tests.
Example 1: A Static Homepage
Consider a homepage for a site which has only static (unchanging) content on it. A single automated test can be set up to check that the page is successfully returned and specific content is displayed, and that gives assurance that the homepage will be accessible to visitors to the site. As there is only a single permutation of the page, a single test is sufficient to cover all possible use cases.
Example 2: A Category Page with Faceted Navigation
Now consider a page with a search filter on, with multiple search options. An automated test could be set up to ensure that the page returns a successful response with no options provided, but to fully guarantee functionality would require multiple tests that check the filter is successful when provided with different combinations of options.
Replicating 'Authentic' User Data
One of the biggest challenges of automated testing is faking data; real data should never be used in tests, and hard-coded test data can be useful but covers a less dynamic range of possible situations.
Example
Consider a form with a single input field "name"
CASE 1: An automated test could use a hard coded value and every time it runs the test it will use the same value to check if the form is submitted successfully;
- name: 'Ellie Roberts'
CASE 2: Alternatively, the test data could be generated randomly each time the test is run
- name: 'Ellie Roberts', or 'Micheal Samson-Bloomingdale’, or 'Vi Lee‘, ...etc
The first case would ensure that the form can be submitted, but the second ensures that the form can be submitted with a greater variety of different data.
If there had accidentally been a character limit of 14 introduced to the name field the test with the hard coded value would never trigger the limit, but the test with dynamically generated data would fail the first time a longer name was generated.
The Balance of Automated Tests
Maintaining a test suite of automated tests is a balance, and trying to write automated tests which account for 100% of possible scenarios and amalgamations is not feasible or the goal. High-impact functionality and validation should be thoroughly tested, but some code is not necessary to test (for example boilerplate when using a framework) and for nonessential functionality the payoff must be weighed against the added time and cost.
Are Automated Tests Really Worth the Effort?
In my opinion an extensive test suite is well worth the time and effort.
Automated tests catch bugs early and reduces number of bugs which get to the live site
Automated tests act as live documentation and requirements enforcement, reducing feature regression
Having a robust test suite significantly increases engineer confidence
Writing automated tests improves code quality and consistency, and improves the maintainability of the codebase
Who Are Evoluted?
Evoluted is an award-winning, full-service digital agency with talented specialists. Powered by a team of highly talented software engineers with diverse skillsets, we offer a full suite of development services that help our clients to improve processes and maximise impact.
If you'd like some help with your testing, or anything else, we'd love to chat with you. Contact us today to learn more about how we can support your business.
