Acceptance Tests
Introduction
Unit tests confirm that we have built the software right. Acceptance tests confirm that we have built the right software.
Acceptance testing aims to determine whether or not the system meets the requirements and specifications of the stakeholders. These tests can take many forms, with the main goal being to ensure that the software is ready for release. All acceptance tests should be written from the perspective of the end user and treat the system as a black box. They should only test the functionality that the user will interact with and should not test the internal workings of the system.
Just like with other types of testing, the exact definition of acceptance testing can vary depending on the organization or context. In some cases, acceptance testing is performed by the end users themselves, while in other cases, it is performed by a separate team of testers. In some cases, acceptance testing is performed manually, while in other cases, it is automated. Be sure to check with your organization to understand how acceptance testing is defined and performed in your context and use the terminology that is most appropriate for your situation.
Manual System Testing
During manual system testing, quality assurance (QA) engineers test the software as a whole to ensure that it meets the requirements and is stable. This type of testing is typically performed after all automated testing and before the software is released to the end users. Manual system testing can be time-consuming, but it is an important part of the testing process.
In many organizations, manual system testing is performed by a dedicated team of QA engineers. These engineers are typically responsible for creating test plans, executing test cases, and reporting bugs to the development team.
As organizations adopt continuous integration and continuous deployment practices, manual system testing is becoming less common. Instead, organizations are relying more on automated testing to ensure that the software is stable and meets the requirements. This moved the burden of testing onto the development team, who are responsible for writing and maintaining the automated tests.
Regression testing ensures that new code changes do not break existing functionality. It is important to perform regression testing whenever new code is added to the system to ensure that the system continues to function as expected.
Relying on manual regression testing is a terrible mistake. It is time-consuming, error-prone, and inefficient. As systems grow in complexity, it becomes increasingly difficult to manually test all possible paths through the system. Automated regression testing is the way to go.
End-to-End Testing
End-to-end testing is a type of testing that tests the entire system from start to finish. It is a black-box testing technique that tests the system as a whole, rather than individual components. End-to-end testing is typically performed after all other types of testing have been completed and is used to ensure that the system is working as expected.
These tests typically run in some sort of staging environment where they can interact with real data and services. They can give us a lot of confidence that the system is working as expected and that it is ready for release, but are difficult to write and maintain.
Using end-to-end tests as the only form of acceptance testing is not recommended. They are slow, brittle, and expensive. As other teams deploy their code to the staging environment, the tests can become flaky and unreliable, reducing the confidence that they provide. Instead, use them selectively as a final check to ensure that the system is working as expected before it is released to the end users, or forgo them entirely in favor of other forms of acceptance testing.
Contract Testing
Contract testing is one of the most useful forms of acceptance testing. Sometimes called API testing, contract testing is a form of automated acceptance testing that seeks to valid the public behavior of your system as a whole. Just like with end-to-end testing, contract testing treats the system as a black box and only tests the functionality that the user will interact with. The big difference is that contract tests are much faster and more reliable than end-to-end tests.
They accomplish this by isolating the system from its dependencies. Instead of running the entire system in a staging environment, contract tests run against a mock server that simulates the behavior of the system's dependencies. This allows the tests to run much faster and be much more reliable than end-to-end tests.
The responses from the mock server are defined in a contract file that is shared between the consumer and the provider. This contract file specifies the expected request and response formats for each API endpoint. The consumer uses the contract file to generate tests that validate the provider's behavior, while the provider uses the contract file to generate tests that validate the consumer's behavior.
If the contract file changes, the tests will fail, alerting the teams that they need to update their code. This ensures that the teams are always in sync and that the system is working as expected.
User Acceptance Testing (UAT)
User acceptance testing (UAT) is a type of testing that is performed by the end users of the system. Commonly called beta testing, UAT is used to ensure that the system meets the requirements and is ready for release. UAT is typically performed in a sandbox environment that is separate from the production environment, allowing the end users to test the system without affecting the live system.
Conclusion
Acceptance testing is an important part of the testing process that ensures that the software meets the requirements and is ready for release. It provides a level of confidence that no other form of testing can provide and is an essential part of the testing process. Be sure to check with your organization to understand how acceptance testing is defined and performed in your context and use the terminology that is most appropriate for your situation.