Acceptance testing

Acceptance testing checks the application as a whole and ensures proper operation. This type of testing performs verification of the program outcome against an expected outcome and is more of a black box type testing. Hence, acceptance testing scripts are readable documentation of a software’s stable behaviour.

If unit testing verifies that the code does exactly what the programmer expects it to do, then acceptance testing verifies that the code does what the user expects it to do.

- D. Sale: Testing Python: Applying Unit Testing, TDD, BDD and Acceptance Testing. 2014.

Tools for writing acceptance tests

Following is a set of tool recommendations for Python projects:

• PyTest
• Robot
• Lettuce
• Custom-made?

Nevertheless, a custom-made acceptance testing framework can be a promising way. For instance, a simple testing framework in python can be easily programmed where one can execute the application (system under test) with os.system() and compare the outcome with the benchmarked outcome. For git users, you can also integrate python-gitlab for an automated issue creation in GitLab issue board.

Hands-on exercise

We continue with the Matrix Calculator project to implement sample acceptance tests.

Write your first acceptance test

Perform the following tasks and observe the code coverage after each step:

1. (Demonstration) Write an acceptance test to check the add routine in main.py
2. Write an acceptance test to check the solve routine in main.py
3. Write an application-death-test to check the “default” case