Automatic software tests play an important role in ensuring quality at every stage of development. […]
Any code change carries the risk of causing unwanted side effects that lead to states of error in the. In order to find such errors at an early stage (and not just at the customer), automated tests must ensure the previous, as well as new functionality of the software.
First, the project objectives (vision) and a rough roadmap should be defined. The necessary budget must not be neglected. As expected, the initial effort to develop fully automated tests at the beginning of the project is much higher than later during operation. However, a certain” background noise ” always remains, because test results have to be evaluated, test cases have to be adapted and new test functionalities have to be implemented.
After the procedures and processes of the test are clearly defined, the tools and frameworks are selected. Subsequently, the first prototypes were to be presented in a kick-off in order to create acceptance for the tools used.
After successful building of the (new), the test automation is automatically started with the software version to be tested. In the case of installable Software, this must be done without manual Intervention in the so-called “Silent mode”. By the way, most software installers offer a “response file”, which contains the corresponding dialogs and entries as a text file, which the user specified when installing it for the first time.
Subsequently, the software is automatically set up and parameterized, as well as the correct functioning of applications, services, browser, GUI, etc.tested. If the software product to be tested supports different platforms (Unix, Linux, Windows), all these operating systems must at best be tested in parallel and at night. Thus, cross-country tests can be carried out efficiently. The utilization of the hardware resources is better distributed and the test results are already available for further analysis in the morning.
If many tests run in parallel and on several different test systems, then the use of a virtual computer system of a “virtual machine” (VM) is recommended. If you don’t want to use your own hardware, cloud computing is an interesting alternative. Here, the IT resources are provided, which are also needed.
For the development of fully automated software tests, the Robot Framework is a clear favorite for me. Decisive factors include the extensive modular principle for expanding the range of functions, as well as the easy handling with good integration into existing projects. A further advantage is the simple, table-like structure of the test cases, which requires only a short training period.
The Robot Framework focuses on user tests (acceptance tests and end-to-end tests), but is also used for system tests and smoke tests. The Robot Framework developed in Python is available as open source software (Apache license). Currently (April 2021) version 4.0 can be downloaded on. The installation of the framework is uncomplicated, since only Python 3 and its package management “pip” is required.
Two established systems are available for development and test execution, but they can also be used together to enjoy the advantages of the respective environment:
As described above, all functions of the software should be tested automatically. The Robot Framework helps us with a variety of libraries and additional packages that can be easily integrated. An example is the very stable SSH library, which makes it possible to run all tests on Windows, Linux, Unix and Co. remotely and recyclable.
In the area of end-to-end user interface (UI) tests for web browsers, the new and modern browser Library from Playwright, or the Selenium, which has been established for many years, has been available since the beginning of 2021. These libraries can be conveniently integrated into the Robot Framework. Thus, checking graphically based functions such as mouse clicks, or user input and output, both on web pages and in the GUI is easy to implement.
Of course, self-written test scripts can also be easily integrated into the Robot Framework. It does not matter whether Java, Python or shell scripts are preferred, only the return values 0 or non-0 should be considered for a working reporting.
Extensive test reports are automatically generated by the Robot Framework as HTML pages. In addition, the test results can be imported into a test management system such as TestLink. In order to be able to analyze errors faster, an accurate error description with output of the log files is very helpful. For many tests, it also makes sense to define the responsibilities of the tests. This allows the corresponding colleagues to be addressed with the bug report in a targeted manner via automated means.
The integration with e.g. a Splunk dashboard can be seen as the icing on the cake for the graphical processing of the test results. This makes it easy to read the current situation at a glance, especially with many software tests.
If that’s not enough, the traffic light system is well equipped. With the previously defined thresholds, the different colors of a USB traffic light can be controlled. If, for example, you come in in the morning and see the traffic lights glow red, then it is immediately clear: there have been errors. Otherwise, the day can start relaxed at a” green ” traffic light, because all tests have gone through and the software has been tested without errors!
Fully automated tests are an indispensable factor, especially in agile software development, in order to ensure software is continuously error-free. Valuable resources can be saved through nightly test automation. The Robot Framework provides an excellent tool with which all software tests can be automated.
* Timo Füg is distinguished by his knowledge in the quality assurance of software developments and works at Uniserv GmbH as Technical Lead Software Quality Assurance.