Automated test framework

For automated testing framework, it is not as profound as most people think. The concept of framework is just a series of pre-defined standards and specifications. In automated testing, we often mention the analysis of test requirements, script design, test execution, test report, maintenance management and so on. They are connected and encapsulated through the framework, so that the end users of the framework can use them more conveniently. However, a good automated testing framework should not only make users easy to use, but also consider many other factors. Let's share some personal experiences.

Choose a type of framework. At present, there are three common automated testing frameworks: data-driven framework, keyword driven framework and hybrid framework. 1. Data driven framework data drives applications with fixed business logic that are most suitable for testing, and only the test data will change. Usually, the test data will be configured in an external file or database. 2. Keyword driven framework keyword driven as the name suggests, it provides a series of general keywords. Users can realize a single operation by calling these keywords and entering some parameters, such as opening a browser, opening a web page, clicking a link, etc., and then form a complete test process by organizing these keywords. 3. Hybrid framework hybrid framework integrates data-driven and keyword driven, and has the advantages of both. Different from the keyword framework, this framework usually provides some keywords for specific applications, such as login, logout, etc. Then, on the basis of the complete test process, a layer of data-driven is applied, which can make the test logic and test data more flexible and configurable.

There is no need to rebuild the wheel. When designing the framework, we should use the functions provided by the automatic test tool as much as possible. For example, in a framework based on selenium, selenium itself has provided functions such as opening browsers and web pages and clicking buttons, so you don't have to spend time developing such keywords to reduce development costs.

Reusability a good framework must have high reusability. We can combine some individual operations into some of the most commonly used test processes. For example, the three operations of "enter user name", "enter password" and "click login" can be combined into a keyword "login".

Configuration management like the development team, automated scripts also need configuration management, so as to more effectively manage script submission, modification, version control, baselining and other operations. Therefore, when designing the framework, it is necessary to consider combining configuration management tools, such as CVs, VSS, SVN, etc.

Configurability 1 The configuration items of external configurable script should be placed in external files, such as URL, path, version information, etc., so that the script can run in different environments. In addition, the path of the configuration file cannot be written dead, and the relative path should be used to ensure that the framework can run smoothly on different machines. 2. Internal configurability when the framework is deployed to different machines, there will be different environments, and it should be able to automatically complete the necessary configuration according to different system environments. For example, if a selenium framework is deployed to machines with different browsers, the framework should be able to run them separately according to which browsers are installed on the current system.

Most of the problems encountered in object library maintenance in automated testing are basically script failures caused by changes in object properties. Therefore, the maintenance ability of object library is very important for a framework. We can treat the object library as a shared resource and maintain it by special personnel. The object library can be an external XML, Excel, database, etc.

The execution mode needs to consider the following use case execution requirements: 1 Execute a separate use case 2 Execute a test case set 3 Re execute the failed use case 4 Execute the corresponding use cases or use case sets according to the operation results of other use cases or use case sets. In addition, there may be other methods, which are mainly based on the requirements of the project. Therefore, it is only necessary to meet the specific project test requirements, not all of them.

During the execution of the script, the framework should be able to monitor the operation of the script in real time. In case of operation failure, it should be able to carry out basic fault-tolerant recovery. In this way, the script will not be in a blocked state, thus wasting a lot of time.

Test report different applications often have different test report requirements, Sometimes it is necessary to combine the running results of many use case sets (general operation report) see how many successes, how many failures, and the passing rate; sometimes it depends on the implementation of a single use case, which step fails, and what the failure reason is. On the other hand, diversified test report forms also need to be considered. Excel, word, web, pdf,... And other forms can be selected according to the actual project needs, but no matter how they are made At least the test report shall be readable and accessible in any form.

Easy debugging generally, debugging will occupy a lot of time in the process of developing test scripts, so whether it is easy to debug is also a very important factor, which directly affects the cost of developing and maintaining the framework, which can not be ignored.

A good framework of test log should be able to produce sufficient detailed log information (text, screenshots, etc.) during test execution, which is very helpful for debugging. At the same time, it can also quickly locate the problem and save time.

Easy to use, easy to learn and easy to use is also very important for the automated testing framework, because it is oriented to end users after all. If the framework is difficult to start, it will lose the user group, and the framework will have no meaning to exist. Therefore, on the basis of ensuring ease of use, it is better to have a detailed framework description document, which will be of great help to novices.

Flexibility refers to the ability of the framework to ensure secondary development on the current basis. This is actually the same as the standard of software development. Sufficient scalability is reserved for future version upgrading.

The design of performance framework should not be too complex. Too complex framework will increase the loading and running time of script, resulting in low efficiency of running script. Therefore, when designing the framework, we should also consider the factor of performance.

Reference external tools. Some external tools themselves provide automation interfaces. We might as well integrate them into the framework, which can save a lot of work. For example, the classic QTP + QC framework.

Coding specification good coding specification can make the script easy to read, maintain and manage. Some points are listed below: 1 Naming conventions for variables, constants, functions, files and scripts 2 Annotation specification of function and function library 3 Object naming conventions

Finally, we need to say that there is never the best automated testing framework, only the most suitable one!