Automated Functional Testing Techniques and Risks Associated with Automation

Automated Functional Testing Techniques and Risks Associated with AutomationAutomated functional test tools make use of few techniques that differentiate themselves primarily due to the contents of the scripts generated by them, like record & playback, scripts programming, data-driven and keyword-driven. An outline of these techniques is given below. 1) Record and Playback Technique: This technique consists of recording a test execution made on the application's interface and playing back this execution later. The generated scripts contain unalterable data, test procedures, and expected results. The advantage of this technique is the simplicity of generating scripts. However, there is a high sensibility to changes, which restricts a script's lifetime. For example, a simple change of an interface may lead to the necessity of recording all the scripts once again, resulting in a substantially high script maintenance cost. 2) Scripts Programming Technique: Considering the components of the generated scripts, this technique is similar to the record & playback technique as described above. However, in script programming technique we can update the generated scripts. With this resource, script programming has a higher rate of reuse, longer lifetime, and less difficulties to be maintained, compared to the technique of Record & Playback. This technique also involves a big volume of scripts and calls for high-maintenance cost. 3) Data-driven Technique: This is a technique that approaches test-data extraction through scripts, storing them in separate files. Therefore, scripts will only contain test procedures and actions for the application. This technique is useful for tests using different inputs and combinations of scripts with common test procedures. This way, maintenance requires a reduced effort when it is necessary to include, update, or delete any test data. However, data-driven technique depends on the test-execution logic, in other words, if any step is added to the procedure (or even removed from it) the script needs to be generated again. It allows automating in parallel: test data can be created while test procedures are being generated. The extraction of test data from scripts provides tests that are easy to understand and revise. Many test tools include direct support for the data-driven technique. 4) Keyword-driven Technique: Based on retrieving test procedures from scripts, remaining only test data and specific test actions, which are identified by keywords. This resource's operation, being invoked by key words and even receiving parameters, is similar to that of structured-program functions. An important enhancement of this technique is the reduction of script-maintenance effort if there is incorporation, removal, or modification of any step of a test-procedure execution. In spite of many testing tools available in the market and recommending any specific testing technique, the organization needs to establish the most adequate technique according to their own test context and needs after analyzing all possibilities thoroughly. Risks associated to implementation of Test Automation Fact remains that the implementation of any automated tests solution is not simple, even with the best of best test tools available. Implementation can still fail possibly due to following risks associated to it. 1) Non qualified Test Team: An automated software testing tool requires new skills for software testing engineers & test analysts, therefore additional training is required. An efficient automation is not that simple. Test scripts automatically generated by the tool during recording must be modified manually, which requires knowledge of scripting, so as to make the scripts robust, reusable, and maintainable. To be able to modify the scripts, the software testing engineer must be trained on the tool and the tool's built-in scripting language. 2) Inadequacy of the Test Tool: The automation tool must be in accordance to the company's business needs with the test process introduced. There are many software testing tools that automate tests and a correct choice cooperates to fully use features offered. It can be said that we can’t find a single tool that proves to have clear superiority over other tools for every situation. The choice of the most adequate tool depends on characteristics of applications, test stages to be automated, as well as on the test policy adopted by the organization. Simply using tools, by themselves, will not promote a significant quality enhancement of developed products, unless accompanied by an adoption of a work methodology. An incorrect choice may lead to automation failure, due to incidental deviation from the organizational testing needs and diversion of resources and efforts towards the manual tests. Once there is a dent on the credibility of the automation tool, it can become difficult to convince the higher management to go in for more investments on further automation or even to reinforce the manual tests. 3) High Script Maintenance Costs:  This aspect may not compensate automation costs. System wide test scripts have to be flexible enough to easily support various changes and insertion of new functionalities that appear during a project's life cycle. To create easily maintainable scripts, software testing professionals with some experience in development are important. They are supposed to be capable of designing a modular script structure, in the same way as when doing a normal software design. 4) Attempt to automate every type of test: When tests require high human intervention it is not recommended to focus on automated tests. 5) Lack of planning and control of the automation process:  Test automation must be considered a separate project from the software development process. Cautions related to project management have to be taken, such as concerns with scope, time, cost, quality, integration, resources, and risks. Execution of automated tests helps to provide reliable and stable applications. However, to reduce associated costs and risks, it is important to analyze factors discussed above. Thus automation benefits can be more evident in the organization and risks can be controlled by mitigation and/or contingency. Brief outline of Benefits of Automation: Despite many risks described above, benefits of test automation as outlined below can’t be ignored. Thus it depends upon the organization to either intelligently capitalize on the benefits of the test automation or silently succumb to the risks. 1) Reduction in testing effort 2) Reduction in testing schedule 3) Improved regression testing 4) Improved focus on advanced test issues 5) Increased test coverage 6) Productivity increase Conclusion:The risks associated with either of the test automation techniques can be mitigated to certain extent with the approach of building sound test automation frameworks. Automation frameworks can’t be viewed as a replacement of any of the automation tools that are available as built as per the best international standards. Test automation frameworks simply lay down the process of building an efficient framework using any type of tool, by minimizing the obstacles of the tool, thereby increasing the tool’s productivity. Very shortly, I will bring out a new very simple article on building a generic test automation framework that I feel, will be helpful to all types of organizations involved in testing. (Source : http://www.softwaretestinggenius.com)