General Functional Automation Tool Architecture

The functional test automation tool provides the following functions:

Definition of tests: The definition of tests is done by recording an interaction with the applications

Application to test: The record outputs a test script that can be edited by using the integrated script editor. In order to handle data-driven tests, the functional test automation toolprovides data access capabilities that help select the data source accurately. In order to facilitate the test result analysis, the tool can be used to define control points that can be set either on graphical objects or on data.

Execution of tests: The execution of tests is automated. Various test cases are executed by reproducing the recorded user interaction. Data-driven tests are executed based on the data access that was set during the definition period.

Reporting on test results: When the test is finished, the test results compare the test execution to the reference state (e.g. based on the control points set during the definition period).

Test Strategy Guidelines

In order to make the most of test strategy, we need to make it reusable and manageable. To that end, there are some essential guiding principles we should follow when developing our overall test strategy:

1. Test automation is a fulltime effort, not a sideline


2. The test design and the test framework are totally separate entities


3. The test framework should be application-independent


4. The test framework must be easy to expand, maintain, and perpetuate


5. The test strategy/design vocabulary should be framework independent


6. The test strategy/design should remove most testers from the complexities of the test framework

Typical Testing Steps

Most software testing projects can be divided into general tasks or steps.
Test Planning – This step determines which applications (or parts of applications) should be tested, what the priority level is for each application to be tested, and when the testing should begin. Applications with high levels of risk or heavy user volumes are identified.

Test Design – This step is for determining how the tests should be built and what level of quality is necessary for application effectiveness. During this phase, individual business requirements and their associated tests should be addressed within an overall test plan.

Test Environment Preparation – This step of the testing process is concerned with establishing the technical environment that the test(s) will be executed in. Without this step, the investment in test automation is at risk because of the inability to re-execute the tests.

Test Construction – At this step, test scripts are generated and test cases are developed based upon the test plans created during the design phase. Most of the time spent in automated testing is typically in the test construction phase.

Test Execution – This step is where the test scripts are executed according to the test plans. As test execution is the most tedious and repetitious step during a manual testing process, the automation of this step is where the most significant time savings are made.

Test Evaluation – After the tests are executed, the test results are compared to the expected results and evaluations can be made about the quality of an application. At this stage, application errors or problems are identified and appropriate corrective actions can be considered. Decisions can be made as to the readiness of the application for release.

Delusion of Automation Testing

While it has been proven that automated testing is valuable and can produce a successful return on investment, there isn't always an immediate payback on investment. It's important to address some of the misconceptions that persist in the software industry and to manage the automated testing delusion. Following is a list of just a few of the misconceptions that need to be addressed. People often see test automation as a silver bullet; when they find that test automation requires a significant short-term investment of time and energy to achieve a long-term return on investment (ROI) of faster and cheaper regression testing (for example), the testing tool often becomes "shelfware." This is why it's important to manage expectations in order to introduce automated testing correctly into a project.

1. Testing is a "sequence of actions.
2. Testing means repeating the same actions over and over.
3. We can automate testing actions.
4. An automated test is faster, because it needs no human intervention.
5. Automation reduces human error.
6. We can quantify the costs and benefits of manual vs. automated testing.
7. Automation will lead to "significant labor cost savings."

Understanding the Testing Process

The introduction of automated testing into the business environment involves far more than buying and installing an automated testing tool. In fact, effective automation is predicated on the idea that a manual testing process already exists since there is not a technology in existence today that performs automatic testing. So it is recommended that testing organizations begin their testing projects with a structured approach. Within the testing environment, a quality assurance process is defined as a set of related steps designed to ensure or verify that a software application meets the requirements of the business user. Before attempting to automate a test, a solid grasp of basic testing processes is needed, as is an understanding of what automated testing can accomplish, and an idea of which tests are good candidates for automation. In fact, not all tests should, or can be, automated. When considering which tests to automate, focus should be placed on those manual test activities which take the longest time to set up including those manual tests that require the highest number of repetitive tasks and which are run the most frequently. Automated testing should be used to augment current testing methods, and the process of planning for automated testing should be an extension of the current test planning process.

Criteria to assist in deciding what to automate

Identifying Tests Requiring Automation
Most, but not all, types of tests can be automated. Certain types of tests like user comprehension tests, tests that run only once, and tests that require constant human intervention are usually not worth the investment to automate. The following are examples of criteria that can be used to identify tests that are prime candidates for automation.

High Path Frequency - Automated testing can be used to verify the performance of application paths that are used with a high degree of frequency when the software is running in full production. Examples include: creating customer records, invoicing and other high volume activities where software failures would occur frequently.

Critical Business Processes - In many situations, software applications can literally define or control the core of a company’s business. If the application fails, the company can face extreme disruptions in critical operations. Mission-critical processes are prime candidates for automated testing. Examples include: financial month-end closings, production planning, sales order entry and other core activities. Any application with a high-degree of risk associated with a failure is a good candidate for test automation.

Repetitive Testing - If a testing procedure can be reused many times, it is also a prime candidate for automation. For example, common outline files can be created to establish a testing session, close a testing session and apply testing values. These automated modules can be used again and again without having to rebuild the test scripts. This modular approach saves time and money when compared to creating a new end-to-end script for each and every test.

Applications with a Long Life Span - If an application is planned to be in production for a long period of time, the greater the benefits are from automation.

When should you use an automated testing

Why Automate the Testing Process?
In the past, most software tests were performed using manual methods. This required a huge man power to perform expensive, and time-consuming manual test procedures. Owing to the size and complexity of today’s advanced software applications, manual testing is no longer a viable option for most testing situations. Every organization has unique reasons for automating software quality activities, but several reasons are common across industries.

Using Testing Effectively
By definition, testing is a repetitive activity. The very nature of application software development dictates that no matter which methods are employed to carry out testing (manual or automated), they remain repetitious throughout the development lifecycle. Automation of testing processes allows machines to complete the tedious, repetitive work while human personnel perform other tasks.
Automation allows the tester to reduce or eliminate the required “think time” or “read time” necessary for the manual interpretation of when or where to click the mouse or press the enter key.
An automated test executes the next operation in the test hierarchy at machine speed, allowing tests to be completed many times faster than the fastest individual. Furthermore, some types of testing, such as load/stress testing, are virtually impossible to perform manually.

Reducing Testing Costs
The cost of performing manual testing is prohibitive when compared to automated methods. The reason is that computers can execute instructions many times faster, and with fewer errors than individuals. Many automated testing tools can replicate the activity of a large number of users (and their associated transactions) using a single computer. Therefore, load/stress testing using automated methods require only a fraction of the computer hardware that would be necessary to complete a manual test. Imagine performing a load test on a typical distributed client/server application on which 50 concurrent users were planned. To do the testing manually, 50 application users employing 50 PCs with associated software, an available network, and a cadre of coordinators to relay instructions to the users would be required. With an automated scenario, the entire test operation could be created on a single machine having the ability to run and rerun the test as necessary, at night or on weekends without having to assemble an army of end users. As another example, imagine the same application used by hundreds or thousands of users. It is easy to see why manual methods for load/stress testing is an expensive and logistical nightmare.

Replicating Testing Across Different Platforms
Automation allows the testing organization to perform consistent and repeatable tests. When applications need to be deployed across different hardware or software platforms, standard or benchmark tests can be created and repeated on target platforms to ensure that new platforms operate consistently.

Repeatability and Control
By using automated techniques, the tester has a very high degree of control over which types of tests are being performed, and how the tests will be executed. Using automated tests enforces consistent procedures that allow developers to evaluate the effect of various application modifications as well as the effect of various user actions. For example, automated tests can be built that extract variable data from external files or applications and then run a test using the data as an input value. Most importantly, automated tests can be executed as many times as necessary without requiring a user to recreate a test script each time the test is run.

Greater Application Coverage
The productivity gains delivered by automated testing allow and encourage organizations to test more often and more completely. Greater application test coverage also reduces the risk of exposing users to malfunctioning or non-compliant software. In some industries such as healthcare and pharmaceuticals, organizations are required to comply with strict quality regulations as well as being required to document their quality assurance efforts for all parts of their systems.

Results Reporting
Full-featured automated testing systems also produce convenient test reporting and analysis. These reports provide a standardized measure of test status and results, thus allowing more accurate interpretation of testing outcomes. Manual methods require the user to self-document test procedures and test results.

Introduction to tests automation

In today’s software development life cycle, testing is a very important phase as its time taking and monotonous test automation has a emerged as wonderful solution and become very popular for both Client Server and Web environments.

As the Scope and requirements for testing is increasing day by day and time for testing is getting smaller and smaller every day the top management are realizing a greater need for test automation or automated tests.

Definition

"Automated tests execute a sequence of actions without human intervention".

This approach helps eliminate human error, and provides faster results. Since most products require tests to be run many times, automated testing generally leads to significant labor cost savings over time. Typically a company will pass the break-even point for labor costs after just two or three runs of an automated test.


Overview

Application testing is a critical part of all software development projects today . As more organizations develop mission-critical systems to support their clients and business activities, the need is greatly increased for testing methods that support business objectives. It is necessary to ensure that these systems are reliable, built according to specification, and have the ability to support business processes.

Many factors are forcing organizations to ensure a high level of software quality and reliability. Some of the most commonly cited reasons requiring organizations to adopt software quality initiatives include the increasing quality standards of application end-users, regulated quality requirements, out-of-control maintenance costs, and missed deployment schedules.

Application managers and developers increasingly use testing to support the goals of their organization and ensure an application performs as designed.

The benefits of test automation
1. The possibility to conduct regression tests on a new version of software, particularly in an environment where programs are changed rather frequently.
2. The possibility to carry out several tests and to carry them out a greater number of times is clearly one of the benefits of automation.
3. The possibility of running tests that are impossible to carry out manually
4. A better use of human resources. Automate the repetitive activities such as the repeated use of the same input data, leaving the testers more time to plan better test cases or to more thoroughly carry out test cases that are better done manually.
5. Hardware resources can be managed more carefully by planning the running of the test when the hardware resources are not active (at night, on holidays...)


(To be continued......)