How I Automate Tests...

To automate tests, first I think the requirements that I need to start.

The figure 1 bellow shows the inputs and the outputs that I consider in test automation.

Figure 1 Test Automation Inputs and Outputs

To start the test automation activities I need the inputs:

1. Test Cases selected to be automated, they must be repeatable.
2. Test Data.
3. Test Tools environment configured and preferably Continuous Integration.
4. Code, even using TDD (test driven development) the code (can be fake)  is needed to run the tests.

As desirable outputs of test automation, basically I consider the following artifacts:

1. Test Scripts.
2. Execution Log.
3. Results Report.
4. Traceability (tests and requirements or tests and failures) and code coverage.

Depending of  test tool, other metrics could be extracted from test automation.

During the project a lot of external events can happen to disrupt the activities of automation as changes in requirements, extra requests from customer, change of deliverable dates and etc. For this reason I adopt some procedures listed bellow to automate tests and execute the software test activities.

1.  Plan the test cases for the priority functionalities developed using a test tool (for example Testlink). 
2. Select the test cases to be automated (selection can be according to acceptance criteria).
3.  Automated test scripts creation according to test cases selected to be automated. These test scripts must be possible to be edited and gathered in a test suite, and reused to others test cases;
4. Update the test scripts is important to avoid wrong results.
5. Then, the testers executed the automated tests using a test tool and the exploratory manual tests also. The defects found were registered in Testlink and in the defect tracking tool.
6. The automatic regression tests for all functionalities  must be executed also when defects were validated to ensuring that another part of system was not affected by code changes.
7. The automated TestLink reports can be generated. The development team released another system version and the same activities process began again. 

I call these procedures Test Iteration because it will be repeated again for next version of software until the end of project.

I executed it in Scrum projects and Waterfall projects as well.My experience is reported in my paper "Iterative Software Testing Process for Scrum and Waterfall Projects using Open Source Testing Tools Experience". It was presented in the event ICTSS 2010 (International Conference of Testing Software and Systems).

Executing these procedures is possible to ensure the test automation activities in the project and avoid problems as outdated test scripts.

According to project platform is possible to use many test automation tools, for example:

Web projects: Selenium Test tool and Jmeter

For desktop projects (Java AWT or Swing) : Marathon test tool

For C# or Java application is possible to use Fitnesse test tool

For mobile J2ME applications : J2MEUnit

My favorite tools to use are Selenium, Fitnesse and Jmeter. In next posts I will detail how to use them.