I use the definition of the book Automated Software Testing [1], it says that Software Test Automation means to automate
software testing activities including the development and execution of test
scripts, verification of testing requirements, and the use of automated testing
tools. Automating the test activities can reduce the effort of manual and repetitive tests and can provide quick results feedback for project team.
I consider that Test Automation is part of Software Development (not just part of Software Testing). It is a multidisciplinary activity since it envolves programming skills and testing skills, furthermore, the collaboration between the project team is a key factor for the success of this activity but that relationship I'll explain in another post.
According to Mark Fewster and Dorothy Graham [2] Automating tests is a skill very different from Testing, because knowledge of good tools and software architecture is required. But in order to gain benefits the project team must wonder why to automate tests, what automate, when to automate testing and the risks of automation.
Why to Automate Tests?
First , we all need to be quick (time to market).
In second, commonly in a system there is a set of many test cases to be run several times and soon, when they are executed only manually is risky because all tests can not be performed very carefully and effective all times due to factors such as the short time to delivery results, tiredness and even human distraction. In this sense Test Automation provides the execution of many repetitive tests quickly and facilitate the generation of documents. So, the test professional can dedicate time to execute tests that agregate value to the project.
Other reason to automate tests is the cost. The initial investiment is high, the team need to be trained, need time to analyze tools, buy tools and there is the learning curve to get test automation effectively in the project. But when the tests are automated and organized, they cost is cheaper than running manually because what needed to be run on a day for more than a person can run automatically at one hour and the results are generated fast.
Some other benefits for the project include: improvements on software development lifecycle, software reliability, quickly regression tests, generation of documents, failures monitoring and reproducing defects.
What Automate?
The candidate tests to be automate are :
- Development tests: Unit Tests and Integration tests. They must be automated to prevent defects in the code, facilitate changes, decrease dependency between classes and improve the software design.
- Acceptance Tests : basic requirements from customer.
- API level: functional tests of the system in the level of API, without GUI dependence. It identifies business rules failures.
- GUI Level : the GUI must be tested when the system interface is stable and will not change.
- Non Functional Tests: Security Tests, Performance tests, Stress and Volume tests, they must be performed when the system funcionalities are completed and can simulate the real production environment.
- Failures found: The critical or major failures found during exploratory test execution are important candidates to be automated to ensure that the failure will not return to the system after code changes.
- Repetitive tests: The tests that have to be executed many times have to me automated.
When to Automate Testing?
It's better to automate tests when ALL project team are envolved, commited in the automation activities and conscious of the importance and benefits of this activity. Everybody must to understand that it requires time to be spent and all should be willing to help. The automation must be planned and in the project schedule. With this, the automation can be sucessful.
When the automation activity is delegated to the responsibility of one person or one team it gets more difficult communication process and synchronization of tasks.
Common Risks of Test Automation [2]:
- Unrealistics expectations: it is not possible to automate 100% of the tests and manual tests still must be performed.
- Maintenance cost: depending of the system platform, the cost of maintenance of the test automation can be expensive and requires time when the system requirements change.
- Technical difficulties: when the system has a complex platform and lack of testing tools adequated and have no time to build one, it is not worth to automate tests.
- Lack of development process: without a development process organized, customer requirements and architecture documentation, it is not possible to automate the test activities.
- Lack of management support: the management of the organization must to know the value of test automation activity and the benefits for the project. Many projects has no test automation because the manager think that test automation is expensive and it's not agregate value to build the system. The old idea of the just development code is important to build a system.
References
