Software engineering is by nature a highly collaborative activity. However, this collaboration is more difficult when the teams are geographically separated, as several factors, such as work-time, cultural differences, communication, technical capability, among others, may impact on its success. Moreover, each activity in the software development process has specific needs in a distributed software development (DSD) environment.
In this post, I will report an industrial experience of a testing team separated geographically in the context of a software project that followed an agile method. This experience was possible through a collaboration between industry (Nokia Institute) and the federal university of amazonas, and its generated a paper presented in ICGSE 2012.Authors: Eliane Collins, Gisele Macedo, Nayane Maia and Dr. Arilo Dias-Neto.
APPLYING AGILE TESTING IN A DSD ENVIRONMENT
A. Software Project Characteristics
This project had been conducted in the context of a research institute in Manaus/Amazonas/Brazil aiming at
developing a web application to manager advertising campaigns, called Project X. This software project was developed following the Scrum agile methodology , and it consisted of 30 stories (Product Backlog) divided into 9 Sprints (iterations) varying from 2 to 3 weeks. This project had a development team composed of 1
Scrum Master and 3 full-time developers. The Testing Team, the focus of this paper, consisted of 6 professionals, being 2 full-time testers located in the same physical environment of the developers and four part-time testers who worked in another site (geographically separated) as a result of a cooperation between the research institute and the Federal University of Amazonas (UFAM).
The Project X was developed using a web platform (PHP + MySQL database), and IDE Eclipse. The system was composed of 8 screens (4 forms to register campaigns and users and 4 screens to search and generate reports).
B. Testing Process
The Project X’s software testing process followed the Scrum methodology, where the testing team was integrated to the project team, participating in Scrum ceremonies (sprint review, daily, retrospective and planning meetings). Their responsibilities were to plan test cases through the stories described in the Product Backlog, specify the acceptance criteria, and use test automation tools to speed up the execution activities of each sprint.
Due to the geographic distribution of the testing team, the testing process was designed considering the characteristics of a DSD (distributed software development) environment. We needed to provide a testing structure allowing access to the project’s information by all team members. Thus, in order to attend this demand, the testing team used a server dedicated to be connected remotely with the following tools:
• TestLink: used to manage test plans, write test cases, and report tests execution. Besides, the selection of tests to compose a test suite was done manually, TestLink acted as an editor and organizer of test cases, storing all information. It facilitated the creation of test plans and reports documentation, and the controlling of the tests execution versions;
• Mantis Bug Tracker: through this tool, which was already used in the institute, the tester registered the
defects found, sent them to the developers and controlled the lifecycle of each defect.
• FireScrum: a web tool for Scrum taskboard was used to detail tasks defined for the project sprint. This tool was used for both testing teams (remote and local) to record and update the tasks progress. Thus, the test coordinator used these as data in the project daily meeting.
• Subversion: it was used to share and manage information among the test team members.
Moreover, in order to ensure the operation of this infrastructure and facilitate communication among the teams, a Test Leader located in the institute was responsible for defining the testing tasks, planning the sprint activities and reporting the progress of these activities at the daily meetings. Only the Test Leader could communicate with the development team, avoiding a larger communication network in the testing team. The remote testing team was allocated to the tasks of designing, automated (creating scripts), and executing test cases for the stories developed in each sprint.
The tasks of test process during the sprint were incremental and iterative following the scrum process. They are represented in Figure 1. The dark rectangles represent the Test Leader’s activities and the lighter rectangles represent the Remote Testing team’s activities.
Figure 1. Overview of Testing Process
The task Incremental Test Execution includes execution of exploratory tests and automation of regression tests. For Project X project, the tool chosen for the functional tests automation was the Selenium RC and IDE5. Selenium uses the approach record-play support and tests web applications for the browser Mozilla Firefox. Selenium IDE can record the user actions in the browser, creating test scripts in several programming languages and executing them later. The Selenium Java API used in the project allows running the test scripts in other browsers such as the versions of Internet Explorer. The automated test suite is updated and executed in every sprint. The tools TestLink and Selenium were integrated, thus, when
test cases are executed in the Selenium tool, their results are recorded automatically in TestLink.
C. Communication Process
According to agile practices, communication is an essential factor for the success of the agile project. Project members who have good communication process can cooperate more and mitigate risks of changes during the project. It is an essential factor when some teams of the project are geographically distributed.
To work with distributed teams, it is important to use tools to facilitate communication. In this project we used several tools like e-mail, a free scrum tool for task board (FireScrum), and the chat and video-conference tool (Skype) to make possible online communication among team’s members, resolving doubts as soon as possible. Thus, e-mail was used only for offline communication (invitations to meetings and
project documents).
Daily, all team members needed to access the online FireScrum tool, where the tasks were created and allocated to the testing team members by the Test Leader. The Remote Testing team was responsible for updating the FireScrum tasks every day, reporting the progress and impediments.
Figure 2 represents the communication flow among the project teams. According to the flow, communication between the Development Team and the Test Leader was straight (face- to- face) while communication with the Remote Testing team occurred through the server test tools.
Figure 2. Communication Network.
D. Tasks Allocation
Using the Scrum methodology, tasks are defined and estimated in Sprint Planning Meeting. In this ceremony, each professional involved must participate to specify the tasks necessary to accomplish the stories. All members were invited to this event, including the remote team. On presentation of the definition and clearly understanding the tasks, the Test Leader inserted them into the FireScrum tool and allocated them to the testing team members. During the sprint, the Remote Testing team could add and assign new tasks, when appropriate. With all communication tools available, any possible impediment from the remote testing team could have been quickly detected and easily solved avoiding delays. Each professional team was responsible for his/her external tasks and he/she had to communicate any eventual difficulty as soon as possible. The task status could be defined as: to do (to be made), in progress (ongoing task) and done (tasks completed).
Every day, before the daily meeting of the project in the company, the Test Leader checked the progress of tasks to inform the development team. Among the main tasks assigned to the Remote Testing team, we can cite: specification of new test cases, update of test cases from previous stories, creation and automation of
test scripts, execution of test cases and test scripts of the stories in sprint, defects registration, execution of automated scripts to regression tests and validation of solved defects.
The main Test Leader’s tasks can be summarized in: review created test cases, monitor tasks in FireScrum, facilitate test tasks, send project information and report test execution.
E. Planning and Restrospective Meetings
Following Scrum, there are two other important meetings to be performed: planning and retrospective. Planning is the meeting where the whole team decides to express their opinion about how to get the stories according to the prioritized backlog. In Project X, all testing team participated in this meeting (including the remote team), choosing the stories to be developed and estimating the complexity of the selected stories. The retrospective meeting is considered a very important ceremony because it is where is said what went right and what should be improved for the next sprint. All team members, including the remote testing team, participated in this event, sharing experiences and understanding the difficulties that would be improved in the next sprint. With the participation of the remote testing team in both Scrum ceremonies, it’s possible to get the unit of the project team and everybody feels as part of the team.
CHALLENGES AND LESSONS LEARNED
In this experience on conducting distributed testing in an agile software project, we could observe it can work very well. However, some issues need to be managed to avoid the risks introduced by the combination of these software engineering practices (DSD + agile practices). Thus, we identified some challenges and key lessons that we learned for minimizing the impact of the geographical distance between the testing teams:
A. Communication and Coordination are essential factors for the success of distributed testing All main research works in the DSD field indicate communication and coordination as important aspects to carry the success in a distributed software project, and we could also confirm these claims. Some practices were essential to reach this success:
• Allocating one person (test leader) as a link between the local and remote testing team is very important to avoid a large communication network and, consequently noises. Thus, all information and decisions would always pass through this professional, responsible for distributing information, solving impedances and communication problems, and making the testing tasks easier.
• A communication protocol should be formalized, regulating how the teams should keep contact with each
other. That includes structure of emails, communication tools, meetings, timetable, and so on. This is essential to avoid losing data, effort, and quality in the software project. For instance, the definition of a standard for test cases specification and bug reporting allows a tester to complete a task started by another tester or validate a bug reported by another professional.
• An online chat tool should be used and the key persons in the project (Scrum master, development and test leaders) should be always available to clarify doubts.
• Periodic meetings between the development and testing teams should be scheduled. This contributes to understand the complex user’s stories. Moreover, the physical presence of the remote testing team in the
Scrum meetings (Planning, Retrospective and Review) is very important to share, among all software project
members, information regarding problems, suggestion of improvements and planning of new activities.
B. The project information should be available with details to all members As part of the testing team is separated geographically, only short stories and acceptance criteria descriptions are not enough for test cases’ specification. Thus, user’s scenarios and the application wireframe should be available to cover all features to be developed/tested, bypassing the limitations imposed by the distance between part of the testers and the product owner. Moreover, changes in the user’s scenarios result in a high effort to update test cases and their automation scripts. Therefore, these scenarios should be kept updated, otherwise they can affect the quality of the testing activity.
C. Automation reduces the needs of physical presence in the testing process The tests automation has an essential role in the success of distributed testing, because both teams would be able to run the regression tests suite constantly without depending on time zones between the remote and local testing teams. Moreover, automation speeds up the time required for running the tests even when new releases are published by the development team close to the deployment deadline.
D. Supporting tools are always important, but testing team organization is more Tools are also highlighted in the technical literature as an important element in a distributed software project. In the context of distributed testing in an agile project, two special tools should be cited:
• A Scrum dashboard tool presents all activities performed daily by the remote testing team, showing information regarding the performance of testers and the tasks’ progress. Thus, the test leader is provided with information to support the testing process management, mitigating eventual risks. In our software project we used the FireScrum tool.
• A test management tool controls all information regarding the testing activities, test cases specification and validation, creation of new releases, test running, number of detected failures, and bug tracking (creation, correction, and validation). In our software project we used the TestLink and Mantis tools. Besides their importance, tools can fail, and we need to be prepared for this moment. If the network or a server crashes, all remote teams’ activities can be affected because communication between the teams will cease to exist. Thus, we cannot be dependent on tools to perform our activities.
CONCLUSIONS
We could observe that it is feasible to integrate the DSD with the agile practices, maintaining management and organization of a software project, because some of them are considered in the testing activities for both scenarios, as the need of an efficient communication, automation as a resource to reduce cost, and task allocation in small parts. On the other hand, the difference in these scenarios regarding the testing activities, such as continuous integration and daily meetings, could be avoided with some technological solutions already reported in the technical literature.
Some challenges and lessons learned could be extracted from these experience and they could support other software engineers when performing testing in a similar environment.
