Agile Practices to Help Mobile Testing

The growing trend to use mobile devices and software applications to everyday activities and communication resulted in popular use of mobile software applied to many kinds of devices platforms. There is the demand nowadays for agile and flexible testing procedures in the software companies. In this post I will describe an experience to implement some agile practices for mobile application testing.

Mobile Application Testing

Mobile Application Testing is testing using well-defined software test methods and tools to ensure quality in functions, behaviors, performance, and quality of service, as well as features, such as mobility, usability, interoperationability, connectivity, security, and privacy [1].

Mobile Application testing has some peculiarities that include connectivity, convenience (Quality of Design), supported devices (diversity of devices and OS), touch screens, new programing languages, resources constraints and context awareness.There are challenges for testing process mainly for test selection and test execution because of the rich contextual inputs and lack of test tools [2]. 

According to the technical literature, the important types of testing in mobile application are [3]:

· GUI Functional Testing: Functional testing ensures that the application is working as specified in the requirements. 

· Performance Testing: This testing is undertaken to check the performance and behavior of the application under certain conditions such as low battery, bad network coverage, low available memory, simultaneous access to application’s server by several users and other conditions.

· Memory and Energy Testing: memory leaks may preempt the (limited) memory resource, and active processes (of killed applications) may reduce the device (battery) autonomy.

· Interrupt Testing: An application while functioning may face several interruptions like incoming calls or network coverage outage and recovery. The different types of interruptions are: Incoming and Outgoing SMS and MMS, Incoming and Outgoing calls, Incoming Notifications, Battery Removal, Cable Insertion and Removal for data transfer, Network outage and recovery, Media Player on/off, Device Power cycle.

· Device multitude Testing: Testing an application on a multitude of mobile devices.

· Usability Testing: The usability verifies if the application has friendly interface, achieving its goals and getting a favorable response from users. This is the key to commercial success.

· Installation testing: This testing verifies that the installation process goes smoothly without the user having to face any difficulty. This testing process covers installation, updating and uninstalling of an application.

· Certification Testing: The mobile application needs to be tested against the guidelines set by different mobile platforms to be available in an application store, example: Nokia Test Criteria, IOS Testing criteria, Microsoft Test Criteria and etc.

These types of testing required impact the test activities during application development, especially in agile methodologies.

Experience: Project Description

The project X was a mobile application to help to navigate cities giving the access to scheduled departure/arrival times from public transportation.

The project followed the Scrum development framework and the software platforms used were J2ME programming language and Eclipse IDE. This project had two deliverables for different mobile platforms. The project team had 4 developers, one Scrum master, 1 Product Owner, 2 designers and 2 testers.

Initially, developers were responsible for code new features, unit tests and Continuous Integration (CI), and Testers should take care of functional tests, non-functional tests, and certification tests for mobile applications.

 Testing Process

The testing process was designed initially following the traditional sequential phases: Test Planning, Test Specification, Test Execution and Test Report. The test tools used were: Testlink to manage test cases, Jira to manage failures and J2ME Unit to automate unit tests. In the beginning of project the test team and design team were not allocated along with the development team despite belonging to the same company.

In early iterations of the project development, some problems were identified as: slow response to changes in the project requirements, poor and insufficient unit testing, little time to execute functional and non-functional testing and unstable application running into target device.

In order to solve these issues and accommodate agile development iterations with all types of testing required, the test process was revised and some agile practices were implemented as following:

·     Team Co-location: Testers, designers and developers allocated in the same room to join the team and encourage cooperation.

·     Pair programming: Developers and Testers together in pairs to implement unit testing to improve test coverage.

·    Pair Testing: Testers together to implement and execute exploratory tests and non-functional tests (Performance and Security).

·    Prioritization of regression testing: the regression testing was executed following of most critical features to be delivered (covering Certification Testing).

·     Designers involved in interface tests to identify unconformities of screen flow and screen design.

.

Fig. 1. Testing Process Activities

The Figure 1 shows the testing process activities designed for mobile applications. 

The programming skills tester did the pair programming to improve the unit testing.

The Test Execution includes the execution of functional tests, regression tests, performance tests, security tests, stability tests, regression tests, certification tests and interface (GUI) tests. If failures were found, they were reported to developers, once the failures were fixed, the test team executes the retest to validate the faults. When all critical failures were fixed the test report is generated and sent to the project team.

Results

Reaching the agile practices, the project team found the right way to work. The co-location improved the communication problem between testers, designers and developers.It also promotes fast feedback when changes were required. The pair programming solved the poor unit testing, since testers helped to improve the quality of scripts and test coverage. Running regression tests for prioritized features, involving designers in interface GUI tests and the pair testing approach made the cooperation of testing team happen. It allowed them to run all types of tests required for this mobile application on time. 

The table below shows the results comparison between a past application Project 1 with the traditional test process and the Project X that used the agile practices. Both projects had the same number of testers, developers and designers. The projects had 6 months of time to develop, test and delivery the application.The test team in Project X could execute more types of important tests for mobile applications and the project was delivered with stability and without failures, it was a really good result.

Projects	Number of Features	Failures Found and closed	Failures remain after delivery	Types of testing applied
Project 1 (Traditional Test Process)	9	442	9	─   Unit Testing ─   Functional Testing, ─   Exploratory Testing ─   Regression Testing ─   Performance Testing
Project X (Agile practices applied to testing process)	11	388	0	─   Unit Testing ─   Functional Testing ─   Regression Testing ─   Performance Testing ─   Security Testing ─   GUI Testing ─   Stability Testing ─   Certification Testing ─   Field Testing

References

   1.    Gao, J.; Xiaoying Bai; Wei-Tek Tsai; Uehara, T., "Mobile Application Testing: A Tutorial," Computer , vol.47, no.2, pp.46,55, Feb. 2014
doi: 10.1109/MC.2013.445.

22.    Kirubakaran, B.; Karthikeyani, V., "Mobile application testing — Challenges and solution approach through automation," Pattern Recognition, Informatics and Mobile Engineering (PRIME), 2013 International Conference on , vol., no., pp.79,84, 21-22 Feb. 2013 doi:10.1109/ICPRIME.2013.6496451

33.   Kumar, M.; Chauhan M.; “Best Practices in Mobile Testing”, White Paper Infosys, 2013 http://www.infosys.com/flypp/resources/Documents/mobile-application-testing.pdf.