May 15, 2014

Testing Android Applications using MonkeyRunner

Monkeyrunner  is a tool from Android SDK. It provides an API using commands to execute tests in Android devices or Android emulators. It is possible to write a Python program that installs an Android application and perform it actions. 
The API has the following classes of commands: 

  • Monkey Device: it provides connection to a device or emulator.
    • http://developer.android.com/tools/help/MonkeyDevice.html 
  • Monkey Image: It can run packages, provides methods for installing and uninstalling packages, start an Activity, and send keyboard or touch events to an application.  
    • http://developer.android.com/tools/help/MonkeyImage.html 
  • Monkey Runner : It captures screen image, comparing two MonkeyImage objects and writing an image to a file.
    • http://developer.android.com/tools/help/MonkeyRunner.html

MonkeyRunner Example

To use the  monkeyrunner, it is necessary to include in the environment variable PATH the android tools directory. 
... \ sdk \ build-tools \ android-4.4; 
... \ sdk \ tools 


  • Create a folder called monkey in the c drive of the computer and put the scripts monkeyrunner_recorder.py monkeyrunner_playback.py available here: 
    • https://github.com/miracle2k/android-platform_sdk/tree/master/monkeyrunner/scripts
  • Create subfolders images, apks-script, log and apks_to_test


  • Open the emulator android or conect your Android device in USB (usb_debug must eon)
  • Open a command prompt and type: 
  • monkeyrunner c: \ monkey \ monkeyrunner_recorder.py


The interface monkeyrunner recorder will appear. 
With the mouse, try to perform actions on the emulator screen, like opening an application. 
The actions will appear on the right of the recorder interface.

MonkeyRunner Recorder

In the recorder interface , save the scripts clicking on the menu Export Actions
- Save the script as a file extension .mr (test.mr example) 
- Close the recorder interface and reopen the command prompt 
- Type: monkeyrunner c:\monkey\monkeyrunner_playback.py  c: \monkey\test.mr 
- Press enter and note that the emulator will play back your recorded actions on Emulator or in the Android device.

If you prefer, it is possible to create scripts directly in python and run. in monkeyrunner.

Follow the python code bellow, it tests installation of an application, it simulates the buttons' actions Back, Home and Menu and then it generates a log file and saves screen image.  

class DroidTest:

    rootfolder = 'c:\\monkey\\'
    apkDir = rootdir + 'apks_to_test\\'
    imageDir = rootdir + 'images\\'
    logDir = rootdir + 'log\\'

    def __init__(self, device, count):
        self.device = device
        self.count = count

    def run(self):
        eraseLog(self.device)

        for apk in glob.glob(self.apkDir + '/*.apk'):
            print "Testing apk %s" % apk
            packagename,activity = get_package_activity_name(apk)
            componentname = packagename + "/." + activity

            apk_path = self.device.shell('pm path ' + packagename)
            if apk_path.startswith('package:'):
                print"App is already installed."
            else:
                print"App not installed, installing APKs..."
                self.device.installPackage(apk)

            print componentname

            self.device.startActivity(component=componentname)

            MonkeyRunner.sleep(5)

            image = self.device.takeSnapshot()
           image.writeToFile(self.imageDir + 'screenshot_' + packagename  + str(self.count) + '.png','png')

            #Simulate Device Events Home, Back and Menu buttons

            self.device.press('KEYCODE_HOME', 'DOWN_AND_UP')
            MonkeyRunner.sleep(5)
            self.device.startActivity(component=componentname)
            MonkeyRunner.sleep(5)
            self.device.press('KEYCODE_BACK', MonkeyDevice.DOWN_AND_UP)
            MonkeyRunner.sleep(5)
            self.device.startActivity(component=componentname)
            MonkeyRunner.sleep(5)
            self.device.press('KEYCODE_MENU', MonkeyDevice.DOWN_AND_UP)

            log(self.logDir + 'test' + str(self.count) +'.log', self.device);

            self.device.removePackage(apk)

if __name__ == "__main__":
    main()

- Name this code as testDroid.py. 
- Put an apk in the folder \apks_to_test
- Start the emulator or Android device and then type the following command in prompt:  monkeyrunner c:\mokey\testDroid.py

The python program will run and it will execute the buttons tests.


May 8, 2014

Agile Practices to Help Mobile Testing

The growing trend to use mobile devices and software applications to everyday activities and communication resulted in popular use of mobile software applied to many kinds of devices platforms. There is the demand nowadays for agile and flexible testing procedures in the software companies. In this post I will describe an experience to implement some agile practices for mobile application testing.

Mobile Application Testing

Mobile Application Testing is testing using well-defined software test methods and tools to ensure quality in functions, behaviors, performance, and quality of service, as well as features, such as mobility, usability, interoperationability, connectivity, security, and privacy [1].
Mobile Application testing has some peculiarities that include connectivity, convenience (Quality of Design), supported devices (diversity of devices and OS), touch screens, new programing languages, resources constraints and context awareness.There are challenges for testing process mainly for test selection and test execution because of the rich contextual inputs and lack of test tools [2]. 
According to the technical literature, the important types of testing in mobile application are [3]:

· GUI Functional Testing: Functional testing ensures that the application is working as specified in the requirements. 

· Performance Testing: This testing is undertaken to check the performance and behavior of the application under certain conditions such as low battery, bad network coverage, low available memory, simultaneous access to application’s server by several users and other conditions.

· Memory and Energy Testing: memory leaks may preempt the (limited) memory resource, and active processes (of killed applications) may reduce the device (battery) autonomy.

· Interrupt Testing: An application while functioning may face several interruptions like incoming calls or network coverage outage and recovery. The different types of interruptions are: Incoming and Outgoing SMS and MMS, Incoming and Outgoing calls, Incoming Notifications, Battery Removal, Cable Insertion and Removal for data transfer, Network outage and recovery, Media Player on/off, Device Power cycle.

· Device multitude Testing: Testing an application on a multitude of mobile devices.

· Usability Testing: The usability verifies if the application has friendly interface, achieving its goals and getting a favorable response from users. This is the key to commercial success.

· Installation testing: This testing verifies that the installation process goes smoothly without the user having to face any difficulty. This testing process covers installation, updating and uninstalling of an application.

· Certification Testing: The mobile application needs to be tested against the guidelines set by different mobile platforms to be available in an application store, example: Nokia Test Criteria, IOS Testing criteria, Microsoft Test Criteria and etc.

These types of testing required impact the test activities during application development, especially in agile methodologies.

Experience: Project Description

The project X was a mobile application to help to navigate cities giving the access to scheduled departure/arrival times from public transportation.
The project followed the Scrum development framework and the software platforms used were J2ME programming language and Eclipse IDE. This project had two deliverables for different mobile platforms. The project team had 4 developers, one Scrum master, 1 Product Owner, 2 designers and 2 testers.
Initially, developers were responsible for code new features, unit tests and Continuous Integration (CI), and Testers should take care of functional tests, non-functional tests, and certification tests for mobile applications.

 Testing Process

The testing process was designed initially following the traditional sequential phases: Test Planning, Test Specification, Test Execution and Test Report. The test tools used were: Testlink to manage test cases, Jira to manage failures and J2ME Unit to automate unit tests. In the beginning of project the test team and design team were not allocated along with the development team despite belonging to the same company.
In early iterations of the project development, some problems were identified as: slow response to changes in the project requirements, poor and insufficient unit testing, little time to execute functional and non-functional testing and unstable application running into target device.
In order to solve these issues and accommodate agile development iterations with all types of testing required, the test process was revised and some agile practices were implemented as following:
·     Team Co-location: Testers, designers and developers allocated in the same room to join the team and encourage cooperation.
·     Pair programming: Developers and Testers together in pairs to implement unit testing to improve test coverage.
·    Pair Testing: Testers together to implement and execute exploratory tests and non-functional tests (Performance and Security).
·    Prioritization of regression testing: the regression testing was executed following of most critical features to be delivered (covering Certification Testing).
·     Designers involved in interface tests to identify unconformities of screen flow and screen design.
.

Fig. 1. Testing Process Activities

The Figure 1 shows the testing process activities designed for mobile applications. 
The programming skills tester did the pair programming to improve the unit testing.
The Test Execution includes the execution of functional tests, regression tests, performance tests, security tests, stability tests, regression tests, certification tests and interface (GUI) tests. If failures were found, they were reported to developers, once the failures were fixed, the test team executes the retest to validate the faults. When all critical failures were fixed the test report is generated and sent to the project team.


Results

Reaching the agile practices, the project team found the right way to work. The co-location improved the communication problem between testers, designers and developers.It also promotes fast feedback when changes were required. The pair programming solved the poor unit testing, since testers helped to improve the quality of scripts and test coverage. Running regression tests for prioritized features, involving designers in interface GUI tests and the pair testing approach made the cooperation of testing team happen. It allowed them to run all types of tests required for this mobile application on time. 

The table below shows the results comparison between a past application Project 1 with the traditional test process and the Project X that used the agile practices. Both projects had the same number of testers, developers and designers. The projects had 6 months of time to develop, test and delivery the application.The test team in Project X could execute more types of important tests for mobile applications and the project was delivered with stability and without failures, it was a really good result.


Projects
Number of Features
Failures Found and closed
Failures remain after delivery
Types of testing applied
Project 1 (Traditional Test Process)
9
442
9
   Unit Testing
   Functional Testing,
   Exploratory Testing
   Regression Testing
   Performance Testing
Project X (Agile practices applied to testing process)
11
388
0
   Unit Testing
   Functional Testing
   Regression Testing
   Performance Testing
   Security Testing
   GUI Testing
   Stability Testing
   Certification Testing
   Field Testing



References

   1.    Gao, J.; Xiaoying Bai; Wei-Tek Tsai; Uehara, T., "Mobile Application Testing: A Tutorial," Computer , vol.47, no.2, pp.46,55, Feb. 2014
doi: 10.1109/MC.2013.445.
22.    Kirubakaran, B.; Karthikeyani, V., "Mobile application testing — Challenges and solution approach through automation," Pattern Recognition, Informatics and Mobile Engineering (PRIME), 2013 International Conference on , vol., no., pp.79,84, 21-22 Feb. 2013 doi:10.1109/ICPRIME.2013.6496451
33.   Kumar, M.; Chauhan M.; “Best Practices in Mobile Testing”, White Paper Infosys, 2013 http://www.infosys.com/flypp/resources/Documents/mobile-application-testing.pdf.