Test Automation of Mobile based Application

ABSTRACT:

Gone are the days when people used to fill the forms manually, scan it and send it by Email or visit concerned offices and stand in a long queue to get their jobs done or call customer care services to get the work done. Digitalization makes people's life easy. Web based Applications helps people to a certain extent. Be it transferring money or booking a cab or reserving seats in a Hotel, people could do it online using these Web bases applications. But could they use it whenever they wanted it? Of course not. These websites always support Laptop/Desktops, rarely supports Mobile. As a result, people were not able to use it whenever they want as it needs them to be in front of the laptop/Desktop. The websites which support Mobile devices are not 100% user friendly. It opens in the Web browser of the devices and UI is not always good and all the functionality supported in Laptop/Desktop was not supported in the Mobile version. Revolution came into people’s life when Smart Phones made their entry into the market. Cab Booking, Online shopping, Online Banking, and Entertainment, Making reservations, Online Games, etc. are now on everyone’s fingertip. Thanks to Native, Hybrid, and Web-based Mobile App. In addition, B2B or enterprise apps, which facilitate interaction with back end tools and mobile devices by means of web services or cloud platforms, are capturing market share.

With increasing competition to provide the best quality Mobile app that supports all OS and almost all devices, Mobile App Developing Companies are now on immense pressure. To thrive in the new world of digital, companies need to be faster and smarter, finding new ways to reach customers at every touchpoint with good quality Mobile Apps.

The Mobile Apps User Interface should be good and its functionality should work as expected. To ensure the quality of the mobile app, one needs to perform System Testing. To check the regression effect due to the addition of a new feature, regression testing needs to be performed on a regular basis. These tests need to be performed in all supported devices and in different OS versions to check the compatibility of the mobile app along with the desktop-based application. To achieve this either one needs to invest more in time and cost. To reduce time and cost, Mobile App Testing can be automated.

This white paper provides an idea about Mobile Automation, Tool Selection, Test Approach, Challenges, and its solution.

Introduction:

In today’s competitive world delivering high-quality, the robust products has become a necessity. Test Automation is the answer to high quality, effective, quicker, and cheaper delivery. Automated software testing is a cost-effective solution that increases testing efficiency. The process of testing a Mobile based application for its functionality and usability is called Mobile application testing. When the task is performed with the help of an automation tool then this process is called Mobile Automation.

 In today’s Smartphones world Automation of Mobile based application is highly demandable. Automating Web-based /Desktop based application in PC/Laptop is easy using tools like Selenium, UFT, etc. These tools get the application objects through their properties and perform an action on them. But these tools can’t be used for automating Mobile devices as they can’t identify mobile app objects. Also, the GUI of the app varies with devices, size of the device, orientation of the app within the device, type of app. It makes Mobile automation more complex as the probability of varying Object properties with the variance of GUI is more.

 This article gives an idea about challenges in Mobile Automation, appropriate tool selection, Testing Approach for Mobile app testing, an illustration of Mobile automation of SFDC Application in the IOS environment.

Types of Mobile App:

Mobile apps are of 3 types

• Native App
• Hybrid App
• Web App

Mobile Testing Approach:

Mobile Testing is performed for

• Functionality Testing
• Usability Testing
• Regression Testing
• Non Functional Testing like Security Testing, Performance Testing

Functional Testing: Mobile app functionality is tested usually in primary devices or in all supported devices. This is performed to verify if App functionality is working as desired or not. This can be tested manually or by a Mobile Automation testing tool. Functionality Testing can be performed using an Emulator or Simulator or Real Device.

Usability Testing: Usability testing is performed to test the GUI of the apps is as per the design. This type of testing is mostly performed manually in all supported devices, Browsers, and OS. If it is automated then it is performed while testing Functional testing by using the checkpoint. Usability Testing can be performed using an Emulator or Simulator or Real Device.

Regression Testing: Regression testing is needed to check the regression effect on the app due to an update or any changes. This is performed by a Mobile Automation testing tool in all supported devices, OS, and browsers. It can be performed using an Emulator or Simulator or Real Device.

• Emulator: An Emulator is used as a substitute for real devices. It tries to duplicate the inner workings of the device. It’s hosted in the cloud by different Automation testing tool providers. Since it behaves like real devices, a tester can automate it using an appropriate automation testing tool, takes screenshots. It also can be used for SMS and calling features if required for testing. It’s cost-effective as real devices are not needed to be procured.

   During the time of testing only these cloud-based devices can be procured for limited time periods. Emulators can be best used for SIT. The main        disadvantage of emulators is

• One needs to be dependent on the mobile testing tool providers who provide the cloud-based devices, for the installation of any new app that is not present in the device.
• If the device is not hosted in the cloud or Supported OS version device is not present in the cloud one needs to look for another mobile testing tool that hosts real devices in the cloud.
• Simulators: Simulators replicate different apps. It is usually provided by the device manufacturer or Browser extension. One can open the specific app simulator in Laptop/PC’s native browser and select 

 the size of the device and can start testing the app. If the tester wants a native browser to simulate a Mobile browser, then he/she can change the user settings in the native browser.

This can be automated using any functional testing tool like UFT, Selenium. Testing can be done without procuring the device. Hence it’s cost-effective. It’s faster in terms of execution. The main disadvantage of using a simulator is

• In the case of test failure debugging is difficult as it won’t behave completely like a real device. It does not mimic the hardware.
• Real Devices: Actual Devices are used for testing. So one needs to procure all the supporting devices. So it’s a costly method of testing. Since real devices are used, results are accurate, and debugging is possible in case of test failure.

Challenges in Mobile Automation:

The hurdles one would face while automating Mobile Application are as below:

• Variation of Devices, OS, and App type: One need to test the Mobile App is in different devices of different manufacturers and sizes, in the different OS of multiple versions, in hybrid type App, in a different orientation. So App object properties may vary with all the above variation which may cause tests to get failed. Also if the app is native type, the object property would vary OS to OS as a native app is developed for a single platform. As a result, the tester might end up writing different sets of code for the different OS which would be a more time taking process.
• Tool Support: In the market different Mobile automation testing tools available. But they always don’t support all the Mobile Platforms from a single machine. E.g. if one need to automate Mobile App using Appium in windows based PC, they can only automate Android OS based mobile device. If one uses a Mac-based PC, he/ she can automate both android and IOS based mobile devices but not windows based.
• Object Identification: Many times Object added to the object repository or Object spied are not getting highlighted even though the device app is open. Object Class is not identified properly

E.g. even if the object has text property, the SetToProperty method fails. Text Box is identified as MObject instead of MText

• Emulators not always support all app and Device and OS version: As mentioned earlier Emulators are real devices hosted in the cloud by the different Tool manufacturers. So tester needs to be dependent on the Tool manufacturer if they are looking for a specific device or the latest OS version. Many devices may not have carriers so if an SMS or call facility needs to be used it can’t be done. Again Emulators can’t be used for accessing the Camera of the device. If the tester wants to test a particular Mobile App in the Emulator and if it’s not present then he/she needs to install the app in Emulator which is not always possible due to Emulator's Wi-Fi Connectivity issue or due to the device Password issue. So Tester has to connect to the Tool manufacturer to install such an app.
• Real Device Cost: Real Device are costly which make Mobile testing costly
• Mobile Automation Tool Cost: Very few Open source tools are available in the market and most of the Mobile Automation Tools are paid one which makes Mobile Automation testing costly.

Mobile Automation Testing Tool Selection:

There are many Mobile Testing Tools are available in the market. Below are the few names of such Mobile Automation testing tool.

• Perfecto Mobile
• Appium                       
• Eggplant
• FoneMonkey
• SeeTest
• Ranorex

When one selects Mobile Automation Testing Tool he/she needs to consider his Project requirement that is satisfied most by the tool. Below are the following factors one needs to be considered in general while selecting Mobile Automation Tool.

• Cost: The cost of the tool should be nil or very minimum. It should not be very costly to the project. Open-source tools like Appium etc. are available in the market.
• OS Support: The tool should be supporting almost all Mobile OS like IOS, Android, Windows, Symbian OS etc. from both windows based PC/Laptop and Mac-based PC/Laptop. E.g. Eggplant, SeeTest
• One Script for All: The tool should allow users to develop only one script that supports all Devices, all supported App types, all supported type of Orientation (Portrait and Landscape), all supported OS and OS version. E.g. SeeTest, Egg Plant
• Functional IDE support: Usually the Mobile Automation Tool has its own IDE to develop the script. But the tool can be more beneficial if It had Add on /Plugin for Functional Automation Testing Tools like Selenium, HP UFT, Test Complete so that users can generate the same script for different functional automation testing tools and us them for execution. Tools like Perfecto Mobile, SeeTest can have Selenium, QTP plugin.
• Emulator and Real Device Support: The Mobile Automation testing tool should support both the real device and Emulators. When the Tool host the Emulators in Cloud-based lab it should cover all most all Device and OS in their Older and latest version. The Tool should share the Device passwords while procuring the Tool so that users can install the app by themselves. The Emulators should have Wi-Fi Connectivity and carriers. E.g. of such tool are Eggplant, SeeTest
• Full Control: The tool should provide full control over the device. E.g., SeeTest, Appium EggPlant, etc.
• Easy Maintainability: The Script should be easy to maintain and one needs not to spend much effort on the tool. E.g. Ranorex, Appium, Perfecto Mobile, SeeTest, Eggplant.

Mobile Automation of SalesForce1 Application in iOS Environment: An Illustration

Below is the case study that has been performed by the author for a small scenario of the Saleforce1 application in IOS Environment.

Salesforce Development: These days Salesforce Development Application is more popular for Cloud-based Application development and Mobile based application development. It provides the package to the developer which he/she can customize based on the project need. The mobile application of salesforce is known as Salesforce1. It is supported in both Android and iOS.

 SeeTest Tool: SeeTest is a Mobile Automation tool by Experitest. It supports all the platforms like Android Device, iOS Device, Windows Phone Device, BlackBerry Device, Windows Mobile Device, and Symbian Device. It supports both Emulators hosted in the cloud and Real Devices. It’s a licensed tool and has Plugin for HP UFT, Selenium. The author has automated Saleforce1 in iOS 9.3.3 in iPhone 6 using SeeTest Add on for UFT.

Once SeeTest is downloaded and installed, it will appear as Mobile Tab in the toolbar.

To add a device that needs to be tested, one needs to click on Mobile -> Mobile Device Manager-> and Select the Type of Device that needs to be added. If the device is plugged into the system, the tool can pull out all its details like UUID, OS type, etc. as shown below

Once the Device is added, one needs to add Configuration Profile. The tester needs to go http://denabled.com/maple/profileRequests/new?deviceUDID and fill the details. It’s a third party website which provides Device Enablement File(.def File). Click on Mobile->Configuration Profile and browse the place, the def file is saved and Import It

The SeeTest Tool has its own spy feature. One can click on the spy icon (the 3rd Icon from left ) and Copy the Xpath of the object and Add the Object into Object Repository of the UFT Tool by creating a new Object.

Once the Object is added, the appropriate Operation can be performed on it. One may even write a Descriptive Program for the Object without adding the object into the Object Repository. The Object Properties can also be spied through Object Spy of UFT. Once scripting is completed, The script can be Executed from UFT. The script would be executed on a Real device. In case of any defect, the Tester can log defects directly in ALM which is integrated with UFT.

Conclusion:

In today’s Smartphone world, Mobile Automation is a necessity. It facilitates faster execution, Cost-effective solution, Multiple Platform support. As a result, delivery is quicker with better quality and less costly.

About the Author:

Sonali Das is a seasoned Test Automation Professional willing to explore new technology for the betterment of Software QA and its effort reduction. She has over 9.8 years of experience in the Testing domain, serving clients spanning various industries in different locations. She has worked in a variety of testing like Automation testing, SAP Automation, Web services Automation, Functional Testing, Mainframe testing(CICS online), Database Testing, Mobile Application Testing, Product Testing, Performance Testing, SFDC Testing. She has experience in working in a waterfall, V, and Agile Model. She has worked both in onsite and offshore.