Mobile Test Automation

Before we start, let me make some confession first. As a matter of fact, I've never been into serious testing business in my career. From the very beginning I've been working as a programmer whose job is to code, not test. But like any other coder in the world, I have to take my codes to a test drive myself before creating the pull request in GitHub or delivering the end product to the designated people for conducting the test. In this sense, every coder is a QA tester to some extent. Having some skill on this arena surely comes in handy for a programmer and can be considered an added bonus to the expertise.

In software development lifecycle, testing is as important as software designing and coding. Functionality, usability and consistency of a stable application are just as vital. All these elements boil down to a great user experience (UX). Testing features, providing inputs and looking for expected outcomes manually can be very laborious when done repeatedly and may take hundreds, if not thousands of hours. For rapid development and fast delivery of the product, testing needs to be quick as well. And here comes test automation to the rescue.

Test automation is a process to control the execution of tests and the comparison of actual outcomes with predicted outcomes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or perform additional testing that would be difficult to do manually. It is critical for continuous delivery and testing, thus saving a tremendous amount of time in development lifecycle.

There are a number of automation platforms for testing mobile applications out there each having their own pros and cons. A handful of them are open source and some have cross-platform support. Appium (Android and iOS), Robotium (Android), Selendroid (Android), Keep It Functional (iOS), Monkey Talk (Android, iOS) are some of the big players in this game.

These automation tools let you write the test scripts in their respective language. In this script you write down all the instruction or predefined steps for the automator which will simulate human interaction on your app and log messages depending on success or failure of some particular task. Once you write the code, all you need to do is choose the target application on the simulator (for iOS) or emulator (for Android) and run the test. The automator will automatically run the test for you while you can relax and binge-watch the latest season of Game of Thrones. You can come back later and see the test log to figure out if the test passed or failed at some point.

Luckily, as an iOS developer, I have a very handy tool under the hood of Xcode for UI automation. UI Automation is a tool that Apple provides and maintains for higher level, automated, testing of iOS applications. Tests are written in JavaScript, adhering to an API defined by Apple.

Writing tests can be made easier by relying on accessibility labels for user interface elements in your application. Don’t worry though, if you don’t have these defined, there are alternatives available.

UI Automation tests are run from the Automation instrument within the Instruments tool that comes with Apple's developer tools. The tests can be run in the iOS Simulator or on a physical device.

Writing UI Automation Tests

Step 1: Open the Sample Project

Here's a sample project for demonstrating the UI automation tool of Xcode. Open the project and run the application to make sure that everything is working as expected. You should see a user interface similar to the one shown below.

Before we write any tests, feel free to try out the sample application to become familiar with its functionality. As a user, you can enter text in the text field and tap the button to see a label on the screen that displays the reversed input string.

Step 2: Create a UI Automation Test

Now that you’re familiar with the sample application, it's time to add a UI Automation test. UI Automation is a tool that can be found in Instruments. To run the sample application in Instruments, select Product > Profile from Xcode's menu. Select Automation from the list of tools.

The main Instruments window will open with a single instrument ready to run, the Automation instrument (the Automation instrument executes UI Automation test cases). You'll also see an area in the bottom half of the window that looks like a text editor. This is the script editor. This is where you will write your UI Automation tests. For this first test, follow the below instructions, adding each line to the script in the script editor.

Start by storing a reference to the text field in a variable.

var inputField = target.frontMostApp().mainWindow().textFields()["Input Field”];

Set the text field's value.

inputField.setValue("hi”);

Verify that the value was set successfully and, if it was, pass the test. Fail the test if it wasn't.

if (inputField.value() != "hi") UIALogger.logFail("The Input Field was NOT able to be set with the string!");
else UIALogger.logPass("The Input Field was able to be set with the string!");

While this test is fairly trivial, it does have value. We've just written a test that tests the presence of a text field when the application is launched and that tests if a random string can be set as the text field's value. If can remove the text field from the storyboard and run the test just to see how it fails.

This test demonstrates three important pieces of writing UI Automation tests. First, it shows you how to access a simple user interface element, the text field. Specifically, we access a dictionary of all text fields on the base view of the application via target.frontMostApp().mainWindow().textFields() and we then find the text field we are interested in by looking for the one with key Input Field. This key is actually the accessibility label of the text field. In this case, it's defined in the storyboard. We can also set the accessibility label in code using the accessibilityLabel property on NSObject.

Step 3: Saving Tests

While writing tests in the script editor is convenient, it quickly becomes cumbersome and difficult to maintain. If you quit Instruments, any unsaved changes are discarded. We need to save the tests we write. Simply copy and paste your test into a new document in your favorite text editor and save it.

To run the test, select the middle tab in the pane on the right, next to the script editor, and select Add > Import.

You'll be prompted to select the script to import. Navigate to the saved script and import it. You can still change the script in the script editor. Any changes will be automatically saved in the external file you created.

Step 4: Tapping a Button

Let's update our test to test interaction with the button. Our test already adds text to the text field so we only need to add code to tap the button.

target.frontMostApp().mainWindow().buttons()[0].tap();

Step 5: Verify the Jumbled String

As I mentioned earlier, our application takes a string of text as input, and, when the user taps the button, displays the reversed string. We need to add one more test to verify that the input string is properly reversed. To verify that the UILabel is populated with the correct string, we need to figure out how to reference the UILabel and verify the string it displays. This is a common problem when writing automation tests, that is, figuring out how to reference an element in the application to make an assertion on it.

There is a method on nearly every object in the UI Automation API, logElementTree. This method logs the nested elements of a given element. This is very useful to understand the hierarchy of elements in the application and helps to to figure out how to target a specific element.

Let's see how this works by logging the element tree of the main window. Take a look at the following line of code.

target.frontMostApp().mainWindow().logElementTree();

Adding this line to the test script results in the following output:

Recording a session

You can also record a session. Click the red dot like that shown below to start recording:

Instruments will launch your application in the iOS Simulator and you'll be able to interact with it. Instruments will generate a script based on your interactions in real time. Give it a try. Rotate the iOS Simulator, tap at random locations, perform a swipe gesture, etc. It's a really useful way to help explore the possibilities of UI Automation.

In this way you can write script for any test case to conduct an automatic test run on your app. This post was meant for the curious minds to play around with the UI automation tools and to demonstrate a way to do it. Please feel free to experiment with other automation tools in the market. The aforementioned Monkey Talk is a very good platform for the starters. It's very easy to use and free to download. You can get it right here.

Happy testing.