Getting Started

This walkthrough shows you how to use App Functional to test a native Android or iOS app or mobile website. You use the App Functional console to create a project, upload a build in the form of an .apk or an .ipa or choose a default one, select devices, run a suite of standard tests, and then view the results.

Sign in to the Console

You can use the App Functional console to create a project, upload a build, select devices, run test suites, check results. You can learn about projects, builds, devices, test suites, results later in this walkthrough.

Step 01: Sign in to App Functional Console at http://demo-appfunctional.mozark.ai/

You will see a page as shown in Figure 01. Provide your user id, password and click on login to sign in to App Functional Console successfully.

Figure 01: Sign in to App Functional Console

Step 02: Signed in App Functional Console

After you successfully sign in to App Functional Console, you will see the page given in Figure 02.

Figure 02: Signed in App Functional Console

Once you are signed in to the App Functional Console, you can create or select existing projects, upload or select default or existing builds, upload or select scripts, select test suites, choose devices, run or schedule tests, check status of test runs, view results, configure your App Functional Console.

Configure Projects

To test a mobile app or a mobile website, you must create or select an app project.

Create App Project

If you are using App Functional for the first time, you will have to create a project by clicking on Create New App Project in Figure 03 or by going to Configure in Figure 04.

Figure 03: Click on Create New App Project to Go to Configure

• In the Create New App Project, enter a name for your project (for example, MyDemoProject) and description in Figure 04. Both Name App Project and Description are mandatory fields.
• Click Create App Project to create a project with the given project name and description in Figure 04.

Figure 04: Create New App Project

• Select App Project and you will see your newly created app project in the dropdown as shown in Figure 05.

Select App Project

If there are projects associated with your account, then select App Project as shown in Figure 05.

Figure 05: Select App Project

Configure Builds

Now that you have an app project, the next step is to upload a new build or select a default build or an existing build.

Upload New Build

If you are using App Functional for the first time, you will have to upload a build by clicking on Upload New Build or by going to Configure in Figure 06.

Figure 06: Upload New Build or Configure

• In Upload New Build, select an app project from the dropdown (for example, MyDemoProject) and attachment(.apk or .ipa) from your system in Figure 07. Both Select App Project and Attachment are mandatory fields.
• Click Upload Build to upload new build for an app project in Figure 07.

Figure 07: Upload Build

Select a Build – Default

If the builds are already available in the test devices, then select a Build.

The pre-requisite to select a default build as shown in Figure 08 is to ensure that the builds are available in the test devices. Otherwise, it will result into an error.

Figure 08: Select Build – Default

Select a Build

If there are builds associated with your projects, then select Build as shown in Figure 09.

Figure 09: Select Build

Please note:

Add a device on Apple developer portal In order to ensure that .ipa works on the iOS devices of your choice while testing on App Functional. The steps to add the same are as follows:

1. Login into apple developer account(https://developer.apple.com)
2. Select Certificates, Identifiers & Profiles option
3. On Left side click on “Devices” menu
4. Click on Blue “add” button on top
5. Choose platform (iOS,tvOS,watchOS)
6. Enter device name like “iPhone 12”
7. Enter UDID of that device(Contact MOZARK representative to get the UDID of the devices for your instance)
8. Then click on “Continue” Button
9. Now your device has been added successfully on apple developer account
10. Multiple devices can be added in the below format:

Device ID	Device Name	Device Platform
A123456789012345678901234567890123456789	Name 1	iOS
B123456789012345678901234567890123456789	Name 2	iOS
A5B5CD50-14AB-5AF7-8B78-AB4751AB10A8	Name 3	Mac
A5B5CD50-14AB-5AF7-8B78-AB4751AB10A7	Name 4	Mac

Configure Scripts

Now that you have select a build, the next step is to upload a script or select an existing script that can automate the user journey you want to test your mobile app or mobile website for.

Upload New Script

If you are using App Functional for the first time, you will have to upload a script by clicking on Upload New Script or by going to Configure in Figure 10.

Figure 06: Upload New Script or Configure

• In Upload Journey Script, enter a name for journey script, description, features tested, select framework, language, OS, project from the dropdown (for example, MyDemoProject) and attachment from your system in Figure 11. All the fields except selecting an OS are mandatory fields.
• Click Upload Journey Script to upload new script for an app project in Figure 11.

Figure 11: Upload Journey Script

Select a Script

If there are scripts associated with your projects, then select a script as shown in Figure 12.

Figure 12: Select Script

Select TestSuites

If there are test suites associated with your projects, then select a test suite or ALL test suites for running them on test devices as shown in Figure 13. TestSuites are not available for Appium with Java TestNG scripts.

Figure 13: Select TestSuite

Enter Test Description

If you want to provide a name for this test run, then write a test description as shown in Figure 14. It is not a mandatory field.

Figure 14: Enter Test Description

Select Devices

Now that you have selected project, build, script, test suite, test description, the next step is to choose devices to test your mobile app or mobile website.

Click on Continue as given in Figure 15.

Figure 15: Click on Continue

By clicking on Continue, you will see Select Devices page as shown in Figure 16.

You can select devices in two ways –

1. Select device group
2. Select devices

Figure 16: Select Devices

Select from existing device group

Select a device group of choice by clicking on the dropdown as shown in Figure 17.

Figure 17: Select Device Group

Select devices

You can also select devices by viewing each device in either List or Grid view, by going to different pages, by putting filters.

View – List

• Click on the list icon to enable the list view as given in figure 18.

Figure 18: Select Devices in List View

• Select by clicking on the check box in the list view as given in figure 18.

View – Grid

• Click on the grid icon to enable the Grid view as given in Figure 19.
• By hovering over the card, you will be able to see the device details as given in Figure 20
• Select by clicking on the check box in the grid view as given in Figure 21

Figure 19: Select Devices in Grid View

 

Figure 20: Hover over the device card

 

Figure 21: Select Devices in Grid View

Pagination

You can browse and select devices by going to different pages by clicking the arrows at the bottom of the list as given in Figure

Figure 22: Pagination

Figure 23: Pagination

Filter By

You can select devices further based on filters such as OS, Make, Model, Network, Resolution, Location as shown in Figure 24.

Figure 24: Filter By

Remove a Selected Device

By clicking on X, one can remove a selected device before beginning any test run as given in Figure 25.

Figure 25: Remove a selected device

Run Test

After selecting a project, build, script, devices, you can now run tests. There are two options to run tests. One is Start Now, where you can execute the test now and other one is Schedule Test where you can run continuous tests as shown in Figure 26.

Figure 26: Run Tests – Start Now, Schedule Test

Start Now

When ‘Start Now’ is selected, then the test starts instantaneously.

Figure 27: Start Now

Schedule Test

When ‘Schedule Test’ is selected, then the following fields are required to be populated as shown in Figure 28:

1. Start Date and Time
2. End Date and Time
3. Time interval (in minutes)
   Put sufficient interval between test runs based on test execution time

Figure 28: Schedule Test

Next Section: Reviewing Test Results

Reviewing Test Results

In this section, we will see on how to check the test results for different set of devices. One can navigate to ‘Check Status’ and ‘See Results’ sections to know more about the test results.

Check Status

Ongoing Test

Ongoing test displays test details such as the status, date & time, device, app project, build, script of the tests as given in Figure 29.

Figure 29: Ongoing Tests

In ‘Ongoing Test’, you can check for tests by searching for devices, by applying filters such as ‘app project’, ‘build’, ‘journey script’, ‘test case’, ‘status’, ‘date & time’ as given in Figure 30. A ‘Filter By’ field can be selected by clicking on the dropdown, selecting the required value and clicking ‘Apply’.

Figure 30: Ongoing Test: ‘Search for device’ and ‘Filter By’

‘Select Status’ ‘Filter’ option has following status options:

• Queued – tests that are queued to execute on test devices
• Running – tests that are running on the test devices
• Completed – tests that are executed
• Interrupted – tests that did not complete but were aborted

Ongoing Test also facilitates the cancellation of any previous test runs based on their status by giving an option to Abort the test as given in Figure 31.

Figure 31:Abort

Test Run History

‘Test Run History’ displays test details such as date & time, device, app project, build, script and status as shown in Figure 32.

Figure 32:Test Run History

In ‘Test Run History’, you can check for tests by searching for devices, by applying filters such as ‘date & time’, ‘device’, ‘app project’, ‘build’, ‘script’, ‘status’ as given in Figure 33.

Figure 33: Search for device and Filter By

A ‘Filter By’ field can be selected by clicking on the dropdown, selecting the required value, and clicking ‘Apply’ as given in Figure 33.

‘Test Run History’ also facilitates the deletion of any test run history by selecting the check box and clicking on ‘Delete’ as given in Figure 34.

Figure 34: Test Run History – Delete

‘Status’ column in ‘Test Run History’ displays the following messages if the tests fail:

• BUILD NOT FOUND ON DEVICE – When the user chooses ‘default’ as the option for specifying build, but no build present in the device.
• DEVICE NOT AVAILABLE – When the device chosen by a user is not available for tests.
• DEVICE GETS DISCONNECTED WHILE TEST IS RUNNING – When the device gets disconnected while the script is executing.
• PLAN IS EXPIRED FOR THAT USER – When the plan purchased by the user is expired.
• MAX RERUN LIMIT REACHED – When the maximum test rerun limit is reached by the user.

By clicking on ‘Test Run Details’, you can get detailed test run information in the form of video recordings, screenshot recordings, test results, test logs, live logs as given in Figure 35.

Figure 35: Test Run Details

You can view ‘Screenshot Recording’ by dragging the cursor or clicking on left and right arrows as given in Figure 36. The cursor shows the time stamp of a test run.

Figure 36: Screenshot Recording

‘Test Run Details’ show the following results:

• Test Run Details – These are the details about the test runs.
• Video Recording – Video of the tests taken as instructed by the script.
• Screenshot Recording – You can check the screenshot recording by clicking on the arrows given in Figure 36.
• Test Results – The test results such as passed or failed are displayed testcase wise.
• Test Logs – Various logs such as ‘Download Logs’, ‘Download Memory Info Logs’, ‘Download CPU Info Logs’ are generated.
• Screenshots – Snaps of the tests taken as instructed by the script.
• Live Logs – Logs are generated when tests are running.

Please note:

• To record video and capture screenshot, appropriate code should be added to the test script before uploading the script to App Functional. Get in touch with the team for further assistance by sending an email to enquiry@mozark.ai
• For Appium with Java TestNG scripts, you have to import the package org.testng.Assert and call method Assert.assertTrue(true) and Assert.assertTrue(false) to print pass and fail in the test results.

By clicking on ‘Rerun Test’, you can run the failed testcases as given in Figure 37.

Figure 37: Rerun Test

Manage Schedules

In ‘Manage Schedules’, you can ‘View Tray’, ‘Edit Schedule’, ‘Delete’ continuous tests scheduled by you as given in Figure 38.

Figure 38: Manage Schedules

View Tray

Click ‘View Tray’ in Figure 38 to view the devices in that tray as given in Figure 39.

Figure 39: View Tray

Update Schedule Test

Click ‘Update Schedule Test’ in Figure 38 to update ‘start date & time, ‘end date & time’, ‘time interval’ for your continuous tests as given in Figure 40.

Figure 40: Update Schedule Test

Delete

Click ‘Delete’ in Figure 38 to delete a test schedule as given in Figure 41.

Figure 41: Delete

See Results

In ‘See Results’, you can get insights from the test runs by using ‘Set Filters’, ‘Choose X-Axis’, ‘Choose Chart Type’, and ‘Advanced Dashboard’ as given in Figure 42.

Figure 42: See Results

Set Filters such as ‘Select App’, ‘Select Build’, ‘Select Journey Script’, ‘Select Test Cases’ to customize the insights from the test runs. In Figure 43, different test cases are selected by clicking the check boxes to see results for the selected test cases.

Figure 43: Set Filters

Click the dropdown given for ‘Choose X-Axis’ to select ‘Build’, ‘Script’, ‘Testcase’’, ‘OS’, ‘Make’ to customize the insights from the test runs. For example, a testcase is selected by clicking the dropdown to see results for the given testcases.

Click the dropdown given for ‘Choose Chart Type’ to select ‘Bar’, ‘Line’ to customize the insights from the test runs.

Click ‘Advanced Dashboard’ in Figure 44 to view test run details. In Figure 44, you can apply filters to check the test run details for filtered test runs.

Figure 44: Advanced Dashboard

Next Section: Scripting Guidelines

Scripting Guidelines

The scripting guideline is to help users write/modify their automation scripts to be compatible with MOZARK App Functional platform. The Language-Framework supported on the platform are: 

1. Robot-Appium with Python 
2. Robot-UIAutomator with Python 
3. Appium with Java TestNG

Robot-Appium

This section describes how to configure, package, and upload your Robot-Appium tests to App Functional.  

Please refer to our sample build and sample scripts for Robot-Appium with Python:

Android(Build): View on BitBucket

iOS(Build): View on BitBucket

Android(Mobile Website): View on BitBucket

iOS(Mobile Website): View on BitBucket

Use the below instructions to get started with Robot-Appium scripting on App Functional. 

1. Ensure that the Test Package conforms to below folder structure

You will have to create a particular folder structure for your test package. Refer to our sample folder given in Figure 45. This folder structure will ensure that all the dependencies and packages are available.

Figure 45: Folder Structure

Please note:

To ensure the appropriate setup for your Robot-Appium scripts while uploading your test package on App Functional, generate requirements.txt and bundle it inside your test directory/package as given in Figure 45. You can run the below command to generate requirements.txt:

The following code snippet is a glimpse of requirements.txt:

2. Configure your test package

Replace all hard-coded variables in the config.py file, such as Device UDID, Appium Server IP etc with environment variables, to ensure compatibility. Environment variables represent values that are used by your automated tests. App Functional dynamically populates environment variables at runtime by giving the values to the required environment variables.

• Go to MyScriptTestDirectory/VariableFiles/Config.py
• Add code snippet – import os
• Update the below capabilities with the mentioned environment variables 

• Add supported environment variables or desired capabilities (Supported variables are a set of desired capabilities that can be used while writing testcases.)

Please note:

• Put username and password for GMAIL login, Facebook login as part of the script if there is any testcase that requires such credentials.
• Rectify all the build errors and missing dependencies before bundling and uploading your test package to App Functional.

3. Upload the test package(scripts) to App Functional

• Sign in to App Functional console: http://demo-appfunctional.mozark.ai/
• Create or select a project. Follow the instructions on how to create or select a project.
• Upload or select a build. Follow the instructions on how to create or select a project.
• Bundle your test package
• Upload the test package. Follow the instructions on how to upload a test script.

Please refer to our sample build and sample scripts for Robot-Appium with Python:

Android(Build): View on BitBucket

iOS(Build): View on BitBucket

Android(Mobile Website): View on BitBucket

iOS(Mobile Website): View on BitBucket

Robot-UIAutomator

This section describes how to configure, package, and upload your Robot-UIAutomator tests to App Functional. The UIAutomator testing framework provides a set of APIs to build user interface tests that perform interactions on user and system apps for Android.

Please refer to our sample build and sample scripts for Robot-UIAutomator with Python for Android(Build): View on BitBucket

Use the below instructions to get started with Robot-UIAutomator scripting on App Functional.

1. Ensure that the Test Package conforms to below folder structure

You will have to create a particular folder structure for your test package. Refer to our sample folder given in Figure 46. This folder structure will ensure that all the dependencies and packages are being taken care of.

Figure 46: Folder Structure

Please note:

To ensure the appropriate setup for your Robot-Appium scripts while uploading your test package on App Functional, generate requirements.txt and bundle it inside your test directory/package as given in Figure 46. You can run the below command:

The following code snippet is a glimpse of requirements.txt:

2. Configure your test package

Replace all hard-coded variables in the config.py file, such as Device UDID, Appium Server IP etc with environment variables, to ensure compatibility. Environment variables represent values that are used by your automated tests. App Functional dynamically populates environment variables at runtime by giving the values to the required environment variables.

• Go to MyScriptTestDirectory/VariableFiles/Config.py
• Add code snippet – import os
• Update the below capabilities with the mentioned environment variables

• Add supported environment variables or desired capabilities (Supported variables are a set of desired capabilities that can be used while writing testcases.)

Please note:

• Put username and password for GMAIL login, Facebook login as part of the script if there is any testcase that requires such credentials.
• Rectify all the build errors and missing dependencies before bundling and uploading your test package to App Functional.

3. Upload the test package(scripts) to App Functional

• Sign in to App Functional console: http://demo-appfunctional.mozark.ai/
• Create or select a project. Follow the instructions on how to create or select a project.
• Upload or select a build. Follow the instructions on how to create or select a project.
• Bundle your test package
• Upload the test package. Follow the instructions on how to upload a test script.

Please refer to our sample build and sample scripts for Robot-UIAutomator with Python for Android(Build): View on BitBucket

Appium Java TestNG

This section describes how to configure, package, and upload your Appium tests to App Functional. Appium is an open-source tool for automating native and mobile web applications. For more information, see Introduction to Appium on the Appium website.

Please refer to our sample build and sample scripts for Appium with Java TestNG 
 
Android(Build): View on BitBucket

iOS(Build): View on BitBucket

1. Ensure that the Test Package conforms to below folder structure

You will have to create a particular folder structure for your test package. Refer to our sample folder given in Figure 47. This folder structure will ensure that all the dependencies and packages are available.

Figure 47: Folder Structure for Appium with Java TestNG Scripts

Please note:

• Add Maven dependencies and builds to pom.xml as given in the folder structure in Figure 47.

Add dependencies to zip.xml as given in the folder structure in Figure 47.

2. Configure your test package

Step 01: Write the environment variables or desired capabilities to MyTest.json

Environment variables represent values that are used by your automated tests. You can use these environment variables in your MyTest.json and test code. App Functional dynamically populates environment variables at runtime by giving the values to the required environment variables.

Please note:

• The desired capabilities – ‘xcodeSigningId’, ‘xcodeOrgId’, ‘bundleId’ must be added for iOS devices and not for Android devices.

Step 02: Add the filepath of MyTest.json to an environment variable

• Open terminal(MacOS/Linux) or command line(Windows)
• Initialize “DESIRED_CAPABILITIES_FILE_PATH” with the filepath of MyTest.json by executing the below command:

For terminal (MacOS/Linux):

For command line (Windows):

• Open Eclipse app from terminal (MacOS/Linux) or command line (Windows)
• Use the below method to BaseClass.java to read the filepath of MyTest.json file from an environment variable in the folder structure:

Step 03: Create a zipped test package file

The target directory file path will be: java_testng_appium_android/target/java_testng_appium_android.zip

The sample file name of .zip file will be(in the case of 5GMark): java_testng_appium_android.zip

The file with .zip extension will be created as a result. This is your test package.

Please note:

• Put username and password for GMAIL login, Facebook login as part of the script if there is any testcase that requires such credentials.
• Rectify all the build errors and missing dependencies before bundling your test package.

Step 04: Bundle to create your zipped test package file for App Functional

Now, create a test bundle for App Functional by zipping MyTest.json and .zip file created in previous two steps as MyTestPackage.zip to upload your tests to App Functional.

3. Upload the test package(scripts) to App Functional

• Sign in to App Functional console: http://demo-appfunctional.mozark.ai/
• Create or select a project. Follow the instructions on how to create or select a project.
• Upload or select a build. Follow the instructions on how to create or select a project.
• Upload the MyTestPackage.zip as test script. Follow the instructions on how to upload a test script.

Please refer to our sample build and sample scripts for Appium with Java TestNG 
 
Android(Build): View on BitBucket

iOS(Build): View on BitBucket

TestNG Cucumber Appium with Java

This section describes how to configure, package, and upload your TestNG based Cucumber-Appium with Java tests to App Functional.

Please refer to our sample build and sample scripts for TestNG Cucumber using Appium with Java.

Android(Build): View on BitBucket

1. Ensure that the Test Package conforms to below folder structure

You will have to create a particular folder structure for your test package. Refer to our sample folder given in Figure 48. This folder structure will ensure that all the dependencies and packages are available.

Figure 48: Folder Structure for Appium with Java to support TestNG Cucumber Scripts

Please note:

• Add Maven dependencies and builds to pom.xml as given in the folder structure in Figure 47.

Add dependencies to zip.xml as given in the folder structure in Figure 47.

2. Configure your test package

Step 01: Write the environment variables or desired capabilities to MyTest.json

Environment variables represent values that are used by your automated tests. You can use these environment variables in your MyTest.json and test code. App Functional dynamically populates environment variables at runtime by giving the values to the required environment variables.

• Open Eclipse app from terminal (MacOS/Linux) or command line (Windows)
• Use the below method to BaseClass.java to read the filepath of MyTest.json file from an environment variable in the folder structure:

Step 03: Create a zipped test package file

The target directory file path will be: java_testng_cucumber_appium_android/target/Aquamark.zip

The file with .zip extension will be created as a result. This is your test package.

Please note:

• Put username and password for GMAIL login, Facebook login as part of the script if there is any testcase that requires such credentials.
• Rectify all the build errors and missing dependencies before bundling your test package.

Step 04: Bundle to create your zipped test package file for App Functional

Now, create a test bundle for App Functional by zipping MyTest.json and .zip file created in previous two steps as MyTestPackage.zip to upload your tests to App Functional.

3. Upload the test package(scripts) to App Functional

• Sign in to App Functional console: http://demo-appfunctional.mozark.ai/
• Create or select a project. Follow the instructions on how to create or select a project.
• Upload or select a build. Follow the instructions on how to create or select a project.
• Upload the MyTestPackage.zip as test script. Follow the instructions on how to upload a test script.

Please refer to our sample build and sample scripts for Appium with Java to support TestNG Cucumber framework.

Android(Build): View on BitBucket

Next Section: Limits

Limits

The following list describes current App Functional limits:

• The maximum file size of an app that you can upload is 4GB.
• The maximum file size of a test bundle that you can upload is 4GB.
• There is no limit to the number of projects that you can upload to test.
• There is no limit to the number of apps that you can upload to test.
• There is no limit to the number of test bundles that you can upload to test.
• There is no limit to the number of devices that you can include in a test run.
• There is no limit to the number of runs that you can schedule.
• There is no limit to the duration of an automated test run.
• You can retain the test results for one month.
• You can retain test logs for one month.
• You can retain screenshots and videos for one month.

Next Section: Document History

Document History

S.No.	Date	Version No.
1	Dec 8, 2020	1.0

Next Section: Resources

Resources

This section contains links to product video tutorials that can help you in getting started with App Functional.

Product Videos [YouTube Channel]

1. How to run an automated test?

1. How to write scripts and add log results?

Next Section: Support

Support

For further assistance, you can reach out to enquiry@mozark.ai.