Skip to main content
Test Automation User Guide
Doris Sooläte avatar
Written by Doris Sooläte
Updated this week

Introduction

This document is intended to be used by Scripters in order to get directions about script creation.

This document starts with a Test Automation Process Flow (next image). The Scripters should follow all the steps for successful script creation and a consequence for a successful automated run.

  1. Environment setup

    1. in this step, you will configure your environment with all you need.

    2. if you need assistance you can contact the coordinator anytime by opening a ticket

  2. Script development

    1. write the requested scripts using our guidelines

  3. Script validation in Scripter's local machine

    1. validate first your script in your local machine

  4. Sandbox Package Run

    1. you can use our sandbox to validate the scripts, by creating a run and waiting for the results. Depending on the results you can see if the script needs any fixes or it's ready

    2. if no error is verified in the prior step, please send the test package to the coordinator

    3. f you need assistance you can contact the coordinator anytime by opening a ticket

  5. Script validation Testlio

    1. in this step, the coordinator will execute it in PRD

  6. Script acceptance

    1. if no error is detected your script development has been accepted!

If you have any suggestions, related to this document please feel free to give the suggestions to the coordinator anytime.

Have good work!


Test Automation Process Flow

This is a visual representation of the needed steps for successful script creation.

Need Help? Need to contact the coordinator?

If you need any help or need to contact the coordinator for any reason, please use this Google Form.

Guidelines

Please read and understand this provided user guide step by step. Also, notify your coordinator through Google Form if additional libraries other than listed are required. For any blockers please submit a ticket through Google Form.

  • Provided below ‘Comment section at the very top of the code with details:

    /**
    * Scripter's Name:
    * Scripter's Email:
    * Script Creation Date:
    * Additional Libraries Used and Version:
    * Test Case Name:
    * Test Case Description:
    */
  • Add proper comments for a better understanding of logic/implementations in code.

  • Avoid hardcoded data in the script. Declare all variables and respective values at the beginning of your script.

  • If you’re navigating to some views directly, do not hardcode URLs in code, but use constants instead

    • In Java you can use com.testlio.constants.WebAppUrls class for such purposes:

      testpath=suite.xls
      testlibrary=AppObjectsRepository.xls
      reportsType=2
      APKPATH=Simika_School_Management_App_v1.0_21.apk

      … and then you can use it in tests like that:

      ConfiguratorSupport configProps = new ConfiguratorSupport("Environment.properties"); configProps.getProperty("AndroidAppiumURL");

  • Use the ‘Testsuite.xls ’ file to add the project, test cases, test data details explained in the user guide.

  • Use ‘AppRepository.xls ’ file to add object properties details explained in the user guide.

Creating a PR

  • Scripters need to branch out staging branch for creating new test scripts or maintaining existing scripts

  • New branch should be named with test case id, such as Test-43

  • Maximum number of new test scripts, is 2 in a single PR

  • Changes in PR should be linear, i.e. only changes from the staging branch can be merged onto the branch you are working on. Merge from any other branch should be avoided

  • Once, a branch is merged through a PR, into the staging branch, it can not be used further either for new test script creation or maintaining existing scripts

  • Once you are creating PR, please share your excel sheet in the google folder(will be provided soon), naming convention will be discussed soon

  • To Merge conflict for excel sheets, please accept your excel sheet

  • Main is only for production run, once, all tests are working in staging branch, it will merge into main

1. Environment Setup

Mobile App testing

Java Appium TestNG

Prerequisites

To develop mobile application tests in Appium for Android/iOS using the TestNG framework you need to configure the environment locally as described in the instruction below.

  1. Install the latest version of Appium: Appium: Mobile App Automation Made Awesome.

  2. Install the latest version of Appium Inspector: Releases · appium/appium-inspector

  3. (recommended) Install one of the following Java IDEs:

Additional steps for Android:

  1. Install the latest version of Android Studio: Download Android Studio and SDK tools | Android Developers

  2. Connect the Android device to the computer or Configure the Android device emulator

  3. (recommended) Install Android SDK platform tools

Additional steps for iOS:

  1. Install the latest version of XCode: Xcode - Apple Developer

  2. Connect the iOS device to the computer or launch the iOS device simulator from XCode

  3. Configure WebDriverAgent to work with your simulator or a real device. Use this article as a reference: Appium vs iOS Simulator/iOS Real Device setup for successful Mobile Automation

Create a project

To create a test script compatible with Testlio platform you have to use our framework, containing all the supported libraries by our engine as well as some tools and helper functions which you’re welcome to use during your tests development process. You can find a JAR library with the Testlio framework in the parent directory.

Also, to help you with the process of creating a test automation project, we prepared a template project. Here are the steps, how to use this template project as a starting point:

  1. Pull template projects from GitHub: GitHub - Testlio/testlio-automation-templates

  2. Open the Selenium-Appium-Java-Framework project in your IDE

  3. Note identifiers of Android devices to be used for testing

  4. Update the path to the apk file in Enviroment.properties file.

  5. Launch Appium server

  6. Check that Testlio Login Test works fine in your environment by running the tests suite:

    1. For Android App:

      mvn clean test \
      -Dprovider=local \
      -Dexecution.type=mobile-native \
      -Dmobile.platform=android \
      -Dmobile.app.path=testlio-sample-mobile-app.apk \
      -Dmobile.device.name=<your device name> \
      -Dmobile.platform.version=<your device os version>

      Note: Don’t forget to put the proper platform version -Dmobile.platform.version and device name -Dmobile.device.name you noted before.

      1. If the above is not working, please add below go to Run --> edit configuration --> modify options -->Add VM Options. Add the below command:

        -ea -Dprovider=local -Dexecution.type=mobile-native -Dmobile.platform=android -Dmobile.device.name< your device model name >
        -Dmobile.platform.version==<your device os version>

    2. For iOS App:

      mvn clean test \
      -Dprovider=local \
      -Dexecution.type=mobile-native \
      -Dmobile.platform=ios \
      -Dmobile.app.path=testlio-sample-mobile-app.ipa \
      -Dmobile.device.udid=<your device identifier> \
      -Dmobile.device.name=<your device model name> \
      -Dmobile.platform.version=<your device os version>

      Note: Don’t forget to put the proper platform version -Dmobile.platform.version and device name -Dmobile.device.name you noted before and path to the apk file -Dmobile.app.path you have downloaded.

      1. If the above is not working, please add below go to Run --> edit configuration --> modify options -->Add VM Options. Add the below command:

        ea  -Dprovider=local \ 
        -Dexecution.type=mobile-native \
        -Dmobile.platform=ios \
        -Dmobile.device.udid=<your device identifier> \
        -Dmobile.device.name=<your device model name> \
        -Dmobile.platform.version=<your device os version>
        Apply and Run ‘Suite Executor’

Mobile Web testing

Java Appium TestNG

Prerequisites

To develop mobile application tests in Appium for Android/iOS using the TestNG framework you need to configure the environment locally as described in the instruction below.

  1. Install the latest version of Appium: Appium: Mobile App Automation Made Awesome.

  2. Install the latest version of Appium Inspector: Releases · appium/appium-inspector

  3. (recommended) Install one of the following Java IDEs:

  4. Set Android_Home environment variable

    • If you are using bash profile:

      • Open terminal and type

        nano ~/.bash_profile
      • Add the lines

        export ANDROID_HOME=/Users/<username>/Library/Android/sdk export PATH=$PATH:$ANDROID_HOME/platform-tools export PATH=$PATH:$ANDROID_HOME/tools export PATH=$PATH:$ANDROID_HOME/tools/bin export PATH=$PATH:$ANDROID_HOME/emulator
      • Reopen the terminal and check if it worked:

        source ~/.bash_profile 
        echo $ANDROID_HOME
    • If you are using .zshrc profile:

      • Go to Home folder from Go

      • Open .zshrc file and add the below lines

        export ANDROID_HOME=/Users/<username>/Library/Android/sdk
        export PATH=$PATH:$ANDROID_HOME/platform-tools
        export PATH=$PATH:$ANDROID_HOME/tools
        export PATH=$PATH:$ANDROID_HOME/tools/bin
      • Reopen the terminal and check if it worked:

        source ~/.zshrc
        echo $ANDROID_HOME

Additional steps for Android:

  1. Install the latest version of Android Studio: Download Android Studio and SDK tools | Android Developers

  2. Connect the Android device to the computer or Configure the Android device emulator

  3. (recommended) Install Android SDK platform tools

  4. If you are using android version 13, Add the below mentioned capabilities

    desiredCapabilities.setCapability("appPackage", "com.android.chrome");
    desiredCapabilities.setCapability("appActivity", "com.google.android.apps.chrome.Main")

Additional steps for iOS:

  1. Install the latest version of XCode: Xcode - Apple Developer

  2. Connect the iOS device to the computer or launch the iOS device simulator from XCode

  3. Configure WebDriverAgent to work with your simulator or a real device. Use this article as a reference: Appium vs iOS Simulator/iOS Real Device setup for successful Mobile Automation

Create a project

To create a test script compatible with Testlio platform you have to use our framework, containing all the supported libraries by our engine as well as some tools and helper functions which you’re welcome to use during your tests development process. You can find a JAR library with the Testlio framework in the parent directory.

Also, to help you with the process of creating a test automation project, we prepared a template project. Here are the steps, how to use this template project as a starting point:

  1. Pull template projects from GitHub: GitHub - Testlio/testlio-automation-templates

  2. Open the Selenium-Appium-Java-Framework project in your IDE

  3. Note identifiers of Android devices to be used for testing

  4. Update the path to the apk file in Enviroment.properties file.

  5. Launch Appium server

  6. Check that Testlio Login Test works fine in your environment by running the tests suite:

    1. For Android Web:

      mvn clean test \
      -Dprovider=local \
      -Dexecution.type=mobile-web \
      -Dmobile.platform=android \
      -Dmobile.device.name=<your device name> \
      -Dmobile.platform.version=<your device os version>

      1. If the above is not working, please add below go to Run --> edit configuration --> modify options -->Add VM Options. Add the below command:

        -ea -Dprovider=local -Dexecution.type=mobile-web -Dmobile.platform=android -Dmobile.device.name< your device model name >
        -Dmobile.platform.version==<your device os version>

      2. Download chrome driver executable file.

      3. Open terminal in IDE and run below command

         appium --chromedriver-executable <chromedriver path> 

      4. Apply and Run ‘Suite Executor’

        Note: Don’t forget to put the proper platform version -Dmobile.platform.version and device name -Dmobile.device.name you noted before

    2. For iOS Web:

      mvn clean test \
      -Dprovider=local \
      -Dexecution.type=mobile-web \
      -Dmobile.platform=ios \
      -Dmobile.device.udid=<your device identifier> \
      -Dmobile.device.name=<your device model name> \
      -Dmobile.platform.version=<your device os version>

      1. If the above is not working, please add below go to Run --> edit configuration --> modify options -->Add VM Options. Add the below command:

        ea jar  -Dprovider=local \ 
        -Dexecution.type=mobile-web \
        -Dmobile.platform=ios \
        -Dmobile.device.udid=<your device identifier> \
        -Dmobile.device.name=<your device model name> \
        -Dmobile.platform.version=<your device os version>
        Apply and Run ‘Suite Executor’

Note: Don’t forget to put the proper device identifier -Dmobile.device.udid, device name -Dmobile.device.name and device OS version -Dmobile.platform.version you noted before

Desktop Web testing

Java Selenium TestNG

Prerequisites

To develop desktop web application tests in Selenium using the TestNG framework you need to configure the environment locally as described in the instruction below.

  1. Install one or several of the supported by Testlio platform web browsers:

  2. (recommended) Install one of the following Java IDEs:

Create a project

To create a test script compatible with Testlio platform you have to use our framework, containing all the supported libraries by our engine as well as some tools and helper functions which you’re welcome to use during your tests development process. You can find a JAR library with the Testlio framework in the parent directory.

Also, to help you with the process of creating a test automation project, we prepared a template project. Here are the steps, how to use this template project as a starting point:

  1. Pull template projects from GitHub: GitHub - Testlio/testlio-automation-templates

  2. Open the Selenium-Appium-Java-Framework project in your IDE

  3. Check that Testlio Login Test works fine in your environment by running the tests suite:

    mvn clean test \
    -Dprovider=local \
    -Dexecution.type=web \
    -Dbrowser.name=<browser name value>

  4. If the above is not working, please add below go to Run --> edit configuration --> modify options -->Add VM Options. Add the below command:

    1. chrome
      firefox
      MicrosoftEdge
      "internet explorer"

2. Testlio Platform Execution

Mobile App testing

  • Navigate to Respective workspace in TestLIO Platform Ex: Demo

  • Click Launch new Platform.

  • Go to “Runs” Tab and Create New Run for Automated Test

  • Choose App “Mobile App”

  • Upload “.apk” or “.ipa” file.

  • Select Test Type.

  • Attach the Test Script Package.

  • Attach the custom spec with the below command should be updated with test config file,

For “ANDROID”

- java -javaagent:./dependency-jars/aspectjweaver-1.9.5.jar -jar -Dprovider=testlio -Dmobile.platform=android -Dexecution.type=mobile-native 
-Dtake.screenshots.on.annotated.steps=true -Dallure.results.directory=$DEVICEFARM_LOG_DIR/allure-results -Dappium.screenshots.dir=$DEVICEFARM_SCREENSHOT_PATH *-tests.jar -d $DEVICEFARM_LOG_DIR/test-output -verbose 10

       post_test:
commands:
- cp -n -R ./reports $DEVICEFARM_LOG_DIR
- cp -n -R ./allure-results $DEVICEFARM_LOG_DIR.

For “IOS”

- java -javaagent:./dependency-jars/aspectjweaver-1.9.5.jar -jar -Dprovider=testlio -Dmobile.platform=ios -Dexecution.type=mobile-native -Dtake.screenshots.on.annotated.steps=true -Dallure.results.directory=$DEVICEFARM_LOG_DIR/allure-results -Dappium.screenshots.dir=$DEVICEFARM_SCREENSHOT_PATH *-tests.jar -d $DEVICEFARM_LOG_DIR/test-output -verbose 10

       post_test:
commands:
- cp -n -R ./reports $DEVICEFARM_LOG_DIR
- cp -n -R ./allure-results $DEVICEFARM_LOG_DIR

  • Select Device and Create a new Run Now.

Desktop Web testing

  • Navigate to Respective workspace in TestLIO Platform Ex: Demo

  • Click Launch new Platform.

  • Go to “Runs” Tab and Create New Run for Automated Test.

  • Choose App “Desktop Web”

  • Select Test Type “Selenium Java TestNG”.

  • Attach the Test Script Package.

  • Select Device and Create a new Run Now.

3. Script Development

Mobile App testing

Java Appium TestNG

Add your first test case

Describe an application page

The concept of our testing environment is that we use the Page Object Model design pattern to navigate across the application. It means that every application view should be represented as a separate class, describing all the necessary UI elements on this particular view as attributes and all the possible actions as methods.

To find a proper mobile element selector, use Appium Inspector session

Here is an example of such a class in Java:

package testlio.lib.pages;

import org.openqa.selenium.By;

import testlio.SuiteExecutor.SuiteRunner;
import testlio.generic.lib.AndroidWrapper;
import testlio.util.lib.ConfiguratorSupport;

public class Demo_BasePage extends AndroidWrapper{

public static ConfiguratorSupport config = new ConfiguratorSupport("Environment.properties");
static String strurl = config.getProperty("url"), strRemoteMachineIP = config.getProperty("RemoteMachineIP"),
strRemoteMachinePort = config.getProperty("RemoteMachinePort");

public void DemoLaunch(String userName , String passWord) throws Throwable {

Thread.sleep(5000);
By DemoSignIn=setLocator(SuiteRunner.appObjectLocatorType.get("DemoSignIn"),SuiteRunner.appObjectLocatorValue.get("DemoSignIn"));
By DemoUserName=setLocator(SuiteRunner.appObjectLocatorType.get("DemoUserName"),SuiteRunner.appObjectLocatorValue.get("DemoUserName"));
System.out.println(DemoUserName);
By DemoPassword=setLocator(SuiteRunner.appObjectLocatorType.get("DemoPassword"),SuiteRunner.appObjectLocatorValue.get("DemoPassword"));
By DemoLoginButton=setLocator(SuiteRunner.appObjectLocatorType.get("DemoLoginButton"),SuiteRunner.appObjectLocatorValue.get("DemoLoginButton"));
By DemoPlaybutton=setLocator(SuiteRunner.appObjectLocatorType.get("DemoPlaybutton"),SuiteRunner.appObjectLocatorValue.get("DemoPlaybutton"));
By Shows=setLocator(SuiteRunner.appObjectLocatorType.get("Shows"),SuiteRunner.appObjectLocatorValue.get("Shows"));
By SearchIcon=setLocator(SuiteRunner.appObjectLocatorType.get("SearchIcon"),SuiteRunner.appObjectLocatorValue.get("SearchIcon"));
By SearchBox=setLocator(SuiteRunner.appObjectLocatorType.get("SearchBox"),SuiteRunner.appObjectLocatorValue.get("SearchBox"));
By HomeIcon=setLocator(SuiteRunner.appObjectLocatorType.get("HomeIcon"),SuiteRunner.appObjectLocatorValue.get("HomeIcon"));
Thread.sleep(50000);
try {
clickOnElement(DemoSignIn, "Sign In");
Thread.sleep(5000);
}catch (Exception e) {
e.printStackTrace();
}

//String Username = TCData.get("Username");
//System.out.println(Username);
Thread.sleep(5000);
//String Password = TCData.get("password");
//System.out.println(Password);
Thread.sleep(20000);
sendKeys(DemoUserName, userName, "Username");
Thread.sleep(5000);
sendKeys(DemoPassword, passWord, "Password");
Thread.sleep(5000);
clickOnElement(DemoLoginButton, "Login");
Thread.sleep(10000);
clickOnElement(DemoPlaybutton, "Play");
Thread.sleep(5000);
Thread.sleep(15000);
navigateBackToPreviousScreen();
Thread.sleep(5000);
clickOnElement(Shows, "Shows");
Thread.sleep(5000);

clickOnElement(SearchIcon, "Search");
//clickOnElement(SearchBox, "SearchBoxClick");
String searchBox = TCData.get("SearchBox");
Thread.sleep(5000);
sendKeyswithoutClear(SearchBox, searchBox, "SearchBox");
Thread.sleep(5000);
hideKeyboard();
Thread.sleep(15000);

clickOnElement(HomeIcon, "HomeIcon");

Thread.sleep(5000);

}



}

Implement a test case class

Using the actions you described on a page, now you’re able to combine them into a particular test case. To do so, first of all, you need to create a class representing a test case under the package src/test/java/com/testlio/tests

Here is an example of such a test case in Java

package testlio.app.lib.SampleTest;

import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

import io.qameta.allure.Description;
import testlio.lib.pages.Demo_BasePage;
import testlio.listeners.TestResultListener;

@Listeners({ TestResultListener.class })

public class DemoCase extends Demo_BasePage {

@Test
@Description("Demo application")
public void demoNation() throws Throwable {
String struser = TCData.get("Username");
String password = TCData.get("Password"); System.out.println(struser);
System.out.println(password);
DemoNationLaunch(struser,password);
}
}

How to find Android device identifiers

Real device

1. On the Android device go to Settings

2. Tap About phone

3. Take note of device name and Android version

Emulator

1. Launch Android Studio

2. Android Studio welcome screen

3. On the Welcome screen click More Actions > AVD Manager

More Actions button

AVD Manager item in the More Actions dropdown

1. On the Android Virtual Device Manager window find the device you’re going to use and note values from Name and Target columns as shown on the screenshot below:Device name and OS version

How to find iOS device identifiers

1. Launch XCode

2. From the system top panel navigate to Window > Devices and SimulatorDevices and Simulators menu item in XCode

3. Select device model for which you want to get identifiers for and note values as shown on the screenshot below:

How to note iOS device identifiers

For example, shown on the screenshot values will be:

  • Device OS version: 15.2

  • Device model name: iPhone 11

  • Device identifier: BCA0D9F8-4B11-4D2B-A164-674FC7102D19

How to launch the Appium Inspector session

1. Connect the device to the computer or launch the simulator. You can use one of the following tutorials:

  • How to configure Android device emulator

  • How to launch iOS device simulator from XCode

2. Launch Appium Server

3. Launch Appium Inspector

4. Click on Edit Raw JSON button

5. Paste one of the following capabilities to the text field and click the Save button:

For Android devices/emulators:

{  
"platformName": "Android",
"appium:platformVersion": "<your device os version>", "appium:deviceName": "<your device name>",
"appium:app": "<local path to apk>",
"appium:automationName": "UiAutomator2", "appium:autoGrantPermissions": true
}

Note: Don’t forget to put the proper platform version appium: platformVersion and device name appium:deviceName you noted before and path to the apk file appium:app you have downloaded

For iOS devices/simulators:

{      
"platformName": "iOS",
"appium:platformVersion": "<your device os version>", "appium:deviceName": "<your device name>",
"appium:udid": "<your device identifier>",
"appium:app": "<local path to ipa or app>",
"appium:xcodeOrgId": "<your apple developer account id>" "appium:xcodeSigningId": "iPhone Developer"
"appium:automationName": "XCUITest”,
}

Note 1: Don’t forget to put the proper platform version appium:platformVersion, device name appium:deviceName, and device identifier appium:udid you noted before, that the path to the ipa/app file appium:app you have downloaded an Apple Developer account id appium:xcodeOrgId

Note 2: Depending on how you configured WebDriverAgent and on the specific iOS version of your device/simulator, you may need also the following properties:

  • "appium:noReset": true together with "appium:fullReset": false - to not re-install the app on the device (way to avoid “trusting“ Apple developer in settings each time you launch tests)

  • "appium:useNewWDA": false - if you don’t want to reinstall the currently installed version of WebDriverAgent

  • "appium:usePrebuiltWDA": true - to use manually prebuilt using XCode version of WebDriverAgent

Click the Start Session button to launch the inspector session

How to configure TestNG execution

To let the TestNG framework know which test cases you want to execute, you need to mention them in the testng.xml configuration file, like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Default Suite">
<test name="Testlio Login Test">
<classes>
<class name="com.testlio.tests.TestlioLoginTest"/>
</classes>
</test>
</suite>

Good practices

  • Keep the name attribute of the <test> tag consistent with the class names listed under it

  • If you need to test several test cases in one run, don’t forget to properly group them in the configuration file. For example:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Default Suite">
<test name="Testlio Login Tests">
<classes>
<class name="com.testlio.tests.TestlioLoginTest"/>
</classes>
</test>
<test name="Testlio Platform Tests">
<classes> <classname="com.testlio.tests.TestlioCreateManualRunTest"/>
<class name="com.testlio.tests.TestlioAssignDevicesTest"/> <class name="com.testlio.tests.TestlioInviteTesterTest"/> </classes>
</test>
</suite>

How to package tests

In order to run tests on our farm, you need to package them in a compatible format. To do that, you may simply use the following command:

mvn clean install  -DskipTests

After the packaging process is finished, you will be able to find it in the target directory relative to the project root. The package file should have a name in format

{project artifact id}-package-{timestamp}.zip

Appium Node

Add your first test case

Describe an application page

The concept of our testing environment is that we use the Page Object Model design pattern to navigate across the application. It means that every application view should be represented as a separate class, describing all the necessary UI elements on this particular view as attributes and all the possible actions as methods.

To find a proper mobile element selector, use Appium Inspector session

Here is an example of such a class:

import {WebdriverIO} from "@wdio/types/build/Options";
import {ChainablePromiseElement} from "webdriverio";
import AlertPopup from "./alert-popup";
import MobileScreen from "../../lib/mobile-screen";

export default class TestlioLoginScreen extends MobileScreen {

public get title(): ChainablePromiseElement<Promise<WebdriverIO.Element>> {
return this.driver.$("//*[@resource-id='login-title']");
}

public get usernameInput(): ChainablePromiseElement<Promise<WebdriverIO.Element>> {
return this.driver.$("//android.widget.EditText[contains(@text, 'Username')]");
}

public get passwordInput(): ChainablePromiseElement<Promise<WebdriverIO.Element>> {
return this.driver.$("//android.widget.EditText[contains(@text, 'Password')]");
}

public get loginButton(): ChainablePromiseElement<Promise<WebdriverIO.Element>> {
return this.driver.$("//*[@resource-id='login-button']");
}

public isPageLoaded(): any {
return this.title.isDisplayed;
}

public async inputUsername(username: string) {
await this.usernameInput.setValue(username);
}

public async inputPassword(password: string) {
await this.passwordInput.setValue(password);
}

public async clickLogin(): Promise<AlertPopup> {
await this.loginButton.click();
return <AlertPopup> await this.waitForPageToLoad(new AlertPopup(this.driver));
}
}

Implement a test case class

Using the actions you described on a page, now you’re able to combine them into a particular test case. To do so, first of all, you need to create a module <test case name>.test.ts representing a test case under the project directory test/specs

Here is an example of such a test case:

import { describe, it, before, after } from 'mocha';
import { expect } from 'chai';
import TestlioStaticPageScreen from '../../src/pageobjects/testlio-static-page-screen';
import { config } from '../../config';
import TestlioLoginScreen from "../../src/pageobjects/testlio-login-screen";
import AlertPopup from "../../src/pageobjects/alert-popup";

const SUCCESSFUL_LOGIN_ALERT_TITLE = 'Successful login';

describe('When on login screen', () => {
let loginScreen: TestlioLoginScreen;

before(async() => {
const staticPageScreen: TestlioStaticPageScreen = new TestlioStaticPageScreen(driver);
loginScreen = <TestlioLoginScreen> await staticPageScreen.goToLoginScreenTab();
});

it('Checks that can login with correct credentials', async() => {
await loginScreen.inputUsername(config.username);
await loginScreen.inputPassword(config.password);
const alertPopup: AlertPopup = await loginScreen.clickLogin();
const alertTitle = await alertPopup.getTitleString();
expect(alertTitle).to.equal(SUCCESSFUL_LOGIN_ALERT_TITLE);
});
}).timeout(10000);

Mobile & Desktop Web testing

Java Appium TestNG

Add your first test case

Describe an application page

The concept of our testing environment is that we use the Page Object Model design pattern to navigate across the application. It means that every application view should be represented as a separate class, describing all the necessary UI elements on this particular view as attributes and all the possible actions as methods.

Here is an example of such a class in Java:

package com.testlio.pages;

import com.testlio.lib.pagefactory.WebPage;
import com.testlio.lib.utility.screenshot.annotations.MakeScreenshot;
import io.qameta.allure.Step;
import lombok.extern.slf4j.Slf4j;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

import java.util.function.Function;

import static com.testlio.lib.pagefactory.TestlioPageFactory.initWebElements;

/**
* This class represents Testlio platform login page
*/
@Slf4j
public class TestlioLoginWebPage extends WebPage {

// Attribute describing Email input
@FindBy(xpath = "//input[@placeholder='Email']")
private WebElement emailInput;

// Attribute describing Password input
@FindBy(xpath = "//input[@placeholder='Password']")
private WebElement passwordInput;

// Attribute describing Login button
@FindBy(xpath = "//button/span[contains(text(), 'Log in')]")
private WebElement loginButton;

public TestlioLoginWebPage(WebDriver driver) {
super(driver);
}

@Override
public Function<WebDriver, ?> isPageLoaded() {
// We consider that page is loaded as far as the login button displayed
return wait -> loginButton.isDisplayed();
}

/**
* Action allowing to input any specific text
* into the Email field on the login page
* @param email
*/
@Step("Input email") // Step annotation needed to mention this action in the test report
@MakeScreenshot // This annotation points that you want to make a screenshot on this step
public void inputEmail(String email) {
typeTextToElement(emailInput, email);
}

/**
* Action allowing to input any specific text
* into the Password field on the login page
* @param password
*/
@Step("Input password")
@MakeScreenshot
public void inputPassword(String password) {
typeTextToElement(passwordInput, password);
}

/**
* Action allowing to click on Login
* button on the login page
* @return TestlioHomeWebPage
*/
@Step("Click login button")
@MakeScreenshot
public TestlioHomeWebPage clickLogin() {
scrollToWebElement(loginButton);
clickOnElement(loginButton);
return initWebElements(getDriver(), TestlioHomeWebPage.class);
}
}

Good practices

  • Create all the classes representing web application page models in package: src/main/java/com/testlio/pages

  • All the page model classes should be inherited from the abstract class com.testlio.lib.pagefactory.WebPage

  • Page model classes should have a name ending with WebPage

Implement a test case class

Using the actions you described on a page, now you’re able to combine them into a particular test case. To do so, first of all, you need to create a class representing a test case under the package src/test/java/com/testlio/tests

Here is an example of such a test case in Java:

package com.testlio.tests;

import com.testlio.lib.BaseTest;
import com.testlio.pages.TestlioLoginWebPage;
import org.testng.annotations.Test;

import java.util.ResourceBundle;

import static com.testlio.constants.WebAppUrls.PLATFORM_TESTLIO_QA;

public class TestlioLoginTest extends BaseTest {

/**
* Test case, verifying, that Testlio user
* is able to login to the Testlio platform
*/
@Test
public void testlioLoginTest() {
// Read credentials from properties file
ResourceBundle credentials = ResourceBundle.getBundle("credentials");
String userEmail = credentials.getString("user.email");
String userPassword = credentials.getString("user.password");

// Navigate to the initial web application page - TestlioLoginWebPage
TestlioLoginWebPage testlioLoginWebPage =
(new TestlioLoginWebPage(webDriver))
.navigateTo(PLATFORM_TESTLIO_QA, TestlioLoginWebPage.class);

// Input user's email address
testlioLoginWebPage.inputEmail(userEmail);

// Input user's password
testlioLoginWebPage.inputPassword(userPassword);

// Click on login button
testlioLoginWebPage.clickLogin();
}

}

Good practices

  • Create all the classes representing test cases in the package: src/test/java/com/testlio/tests

  • All the test classes should be inherited from the abstract class com.testlio.lib.BaseTest

  • Test case classes should have a name ending with Test

Granular videos for Mobile & Web App testing types

The platform does not produce granular recording for each test case by default. In order to enable it, you have to configure it from the test script side. Here are some examples how it might be accomplished:

  • Java (TestNG)

import org.testng.ITestContext;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.AfterTest;

import io.appium.java_client.android.AndroidStartScreenRecordingOptions;
import io.appium.java_client.ios.IOSStartScreenRecordingOptions;
import io.appium.java_client.screenrecording.CanRecordScreen;

import io.qameta.allure.Allure;

import org.apache.commons.lang3.StringUtils;

import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.util.Base64;

public class ExampleTest {
@BeforeTest
public void beforeTest() {
if (/** Is Android execution? **/) {
driver.startRecordingScreen(new AndroidStartScreenRecordingOptions());
} else if (/** Is IOS execution? **/) {
driver.startRecordingScreen(new IOSStartScreenRecordingOptions());
}
}

@AfterTest
public void afterTest(final ITestContext testContext) {
String video = driver.stopRecordingScreen();
if(StringUtils.isNotEmpty(video)) {
byte[] decode = Base64.getDecoder().decode(video);
InputStream is = new ByteArrayInputStream(decode);
Allure.addAttachment(testContext.getName(), "video/mp4", is, "mp4");
}
}
}


3. Script Validation Local

Mobile App testing

Java Appium TestNG

Before running the tests, you need to edit TestNG config in order to point out which tests you want to run.

After the testng.xml config file is properly configured you can run the testing process using the command:

Android:

mvn clean test \
-Dprovider=local \
-Dexecution.type=mobile-native \
-Dmobile.platform=android \
-Dmobile.device.name=<your device name> \
-Dmobile.platform.version=<your device os version> \
-Dmobile.app.path=<local path to apk>


Check here how to find your Android device name and OS version

iOS

mvn clean test \
-Dprovider=local \
-Dexecution.type=mobile-web \
-Dmobile.platform=ios \
-Dmobile.device.udid=<your device identifier> \
-Dmobile.device.name=<your device model name> \
-Dmobile.platform.version=<your device os version>

Check here how to find your iOS device identifier, model name, and OS version

Appium Node

  1. Put the local desired capabilities to the wdio.local.conf.ts:

    capabilities: [
    {
    platformName: '<put your value here>', // Android or iOS
    platformVersion: '<put your value here>', // Device OS version
    app: '<put your value here>', // Absolute local path to the app under test
    deviceName: '<put your value here>', //
    automationName: '<put your value here>', // UiAutomator2, XCUITest or other
    autoGrantPermissions: true // For Android only
    }
    ]
  2. Check that Testlio Login Test works fine in your environment by running the tests suite:

    npm run test-local

Mobile Web testing

ava Appium TestNG

Before running the tests, you need to edit TestNG config in order to point out which tests you want to run. After the testng.xml config file is properly configured you can run the testing process using the command:

Android

mvn clean test \
-Dprovider=local \
-Dexecution.type=mobile-web \
-Dmobile.platform=android \
-Dmobile.device.name=<your device name> \
-Dmobile.platform.version=<your device os version>

Check here how to find your Android device name and OS version

iOS

mvn clean test \
-Dprovider=local \
-Dexecution.type=mobile-web \
-Dmobile.platform=ios \
-Dmobile.device.udid=<your device identifier> \
-Dmobile.device.name=<your device model name> \
-Dmobile.platform.version=<your device os version>

Check here how to find your iOS device identifier, model name, and OS version

Desktop Web testing

Java Selenium TestNG

Before running the tests, you need to edit TestNG config in order to point out which tests you want to run. After the testng.xml config file is properly configured you can run the testing process locally using the command:

mvn clean test \
-Dprovider=local \
-Dexecution.type=web \
-Dbrowser.name=<browser name value>

Possible -Dbrowser.name values:

  • chrome

  • firefox

  • MicrosoftEdge

  • "internet explorer"

4. Sandbox Package run

  1. You can use our sandbox to validate the scripts, by creating a run and waiting for the results. Depending on the results you can see if the script needs any fixes or if it's ready

  2. Without sandbox use ( third party integration) you can submit your packaged script to our QE Manager

  3. If no error is verified in the prior step, please send the test package to the coordinator

  4. If you need assistance you can contact the coordinator anytime by opening a ticket

Please check this next document for more details on how to create automated test runs using Testlio Sandbox.

5. Script Validation (by Testlio team)

Please check this next document “how to understand results” in order to get a clear idea if your script needs any further development or is good enough to send to the coordinator.

6. Script Acceptance

Script quality confirmation!

7. Related instructions

How to install Java Development Kit (JDK) version 1.8

Download and install the distribution

Select the distribution corresponding to your OS, download and install it:

Pay attention, that to download a Java distribution you will be required to login/signup to the Oracle account

Check JAVA_HOME environment variable

Make sure, that the JAVA_HOME environment variable pointing to the JDK 1.8 installation path

Windows

  1. Locate your Java installation directory. If you didn't change the path during installation, it'll be something like C:\Program Files\Java\jdk1.8.0_311 or you can also type where java at the command prompt.

  2. Do one of the following:
    Windows 7 – Right-click My Computer and select Properties > Advanced
    Windows 8 – Go to Control Panel > System > Advanced System Settings
    Windows 10 – Search for Environment Variables then select Edit the system environment variables

  3. Click the Environment Variables button.

  4. Under System Variables, try to find the JAVA_HOME variable and ensure, that it points to the proper JDK path

  5. If JAVA_HOME is not on the list, then click New under System Variables and

    • In the Variable Name field, enter JAVA_HOME

    • In the Variable Value field, enter your JDK installation path

MacOS

  1. To check the JAVA_HOME variable execute the following command in Terminal:

    echo $JAVA_HOME

  2. If the output is not empty, it means that JAVA_HOME is set. But you should make sure that it points to the JDK version 1.8. Compare it with the output from the execution following command in Terminal:

    /usr/libexec/java_home -v 1.8

  3. Otherwise, if the output is empty or it differs from the actual JDK path, you should add the following line to the ~/.bash_profile file:

    export JAVA_HOME=$(/usr/libexec/java_home)

Linux

  1. To check the JAVA_HOME variable execute the following command in Terminal:

    echo $JAVA_HOME
  2. If the output is not empty, it means that JAVA_HOME is set. But make sure that it points to the JDK version 1.8.

  3. Otherwise, if the output is empty or it differs from the actual JDK path, you should add the following line to the ~/.bashrc file:

    export JAVA_HOME=<path to JDK 1.8>

How to install Maven

Windows

  1. Download the latest version of Apache Maven binary zip file (for example, apache-maven-3.8.4-bin.zip): Maven – Download Apache Maven

  2. Unzip downloaded archive to any directory on your computer

  3. Add a MAVEN_HOME environment variable, and point it to the Maven folder

  4. Add %MAVEN_HOME%\bin to PATH

MacOS

Use the following Homebrew command to install Maven:

brew install maven

Linux

Use the following command to install Maven:

sudo apt-get install maven

How to install Node & NPM

Windows

  1. Download the respective distribution for your version of Windows: Download | Node.js

  2. Install the downloaded distribution

  3. Open Command Line

  4. Execute the following command to ensure that Node has been installed, it should print the installed Node version:

    node -v

  5. Execute the following command to ensure that Node has been installed, it should print the installed NPM version:

    npm -v

MacOS

  1. Open Terminal

  2. Use the following Homebrew command to install Node & NPM:

    brew install node

  3. Execute the following command to ensure that Node has been installed, it should print the installed Node version:

    node -v

  4. Execute the following command to ensure that Node has been installed, it should print the installed NPM version:

    npm -v

Linux

  1. Open Terminal

  2. Use the following command to install Node & NPM:

    sudo apt-get install node
  3. Execute the following command to ensure that Node has been installed, it should print the installed Node version:

    node -v
  4. Execute the following command to ensure that Node has been installed, it should print the installed NPM version:

    npm -v

How to install Android SDK platform tools using Android Studio

  1. Launch Android Studio

  2. On the Welcome screen click More Actions -> SDK Manager

  3. In the Preferences for New Projects dialog box, navigate to the SDK Tools tab and select set a checkbox on the latest version of Android SDK Platform-Tools (at least)

    Note: It’s recommended to note the location of Android SDK on your computer

  4. Click Apply. Android Studio starts installing the selected packages and tools on your computer. Once installed, the Status of the installed packages and tools changes from Not Installed to Installed.

  5. Click OK.

How to configure Android device emulator

  1. Launch Android Studio

  2. On the Welcome screen click More Actions -> AVD Manager

    More Actions button

    AVD Manager item in the More Actions dropdown

  3. On the Android Virtual Device Manager window click Create Virtual Device

    Create Virtual Device button on the Android Virtual Device Manager window

  4. On the Virtual Device Configuration window choose an Android device model you want to emulate and then click Next

    Virtual Device Configuration - Select Hardware

  5. On the following step select an Android system image you prefer to be installed on the device and click Next.

    Virtual Device Configuration - System Image

    Note: If there is a “Download” link near a release name, it means that this release has not been downloaded yet on your PC, and to use it, you need to download it from the internet by just clicking this link.

  6. On the latest step, we would recommend specifying a custom AVD Name to be able easily to identify the device when using it for testing. Try to avoid using spaces in the name by replacing it with dashes or underscores.

    Virtual Device Configuration - Android Virtual Device (AVD)

    Suggested naming format: {device model}-{platform version}-{api level}

    Example: pixel5-11-30

  7. Before you proceed, note the device name and OS version. You will need these values when you will be launching the tests on this device.

    Device name and OS version

  8. Click Finish when you’re done with a device configuration. The created device will immediately appear in your Android Virtual Device Manager devices list.

    List of created Virtual Devices

  9. Launch the created AVD emulator by clicking on the play button near it

    Launch Virtual Device button

  10. After a while, your device will be configured and ready for testing

    Launched Virtual Device

How to launch iOS device simulator from XCode

  1. Launch XCode

  2. From the system top panel navigate to XCode > Open Developer Tool > Simulator

    Simulator item in the XCode top panel menu

  3. Some simulated iOS device should be launching immediately. If not, or you want to launch a simulated device of another model, then from the system top panel navigate to File > Open Simulator > iOS > <device model>

    Simulator options

  4. After the device is loaded, it will be ready for testing

    Launched iPhone simulator

How to find Android device identifiers

Real device

  1. On the Android device go to Settings

  2. Tap About phone

  3. Take note of device name and Android version

Emulator

  1. Launch Android Studio

    Android Studio welcome screen

  2. On the Welcome screen click More Actions > AVD Manager

More Actions button

AVD Manager item in the More Actions dropdown

  1. On the Android Virtual Device Manager window find the device you’re going to use and note values from Name and Target columns as shown on the screenshot below:

Device name and OS version

How to find iOS device identifiers

  1. Launch XCode

  2. From the system top panel navigate to Window > Devices and Simulators

    Devices and Simulators menu item in XCode

  3. Select device model for which you want to get identifiers for and note values as shown on the screenshot below:

    How to note iOS device identifiers?

    For example, shown on the screenshot values will be:

    • Device OS version: 15.2

    • Device model name: iPhone 11

    • Device identifier: BCA0D9F8-4B11-4D2B-A164-674FC7102D19

How to launch the Appium Inspector session

  1. Connect the device to the computer or launch the simulator. You can use one of the following tutorials:

    • How to configure Android device emulator

    • How to launch iOS device simulator from XCode

  2. Launch Appium Server

  3. Launch Appium Inspector

  4. Click on Edit Raw JSON button

  5. Paste one of the following capabilities to the text field and click the Save button:

For Android devices/emulators:

{
"platformName": "Android",
"appium:platformVersion": "<your device os version>",
"appium:deviceName": "<your device name>",
"appium:app": "<local path to apk>",
"appium:automationName": "UiAutomator2",
"appium:autoGrantPermissions": true
}

Note: Don’t forget to put the proper platform version appium:platformVersion and device name appium:deviceName you noted before and path to the apk file appium:app you have downloaded

For iOS devices/simulators:

{     
"platformName": "iOS",
"appium:platformVersion": "<your device os version>",
"appium:deviceName": "<your device name>",
"appium:udid": "<your device identifier>",
"appium:app": "<local path to ipa or app>",
"appium:xcodeOrgId": "<your apple developer account id>"
"appium:xcodeSigningId": "iPhone Developer"
"appium:automationName": "XCUITest”,
}

Note 1: Don’t forget to put the proper platform version appium:platformVersion, device name appium:deviceName and device identifier appium:udid you noted before, the path to the ipa/app file appium:app you have downloaded an Apple Developer account id appium:xcodeOrgId

Note 2: Depending on how you configured WebDriverAgent and on the specific iOS version of your device/simulator, you may need also the following properties:

  • "appium:noReset": true together with "appium:fullReset": false - to not re-install the app on the device (way to avoid “trusting“ Apple developer in settings each time you launch tests)

  • "appium:useNewWDA": false - if you don’t want to reinstall the currently installed version of WebDriverAgent

  • "appium:usePrebuiltWDA": true - to use manually prebuilt using XCode version of WebDriverAgent

6. Click the Start Session button to launch the inspector session

How to configure TestNG execution

To let the TestNG framework know which test cases you want to execute, you need to mention them in the testng.xml configuration file, like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Default Suite">
<test name="Testlio Login Test">
<classes>
<class name="com.testlio.tests.TestlioLoginTest"/>
</classes>
</test>
</suite>

Good practices

  • Keep the name attribute of the <test> tag consistent with the class names listed under it

  • If you need to test several test cases in one run, don’t forget to properly group them in the configuration file. For example:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Default Suite">
<test name="Testlio Login Tests">
<classes>
<class name="com.testlio.tests.TestlioLoginTest"/>
</classes>
</test>
<test name="Testlio Platform Tests">
<classes>
<class name="com.testlio.tests.TestlioCreateManualRunTest"/>
<class name="com.testlio.tests.TestlioAssignDevicesTest"/>
<class name="com.testlio.tests.TestlioInviteTesterTest"/>
</classes>
</test>
</suite>

How to package tests

In order to run tests on our farm, you need to package them in a compatible format. To do that, you may simply use one the following command, depending on the environment:

Java

mvn package -DskipTests

Node

npm run package

After the packaging process is finished, you will be able to find it in the target directory relative to the project root. The package file should have a name in format {project artifact id}-package-{timestamp}.zip

Did this answer your question?