T-Plan Home
T-Plan Robot Enterprise 4.2.1 Doc Collection
17/11/16

Automating Android Over The ADB

Last update: 13 June 2016

Contents:
1. Introduction
2. Setting Up
3. Troubleshooting
4. Automation
   4.1 Connecting To The Device
   4.2 Dealing With Screen Rotation
   4.3 Touch Screen Automation
   4.4 Keyboard Automation


1. Introduction

T-Plan Robot Enterprise version 3.1 and higher supports automation of Android devices over the ADB (Android Debug Bridge). ADB is a tool delivered as part of the Android SDK which provides connectivity to Android devices attached to the USB.

Version 4.2 delivers support of screen mirroring on MS Windows which provides a better option for automation of applications requiring fast screen response such as games etc.

Compared to automation over VNC (namely VMLite VNC and DroidVNC):

Factor
Automation  over ADB w/ Screen Mirror
Automation over ADB only
Automation over VNC
Rooting
+ Not required
- DroidVNC: The device must be rooted
+ VMLite VNC: The device doesn't have to be rooted
Network
+ Not required (network is optional)
+ No network required (only USB)
- DroidVNC: Network connection required (Wi-Fi)
+ VMLite VNC: Network or USB connection required
SW Requirements
- Screen mirroring application required on the device
+ No SW installation on the device required
- VNC server required on the device
Keyboard Support
+ Good support of the Android native keys - Support of the Android native keys is subject to
  the server vendor
Screen Performance
+ Good screen refresh rate
- Slower screen refresh rate for large screen devices
  due to lack of image compression in the Android SDK
+ Fair screen refresh rate for large screen devices
  (depends
on the preferred image encoding)
Gaming Support
+ Supported
- Limited support (no OpenGL content)
Screen Rotation
+ Both automatic and manual screen rotation supported
- Manual screen rotation only supported
Pinch & Zoom
+ Supported  (Robot 3.5+/Android 4.3+) - Not supported

The nature of the ADB connection makes this approach suitable for non-intrusive automated testing of Android applications across multiple device types.


2. Setting Up

Though the Android SDK is supported on Windows, Linux and Mac OS X, it is recommended to use a MS Windows host for the automation. Automation on Linux (namely Ubuntu Linux which we used for testing) requires to run Robot from the root account and connection to the device is unstable, especially under a high load during test script execution. Though Robot can in most cases repair the connection automatically it leads to delays and lower reliability.

Set up instructions:
  1. Download and install the Android SDK revision 18 or higher on your computer. If you have and older version of the SDK upgrade it to the latest one using the <SDK_path>/tools/android tool.
  2. Enable the "USB Debugging" option on your Android device:
  1. Connect the Android device through the USB cable to the computer. It is not recommended to connect the device after the connection is initiated from Robot because it leads to intermittent connection crashes.
  2. MS Windows only: Add the folder containing adb.exe to the user or system path:
    1. Open Windows Explorer (Windows+E)
    2. Right click Computer and select Properties
    3. Select "Advanced System Settings" and click the "Environment Variables" button 
    4. Select the PATH user variable, click Edit and insert the path to the folder containing adb.exe to the beginning of the list. To separate the path from the original value use semicolon, ';'. The new value should look like:
       
      <SDK_path>\platform-tools;<originalPaths>

    5. Save with OK.
  3. Start T-Plan Robot Enterprise. If your system is a Linux/Unix one you will have to start it from the root account (sudo may not be sufficient).
  4. In the Login Window, set the "Connection Type" to "Android over ADB" and provide the path to the Android SDK. This path will be saved to the user preferences and it will get populated automatically onwards.
  5. If you have two or more Android devices connected to the computer, set the "Device Serial No." field to the serial number of your device. You may find it out through clicking of the "..." button or by executing "adb devices" from the platform-tools/ folder of your Android SDK. Alternatively leave the field on "default" to connect to the first detected device.
  6. T-Plan Robot Enterprise version 4.2 or higher: Select the "Screen transfer method":
  7. Hit Connect to connect to the device.

Screen Stream Mirror

Mirroring is supported for Robot v4.2 or higher running on MS Windows Vista or newer. It requires to install the Screen Stream Mirroring application by Mobzapp on the Android device. The screen data may be transferred over the network or the USB cable alone. Steps to enable screen mirroring:
  1. Install the Screen Stream Mirroring application on the Android device. The trial version is OK for evaluation purposes (it displays ads and shuts down after a few minutes).
  2. Start the app to verify it works. If you are connected to a network it must display an address like "rtsp://<IP_address>:5000/screen".
  3. No other steps are required if your device is connected to a network visible to Robot and you prefer the screen data transfer over TCP/IP. There's no need to start the app manually prior to connecting.
  4. Should you want to avoid the network and set up the USB screen data transfer:
    1. Start the app and tap the top left button to open the app menu. Choose Preferences.
    2. Scroll down to the bottom and set on "Advanced settings".
    3. Set on the "Stream through USB" option and tap the top left button to save the changes. The main screen must display the address of "rtsp://localhost:5000/screen".
Mirroring over USB instructions

3. Troubleshooting

To resolve issues with the Android connection take advantage of settings available under the "Android over ADB" panel of the Edit->Preferences window: If Robot fails to connect to the target device for no obvious reason and the log or the Login Window provides no reasonable explanation:
Limitations of the keyboard and touch screen support are listed in the Automation chapter. Should you experience any outstanding issues please contact the T-Plan support and attach the log file.


4. Automation


4.1 Connecting To The Device

To connect to an Android device from a test script use the Connect command (TPR scripts) or the connect() method (Java test scripts). The argument URL must be in form of either "adb://default" or "adb://<device_serial_No>". The Android SDK path must be configured prior to running the script either by providing the path in the Login Window and connecting to the device once or by setting it in the "Android over ADB" panel of the Preferences window.

For example, the following command/method call will connect you to the first detected device using the ADB screen transfer:

TPR test scripts:     Connect adb://default
Java test scripts:    connect("adb://default");

To connect to a device with the serial number of MB104PY10519:

TPR test scripts:     Connect adb://MB104PY10519
Java test scripts:    connect("adb://MB104PY10519");

To set on the screen mirror data transfer use:
TPR test scripts:     Connect adb://MB104PY10519 screen=mirror
Java test scripts:    connect("adb://MB104PY10519", "screen", "mirror");
Another alternative is to encode the parameters to the connect URL query. This form is also suitable for the -c/--connect CLI connection. You may copy the URL of any of your recent connections from the Tools->CLI Wizard window. Another alternative is to give your connection a name in the Desktop->Connection Manager window and then use it instead of the URL.
TPR test scripts:     Connect adb://MB104PY10519?screen=mirror
Java test scripts:    connect("adb://MB104PY10519?screen=mirror");

4.2 Dealing With Screen Rotation

The ADB connection supports two screen rotation modes. To change the mode go to the "Android over ADB" panel of the Preferences window.

1. Auto screen rotation (default)

The auto mode will read the screen orientation from the device on every screen refresh and rotate it accordingly. As this has a significant impact on the screen performance it is recommended to switch to the manual mode or to design the test suite against a constant screen orientation where high performance is required.

2. Manual screen rotation

In this mode the screen must be rotated in the Robot view manually (or from the script) through pressing of the F4 key. This behavior is compatible with the Droid VNC server.

To ensure that the touch screen events are properly placed and the image comparison works as expected always keep the view in Robot consistent with the real device state:

Correct view
             
Incorrect view
Correct view

Incorrect view
Search for components using the image comparison
will not work and the touch screen events
will be misplaced.

4.3 Touch Screen Automation

The touch screen actions are mapped as follows:

Device Action
Computer Action
Test Script Action
Tap (touch)
Left mouse click
Call the Mouse click command (TPR scripts) or one of the mouseClick() methods (Java scripts).
Long tap (touch)
Long left mouse click
Call a sequence of Mouse press, Wait and Mouse release commands or the corresponding Java methods.
Swipe (drag) Mouse drag (left button down)
Call the Mouse drag command (TPR scripts) or one of the mouseDrag() methods (Java scripts).
Pinch (zooming with 2 fingers)
Pinch/zoom action
Call the Mouse pinch/zoom command (TPR scripts) or one of the mousePinch() or mouseZoom()
methods (Java scripts). Supported since Robot 3.5. Requires device running Android OS 4.3 or higher.
Back
Right mouse click
Call the Mouse click btn=right command (TPR scripts) or one of the mouseRightClick() methods (Java scripts).
Alternatively use "Press Back" or press("Back").

Robot v3.5.2 and newer supports an option to perform mouse clicks over the UIAutomator interface. This usually resolves occasional touch screen failures for certain app controls. It requires Android 4.1.2 or newer.

4.4 Keyboard Automation

To make the script type a text call the Type or Typeline commands or the type()/typeLine() Java methods. To press a keyboard key call the Press command or the press() method. The keyboard events are supported as follows:
Android Device Key
Computer/Script Key (Action)
Description
BACK
Escape, Right mouse click
Back key.
CALL
PageDown
Call key.
CAMERA
F6
Camera key. Used to launch a camera application or take pictures. 
CLEAR
F7
Clear key.
DEL
Backspace
Backspace key. Deletes characters before the insertion point.
DPAD_DOWN
Down (Arrow Down), Mouse wheel down
Directional Pad Down key. May also be synthesized from trackball motions.
DPAD_LEFT
Left (Arrow Left)
Directional Pad Left key. May also be synthesized from trackball motions.
DPAD_RIGHT
Right (Arrow Right)
Directional Pad Right key. May also be synthesized from trackball motions.
DPAD_UP Up (Arrow Up), Mouse wheel up
Directional Pad Up key. May also be synthesized from trackball motions.
ENDCALL
End
End Call key.
FOCUS F5
Camera Focus key. Used to focus the camera. 
HOME
Home
Home key. This key is handled by the framework and is never delivered to applications.
MENU
PageUp
Menu key.
NOTIFICATION
F9
Notification key. Used to display the list of notifications.
POWER
F10
Power key.
SEARCH
F3
Search key.
SOFT_LEFT
F1
Soft Left key. Usually situated below the display on phones and used as a
multi-function feature key for selecting a software defined function shown
on the bottom left of the display.
SOFT_RIGHT
F2
Soft Right key. Usually situated below the display on phones and used as a
multi-function feature key for selecting a software defined function shown
on the bottom right of the display.
SYM
F8
Symbol modifier key. Used to enter alternate symbols.
On most devices it pops up the "Select input method" window.
VOLUME_DOWN F11
Volume Down key. Adjusts the speaker volume down.
VOLUME_UP F12
Volume Up key. Adjusts the speaker volume up.