Basic Automation¶

This chapter describes the basic automation features provided by FIRERPA. More advanced features will be described in subsequent chapters. This chapter introduces some commonly used basic feature interfaces.

Get Device Information¶

Get current device information, such as device name, screen size, screen orientation, and current application.

d.device_info()

>>> d.device_info()
productName: "bumblebee"
sdkInt: 34
displayHeight: 2400
displaySizeDpX: 411
displaySizeDpY: 914
displayWidth: 1080
screenOn: true
naturalOrientation: true
currentPackageName: "com.android.launcher3"

>>> result = d.device_info()
>>> print (result.displayWidth)
1080

Get Service Information¶

Get information such as the current service version, device unique ID, and ABI.

d.server_info()

>>> d.server_info()
uniqueId: "673abbe0-ff7b-9d82-1792-8876cb72cf56"
version: "9.25"
architecture: "arm64-v8a"
uptime: 293
secure: false

>>> result = d.server_info()
>>> print (result.secure)
False

Turn Off the Screen¶

The following interface turns off the current phone screen, equivalent to pressing the power button once when the screen is on.

d.sleep()

Turn On the Screen¶

The following interface turns on the current phone screen, equivalent to pressing the power button once when the screen is off.

d.wake_up()

Check if Screen Is On¶

You can use the following interface to check whether the current device screen is on, so you can determine whether it is ready for interaction.

d.is_screen_on()

Check if Screen Is Locked¶

You can use the following interface to check whether the current device screen is locked, so you can determine whether it is ready for interaction.

d.is_screen_locked()

Show Toast¶

You can use the following interface to display a Toast message Hello from Lamda! on the phone screen.

d.show_toast("Hello from Lamda!")

Read Clipboard¶

The following interface reads the current clipboard content on the phone. Android 10+ is not supported yet.

d.get_clipboard()

Write Clipboard¶

The following interface writes content to the current device clipboard.

d.set_clipboard("Clipboard content")

Physical Keys¶

You can use the following method to simulate key presses, supporting more than ten keys such as KEY_BACK, KEY_DOWN, and KEY_HOME.

d.press_key(Keys.KEY_BACK)
d.press_key(Keys.KEY_CAMERA)
d.press_key(Keys.KEY_CENTER)
d.press_key(Keys.KEY_DELETE)
d.press_key(Keys.KEY_DOWN)
d.press_key(Keys.KEY_ENTER)
d.press_key(Keys.KEY_HOME)
d.press_key(Keys.KEY_LEFT)
d.press_key(Keys.KEY_MENU)
d.press_key(Keys.KEY_POWER)
d.press_key(Keys.KEY_RECENT)
d.press_key(Keys.KEY_RIGHT)
d.press_key(Keys.KEY_SEARCH)
d.press_key(Keys.KEY_UP)
d.press_key(Keys.KEY_VOLUME_DOWN)
d.press_key(Keys.KEY_VOLUME_MUTE)
d.press_key(Keys.KEY_VOLUME_UP)

Additionally, to support even more keys, you can use this method to simulate them. You can find all supported key names in the official Android documentation: https://developer.android.com/reference/android/view/KeyEvent.

d.press_keycode(KeyCodes.KEYCODE_CALL)

Screenshot¶

We provide a screenshot method so you can record workflows or perform image recognition operations. The following example shows how to capture an image with quality 60 and save it as screenshot.png in the current directory.

d.screenshot(60).save("screenshot.png")

>>> result = d.screenshot(60)
>>> result
<lamda.types.BytesIO object at 0x7a28361d60>
>>> result.save("screenshot.png")
52917
>>> result.getvalue()
b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01....'

Currently, we also support region and element screenshots. Element screenshots are not covered in this chapter; here we introduce region screenshots. You need to understand the concept and definition of regions: in our API, a region is defined by Bound, which contains four parameters: top, left, right, and bottom. The Android screen differs from a conventional coordinate system; the top-left corner of the current device screen is the origin, with coordinates x=0, y=0. Assuming the current screen is portrait with a resolution of 1080×1920, if you want to capture a full-screen region screenshot, you can do the following:

d.screenshot(60, bound=Bound(top=0, left=0, right=1080, bottom=1920)).save("screenshot.png")

This will produce a full-screen screenshot, although it is unnecessary because full-screen is the default. If we want to capture a 200×200 image at the top-left corner, how do we do it? It is also very simple; you only need to do the following:

d.screenshot(60, bound=Bound(top=0, left=0, right=200, bottom=200)).save("screenshot.png")

Here we need to explain the meaning of these parameters: top represents the number of pixels offset downward from the Y-axis, bottom represents the number of pixels offset downward from the Y-axis, and left and right represent the number of pixels offset to the right from the X-axis. In all cases, top is always less than bottom, and left is always less than right. Next, let's demonstrate another example: to capture a 200×200 image at the bottom-right corner of the screen, you only need to call the API like this:

d.screenshot(60, bound=Bound(top=1920-200, bottom=1920, left=1080-200, right=1080)).save("screenshot.png")

Click a Point on the Screen¶

You can use the following interface to simulate a screen click, clicking the point at coordinates (100, 100) on the screen.

d.click(Point(x=100, y=100))

Long-Press a Point on the Screen¶

You can use the following interface to simulate a long press on the screen, pressing the point at coordinates (100, 100) and releasing after three seconds.

d.long_click(Point(x=100, y=100), timeout=1000*3) # 3s, 0 means the default duration

Drag Point A to Point B¶

The following interface can be used to drag an object at point A to point B, for example dragging an icon into a folder.

A = Point(x=100, y=100)
B = Point(x=500, y=500)

d.drag(A, B)

Swipe from Point A to Point B¶

The following interface can be used to swipe from screen coordinates (100, 100) to (500, 500).

A = Point(x=100, y=100)
B = Point(x=500, y=500)

d.swipe(A, B)

More Complex Multi-Point Swipe¶

The following interface can simulate multi-point swipes. For example, the following call will swipe from (100, 100) to (500, 500) and back to (200, 200). The example below only swipes through three points, but you can provide more points and even implement pattern unlock.

p1 = Point(x=100, y=100)
p2 = Point(x=500, y=500)
p3 = Point(x=200, y=200)

d.swipe_points(p1, p2, p3)

Open Quick Settings¶

The following call opens the quick settings panel, but only expands it halfway (half-open state).

d.open_quick_settings()

Open Notification Shade¶

The following call opens the notification shade on the screen.

d.open_notification()

Get Page Layout¶

The following call obtains the XML layout of the current page. You can also parse it yourself to support XPath automation.

d.dump_window_hierarchy()

>>> result = d.dump_window_hierarchy()
>>> result
<lamda.types.BytesIO object at 0x7add660680>
>>> result.getvalue()
b'<?xml version=\'1.0\' encoding=\'UTF-8\' standalone=\'yes\' ?>\r\n<hierarchy rotation="0">\r\n  <node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.android.systemui" content-desc="" checkable="false" checked="false" clickable="false"...

Wait for Window Idle¶

You can execute the following call to wait for the current window to become idle, similar to Selenium's implicitly_wait. The parameter is in milliseconds; 5*1000 means wait 5000 milliseconds (5 seconds).

d.wait_for_idle(5*1000)

Get the Last Toast¶

You can execute the following call to get the most recent Toast message in the system. However, note that only Toast messages sent through the system API are supported; Toast-like UI messages implemented by third parties cannot be retrieved.

d.get_last_toast()

>>> result = d.get_last_toast()
>>> print (result)
timestamp: 1700000000000
package: "com.android.settings"
message: "\346\202\250\345\267\262\345\244\204\344\272\216\345\274\200\345\217\221\350\200\205\346\250\241\345\274\217\357\274\214\346\227\240\351\234\200\350\277\233\350\241\214\346\255\244\346\223\215\344\275\234\343\200\202"
>>> print (result.message)
You are already in developer mode, no need to perform this action.