Basic Automation¶

This chapter describes the basic automation features provided by FIRERPA. More advanced features will be introduced in later chapters. Here we present some commonly used basic function interfaces.

Get Device Information¶

Retrieve current device information, such as device name, screen size, screen orientation, and the currently running application.

d.device_info()

>>> d.device_info()
productName: "bumblebee"
sdkInt: 34
displayHeight: 2400
displaySizeDpX: 411
displaySizeDpY: 914
displayWidth: 1080
screenOn: true
naturalOrientation: true
currentPackageName: "com.android.launcher3"

>>> result = d.device_info()
>>> print (result.displayWidth)
1080

Get Service Information¶

Get information about the current service, such as version, unique device ID, ABI, etc.

d.server_info()

>>> d.server_info()
uniqueId: "673abbe0-ff7b-9d82-1792-8876cb72cf56"
version: "9.25"
architecture: "arm64-v8a"
uptime: 293
secure: false

>>> result = d.server_info()
>>> print (result.secure)
False

Screen Off¶

The following interface will turn off the current device screen, equivalent to pressing the power button when the screen is on.

d.sleep()

Screen On¶

The following interface will turn on the current device screen, equivalent to pressing the power button when the screen is off.

d.wake_up()

Check if Screen is On¶

You can use the following interface to check whether the current device screen is on, to determine whether interaction is possible.

d.is_screen_on()

Check if Screen is Locked¶

You can use the following interface to check whether the current device screen is locked, to determine whether interaction is possible.

d.is_screen_locked()

Show Toast¶

You can use the following interface to display a Toast message “Hello from Lamda!” on the phone screen.

d.show_toast("Hello from Lamda!")

Read Clipboard¶

The following interface reads the current content of the device clipboard. Android 10+ is not yet supported.

d.get_clipboard()

Write Clipboard¶

The following interface writes content to the clipboard of the current device.

d.set_clipboard("clipboard content")

Physical Keys¶

You can use the following methods to simulate key presses. More than ten key codes are supported, such as KEY_BACK, KEY_DOWN, KEY_HOME, etc.

d.press_key(Keys.KEY_BACK)
d.press_key(Keys.KEY_CAMERA)
d.press_key(Keys.KEY_CENTER)
d.press_key(Keys.KEY_DELETE)
d.press_key(Keys.KEY_DOWN)
d.press_key(Keys.KEY_ENTER)
d.press_key(Keys.KEY_HOME)
d.press_key(Keys.KEY_LEFT)
d.press_key(Keys.KEY_MENU)
d.press_key(Keys.KEY_POWER)
d.press_key(Keys.KEY_RECENT)
d.press_key(Keys.KEY_RIGHT)
d.press_key(Keys.KEY_SEARCH)
d.press_key(Keys.KEY_UP)
d.press_key(Keys.KEY_VOLUME_DOWN)
d.press_key(Keys.KEY_VOLUME_MUTE)
d.press_key(Keys.KEY_VOLUME_UP)

In addition, to support more key codes, you can also use the following method. You can find all supported key names in the official Android documentation: https://developer.android.com/reference/android/view/KeyEvent.

d.press_keycode(KeyCodes.KEYCODE_CALL)

Screenshot¶

We provide a screenshot method so you can record processes or perform image recognition tasks. The following example shows how to capture a screenshot with a quality of 60 and save it as screenshot.png in the current directory.

d.screenshot(60).save("screenshot.png")

>>> result = d.screenshot(60)
>>> result
<lamda.types.BytesIO object at 0x7a28361d60>
>>> result.save("screenshot.png")
52917
>>> result.getvalue()
b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01....'

Currently, we also support region and element screenshots. Element screenshots are not covered in this chapter; here we introduce region screenshots. You need to understand the concept and definition of a region: in our API, a region is defined by Bound, which contains four parameters: top, left, right, and bottom. The Android screen coordinate system is different from the usual Cartesian system. The top-left corner of the screen is the origin, with coordinates x=0, y=0. Suppose the screen is in portrait mode with a resolution of 1080×1920. To take a full-screen region screenshot, you can do:

d.screenshot(60, bound=Bound(top=0, left=0, right=1080, bottom=1920)).save("screenshot.png")

This will produce a full-screen screenshot, though doing so is unnecessary because the default is already a full-screen capture. How about capturing a 200×200 image at the top-left corner? That's also simple; you just need to do the following:

d.screenshot(60, bound=Bound(top=0, left=0, right=200, bottom=200)).save("screenshot.png")

Here we need to explain the meaning of these parameters: top represents the offset in pixels downward along the Y-axis, bottom represents the offset downward along the Y-axis to the bottom edge, and left and right represent offsets in pixels to the right along the X-axis. top is always less than bottom, and left is always less than right. Let's demonstrate another example: capturing a 200×200 image at the bottom-right corner of the screen. You only need to call the API like this:

d.screenshot(60, bound=Bound(top=1920-200, bottom=1920, left=1080-200, right=1080)).save("screenshot.png")

Click a Point on the Screen¶

You can simulate a screen tap using the following interface. In the example, we click the point at coordinates (100, 100) on the screen.

d.click(Point(x=100, y=100))

Tap Point A and Drag to Point B¶

The following interface allows you to drag an object from point A to point B, for example, dragging an icon into a folder.

A = Point(x=100, y=100)
B = Point(x=500, y=500)

d.drag(A, B)

Swipe from Point A to Point B¶

The following interface achieves a swipe from screen coordinates (100, 100) to (500, 500).

A = Point(x=100, y=100)
B = Point(x=500, y=500)

d.swipe(A, B)

Slightly More Complex Multi-Point Swipe¶

The following interface can simulate multi-point swipes. For example, the call below will swipe from (100, 100) to (500, 500) and back to (200, 200). The demonstration only uses three points, but you can actually provide more points, even to realize a pattern unlock.

p1 = Point(x=100, y=100)
p2 = Point(x=500, y=500)
p3 = Point(x=200, y=200)

d.swipe_points(p1, p2, p3)

Open Quick Settings¶

The following call opens the quick settings panel, but only partially (half-open state).

d.open_quick_settings()

Open Notification Area¶

The following call opens the notification panel on the screen.

d.open_notification()

Get Page Layout¶

The following call retrieves the XML layout of the current page. You can parse it yourself to support XPath-based automation.

d.dump_window_hierarchy()

>>> result = d.dump_window_hierarchy()
>>> result
<lamda.types.BytesIO object at 0x7add660680>
>>> result.getvalue()
b'<?xml version=\'1.0\' encoding=\'UTF-8\' standalone=\'yes\' ?>\r\n<hierarchy rotation="0">\r\n  <node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.android.systemui" content-desc="" checkable="false" checked="false" clickable="false"...

Wait for Window Idle¶

You can call the following to wait for the current window to become idle, similar to Selenium's implicitly_wait. The parameter unit is milliseconds; 5*1000 means waiting 5000 milliseconds (i.e., 5 seconds).

d.wait_for_idle(5*1000)

Get the Most Recent Toast¶

You can call the following to get the most recent Toast message in the system. Note that only Toast messages sent via the system API are supported; third-party Toast-like UI messages cannot be obtained.

d.get_last_toast()

>>> result = d.get_last_toast()
>>> print (result)
timestamp: 1700000000000
package: "com.android.settings"
message: "\346\202\250\345\267\262\345\244\204\344\272\216\345\274\200\345\217\221\350\200\205\346\250\241\345\274\217\357\274\214\346\227\240\351\234\200\350\277\233\350\241\214\346\255\244\346\223\215\344\275\234\343\200\202"
>>> print (result.message)
您已处于开发者模式，无需进行此操作。