Basic Automation Operations¶
This chapter describes the basic automation features provided by FIRERPA. More advanced usage will be covered later. This chapter introduces some commonly used basic function interfaces.
Get Device Information¶
Gets current device information such as device name, screen size, screen orientation, and the current application.
d.device_info()
>>> d.device_info()
productName: "bumblebee"
sdkInt: 34
displayHeight: 2400
displaySizeDpX: 411
displaySizeDpY: 914
displayWidth: 1080
screenOn: true
naturalOrientation: true
currentPackageName: "com.android.launcher3"
>>> result = d.device_info()
>>> print (result.displayWidth)
1080
Get Service Information¶
Gets information about the current service, such as its version, device unique ID, architecture, ABI, etc.
d.server_info()
>>> d.server_info( )
uniqueId: "673abbe0-ff7b-9d82-1792-8876cb72cf56"
version: "8.28"
architecture: "arm64-v8a"
uptime: 293
secure: True
>>> result = d.server_info()
>>> print (result.secure)
False
Turn Screen Off¶
The following interface will turn off the current phone screen, which is equivalent to pressing the power button when the screen is on.
d.sleep()
Turn Screen On¶
The following interface will turn on the current phone screen, which is equivalent to pressing the power button when the screen is off.
d.wake_up()
Check if Screen is On¶
You can use the following interface to check if the device's screen is on, to determine if operations can be performed.
d.is_screen_on()
Check if Screen is Locked¶
You can use the following interface to check if the device's screen is unlocked, to determine if operations can be performed.
d.is_screen_locked()
Show Toast¶
You can use the following interface to display a Toast message "Hello from Lamda!" on the phone's screen.
d.show_toast("Hello from Lamda!")
Read Clipboard¶
The following interface is used to read the current content of the phone's clipboard. It does not currently support Android 10+.
d.get_clipboard()
Write to Clipboard¶
The following interface is used to write content to the current device's clipboard.
d.set_clipboard("Clipboard content")
Physical Keys¶
You can use the following methods to simulate key presses. It supports more than ten keys such as KEY_BACK, KEY_DOWN, KEY_HOME, etc.
d.press_key(Keys.KEY_BACK)
d.press_key(Keys.KEY_CAMERA)
d.press_key(Keys.KEY_CENTER)
d.press_key(Keys.KEY_DELETE)
d.press_key(Keys.KEY_DOWN)
d.press_key(Keys.KEY_ENTER)
d.press_key(Keys.KEY_HOME)
d.press_key(Keys.KEY_LEFT)
d.press_key(Keys.KEY_MENU)
d.press_key(Keys.KEY_POWER)
d.press_key(Keys.KEY_RECENT)
d.press_key(Keys.KEY_RIGHT)
d.press_key(Keys.KEY_SEARCH)
d.press_key(Keys.KEY_UP)
d.press_key(Keys.KEY_VOLUME_DOWN)
d.press_key(Keys.KEY_VOLUME_MUTE)
d.press_key(Keys.KEY_VOLUME_UP)
To support more keys, you can also use this method for simulation. You can find all supported key names in the official Android documentation here: https://developer.android.com/reference/android/view/KeyEvent.
d.press_keycode(KeyCodes.KEYCODE_CALL)
Screenshot¶
We provide a method for taking screenshots, allowing you to record workflows or perform image recognition tasks. The following call captures an image with a quality of 60 and saves it to the screenshot.png file in the current directory.
d.screenshot(60).save("screenshot.png")
>>> result = d.screenshot(60)
>>> result
<lamda.types.BytesIO object at 0x7a28361d60>
>>> result.save("screenshot.png")
52917
>>> result.getvalue( )
b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01....'
Currently, we also support region and element screenshots. Element screenshots are not covered in this chapter. Here, we will introduce region screenshots. You need to understand the concept and definition of a region. In our interface, a region is defined by Bound, which includes four parameters: top, left, right, and bottom. The Android screen coordinate system is different from a conventional one; the top-left corner of the device screen is the origin, with coordinates x=0, y=0. Assuming the current screen is in portrait mode with a resolution of 1080x1920, if you want to take a full-screen region screenshot, you can do it like this.
d.screenshot(60, bound=Bound(top=0, left=0, right=1080, bottom=1920)).save("screenshot.png")
This will produce a full-screen screenshot. Of course, this is unnecessary because the default is a full-screen screenshot. So, if we want to capture a 200x200 image at the top-left corner, how do we do it? It's also very simple, you just need to do the following.
d.screenshot(60, bound=Bound(top=0, left=0, right=200, bottom=200)).save("screenshot.png")
At this point, we should explain the meaning of these parameters. In the parameters, top and bottom represent the y-coordinates of the top and bottom edges of the rectangle, while left and right represent the x-coordinates of the left and right edges. Note that top must always be less than bottom, and left must always be less than right. Now let's demonstrate another example: capturing a 200x200 image at the bottom-right corner of the screen. You just need to call the interface like this.
d.screenshot(60, bound=Bound(top=1920-200, bottom=1920, left=1080-200, right=1080)).save("screenshot.png")
Click a Point on the Screen¶
You can use the following interface to simulate a screen click. In the example, we click the point at coordinates (100, 100) on the screen.
d.click(Point(x=100, y=100))
Drag from Point A to Point B¶
The following interface can be used to drag an item from point A to point B, for example, dragging an icon into a folder.
A = Point(x=100, y=100)
B = Point(x=500, y=500)
d.drag(A, B)
Swipe from Point A to Point B¶
The following interface can be used to swipe from coordinates (100, 100) to (500, 500) on the screen.
A = Point(x=100, y=100)
B = Point(x=500, y=500)
d.swipe(A, B)
More Complex Multi-point Swipe¶
The following interface can simulate a multi-point swipe. For example, the following call will swipe from (100, 100) to (500, 500) and then to (200, 200). The demonstration below only uses three points, but you can actually provide more points, even to implement a pattern lock.
p1 = Point(x=100, y=100)
p2 = Point(x=500, y=500)
p3 = Point(x=200, y=200)
d.swipe_points(p1, p2, p3)
Open Quick Settings¶
The following call can open the Quick Settings panel on the screen, but in a partially open state.
d.open_quick_settings()
Open Notification Shade¶
The following call can open the notification shade on the screen.
d.open_notification()
Get Page Layout¶
The following call can get the XML layout of the current page. You can also parse it yourself to support XPath-based automation.
d.dump_window_hierarchy()
>>> result = d.dump_window_hierarchy()
>>> result
<lamda.types.BytesIO object at 0x7add660680>
>>> result.getvalue()
b'<?xml version=\'1.0\' encoding=\'UTF-8\' standalone=\'yes\' ?>\r\n<hierarchy rotation="0">\r\n <node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.android.systemui" content-desc="" checkable="false" checked="false" clickable="false"...
Wait for Window to be Idle¶
You can execute the following call to wait for the current window to be in an idle state, similar to Selenium's implicitly_wait. The parameter unit is milliseconds, so 5*1000 means waiting for 5 seconds (1000 milliseconds = 1 second).
d.wait_for_idle(5*1000)
Get Last Toast¶
You can execute the following call to get the most recent Toast message from the system. However, note that it only supports Toast messages sent via the system API; messages from third-party Toast-like UI implementations cannot be retrieved.
d.get_last_toast()
>>> result = d.get_last_toast()
>>> print (result)
timestamp: 1700000000000
package: "com.android.settings"
message: "\346\202\250\345\267\262\345\244\204\344\272\216\345\274\200\345\217\221\350\200\205\346\250\241\345\274\217\357\274\214\346\227\240\351\234\200\350\277\233\350\241\214\346\255\244\346\223\215\344\275\234\343\200\202"
>>> print (result.message)
You are already a developer, no need to perform this operation.