# Advanced Interface

This chapter introduces more advanced automation interfaces. You can use these interfaces to perform a variety of detailed operations. The content in this chapter is extensive. If this is your first encounter, we recommend reading through each section patiently.

```{tip}
When writing automation code, you can directly enter the command `lamda` in the terminal on the right side of the remote desktop and execute the following test code inside it, or manually select elements, perform click tests, etc., to speed up both coding and verification.
```

## Getting Elements

You may have already gained some understanding of this in the Basics or earlier sections. You need to locate relevant elements via selectors to perform operations. You should also have seen where to obtain selector parameters. The following introduction will revolve around these elements. You can see information about the element “同意” (Agree) on the right side of this image.

![Sample Element](/assets/images/auto-eyeselect.png)

```{attention}
The element you click directly in the left interface may not be the actual element because it might overlap in size and position with other elements. Typically, multiple overlapping elements will be listed in the information panel on the right; you can scroll up and down to see which one you actually need. You can also manually traverse all elements by pressing the TAB key in the left selection interface.
```

For the above element, we generally obtain it via `text`. Using `text` works if there is no other element on the current screen that also has the text “同意”—this is the simplest method. Alternatively, you can use `resourceId`, but note that `resourceId` here does not represent a unique ID; it represents a resource ID, and a single screen may contain many elements with the same resource ID. Other fields like `packageName`, `checkable`, etc., are not commonly used, but if `text`, `resourceId`, `description`, etc., are unavailable, you can try these fields. We can locate this element in the following ways:

```python
element = d(text="同意")
element = d(text="同意", resourceId="com.tencent.news:id/btm_first_agree")
element = d(resourceId="com.tencent.news:id/btm_first_agree")
```

## Element Click

Call the following interface to perform a normal element click, which will achieve the effect of clicking “同意” (Agree) manually in the context.

```python
element.click()
```

If you need to specify the click position on the element, you can provide the `corner` parameter when calling the click method. For example, `Corner.COR_CENTER` means clicking the center of the element, and you can also click its top-left corner or bottom-right corner (`Corner.COR_BOTTOMRIGHT`).

```python
element.click_exists(corner=Corner.COR_TOPLEFT)
```

Perform a long click on this element, raising an exception if it does not exist. This interface also supports `corner`, but you cannot specify the long‑click duration.

```python
element.long_click()
```

Click the element if it exists; if not, calling this method will not raise an exception. This interface also supports `corner`.

```python
element.click_exists()
```

```python
>>> element.click_exists()
True
```

## Existence Check

In many situations, it is necessary to check the existence of an element before proceeding with further operations; otherwise, the subsequent flow may encounter exceptions or even perform wrong actions on an incorrect screen. You can use the following interface to determine existence.

```python
element.exists()
```

## Element Information

In some cases, you may want to retrieve part of an element’s information, such as its coordinates, region information, or the text, description, etc., displayed on the element. You can read element information with the following interface.

```python
element.info()
```

For our test element above, the output information is as follows.

```python
>>> info = element.info()
>>> print (info)
bounds { ... }
className: "android.widget.TextView"
clickable: true
enabled: true
focusable: true
packageName: "com.tencent.news"
resourceName: "com.tencent.news:id/btn_first_agree"
text: "\345\220\214\346\204\217"
visibleBounds { ... }
```

```{hint}
You may notice that the printed information is missing some fields, such as `description`. This usually indicates that the field value is empty or false; you can still access the related field normally via the attribute to obtain its value.
```

As you can see, this information is somewhat complex, in the default protobuf printing format. You can directly access the corresponding attributes to print their actual values. For example, to read the element’s text value, you can simply use the following.

```python
>>> info = element.info()
>>> print (info.text)
同意
```

Of course, there is also element region/coordinate information, which you can likewise access. For instance, if you want to obtain the region information corresponding to this element, you can print the region as shown below, or save it as a variable for later operations.

```python
>>> info = element.info()
>>> print (info.bounds)
top: 947
left: 338
bottom: 997
right: 743
```

The output or returned value is a region object (`Bounds`). You will find that this parameter is also used by some screenshot interfaces. You can pass this parameter to a screenshot interface to capture only that element. However, we have already provided a more convenient method for you.

You may also want to obtain the element’s width and height to calculate offsets, such as relative offsets of other elements. You can use:

```python
>>> info = element.info()
>>> print (info.bounds.width, info.bounds.height)
484 138
```

Or obtain the center or corner points of the element, such as top-left, bottom-right, etc. The following interfaces typically return a `Point` object, from which you can retrieve the corresponding X and Y screen coordinates.

```python
>>> info = element.info()
>>> print (info.bounds.center())
x: 792
y: 1908
>>> print (info.bounds.center().x)
792
```

The following call obtains the coordinates of a corner point. This example gets the coordinates of the top-left corner; additionally, you can obtain the coordinates of the other three corners: `bottom-right`, `top-right`, and `bottom-left`.

```python
>>> info = element.info()
>>> print (info.bounds.corner("top-left"))
x: 550
y: 1839
>>> print (info.bounds.corner("top-left").x)
550
```

## Element Traversal

You can also traverse all elements matched by a selector. Under normal circumstances, this selector in the current context may match only one element. If you want to test traversal, choose a selector that matches multiple elements. You can directly use a `for` loop or other methods on the selector to iterate.

```python
for i in element: print (i.info())
```

Or if you know there are multiple matching elements and want a specific Nth match, you can obtain it as follows.

```python
element_3rd = element.get(3)
```

## Element Count

You usually will not use this interface directly. The following call returns the number of elements matched by your current selector.

```python
>>> element.count()
1
```

## Element Screenshot

We support element‑level screenshots; you can capture just the element’s image without taking a full‑screen screenshot and then cropping it.

```python
element.screenshot(quality=60)
```

After capturing, you can directly use the `getvalue()` method to obtain the binary data of the screenshot, or pass it directly to `PIL.Image`.

```python
>>> element.screenshot(quality=60).getvalue()
b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01\x01\x00\x00\x01\x00\x01\x00\x00\xff\xe2\x02(ICC_PROFILE\x00\x01\x01\x00\x00\x02\x18\x00\x00\x00\x00\x02\x10\x00\x00mntrRGB XYZ \x00\x00...
```
Or, if you do not need further processing, you can save the screenshot directly to a local file.
```python
>>> element.screenshot(quality=60).save("image.png")
```

## Waiting for an Element

In some situations, you may need to determine whether the current page has finished loading. This can often be judged by checking whether related elements have appeared. The following example waits for the “同意” element to appear, with a maximum wait time of 10 seconds.

```{hint}
The wait duration here is in milliseconds, so 10 seconds means `*1000`; 10 seconds = 10000 milliseconds.
```

```python
element.wait_for_exists(10*1000)
```

```python
>>> element.wait_for_exists(10*1000)
True
```

Additionally, we support waiting for an element to disappear, i.e., waiting until the element is gone from the screen.

```python
element.wait_until_gone(10*1000)
```

```python
>>> element.wait_until_gone(10*1000)
False
```

## Text Input

Text input requires special attention. You cannot input text onto a button because it is a button. Now we will reselect an input field element to introduce this; the basic information of this element is as follows.

![Text Input](/assets/images/input-text.png)

```{attention}
There are some points to note when capturing an input field element: please note that when locating an input field element, your input method must be in **pop‑up state**, and then you should search for the related element. It is recommended to look carefully, otherwise you might not find the real input field.
```

```{hint}
In an automation flow, all you need to do to make the input method pop up is to first click the input field displayed in the parent container in your code.
```

For the above input field, we can call the following interface to enter the string “你好世界” (Hello World). You can also input English or other Unicode strings; simply use it as follows to input text into the box.

```python
>>> element = d(text="搜索感兴趣的内容")
>>> element.set_text("你好世界")
True
```

If you want to retrieve the currently displayed text content of that input field, you can call it as follows.

```{attention}
Please note that here we have changed the selector. The initial selector used the `text` attribute, but after entering text, the element’s content changed, causing the original selector to no longer match; therefore we switched to another selector. Choosing the right selector is important, but this example is for demonstration only, so it will do as such.
```

```python
>>> element = d(className="android.widget.EditText")
>>> element.get_text()
'你好世界'
```

You can also clear the currently input content. Usually, inputting text automatically clears the existing text, but you can also clear it manually.

```{hint}
Pressing the BACKSPACE key repeatedly via the key interface in a loop can achieve a similar effect.
```

```python
>>> element = d(className="android.widget.EditText")
>>> element.clear_text_field( )
True
```

```{note}
In extreme cases, there may be some places where this interface cannot be used to input text normally; we are working on supporting these.
```

## Normal Swiping

Use the following interface to perform swipe operations on the screen, such as scrolling up and down in a list. The following call performs an upward swipe; the larger the `step` value, the slower the swipe, making it suitable for swipes requiring higher precision.

```{attention}
In simple cases, this operation does not require providing a selector parameter. If you encounter situations where swiping does not work, set the selector condition to a suitable element, such as an element with the `scrollable` attribute or the first‑level container of the list.
```

```python
d().swipe(direction=Direction.DIR_UP, step=32)
```

```python
>>> element = d(resourceId="com.tencent.news:id/important_list_content")
>>> element.swipe(direction=Direction.DIR_UP, step=32)
True
```

| Direction               | Description        |
|-------------------------|--------------------|
| Direction.DIR_UP        | Swipe up           |
| Direction.DIR_LEFT      | Swipe left         |
| Direction.DIR_DOWN      | Swipe down         |
| Direction.DIR_RIGHT     | Swipe right        |

## Fling (Fast Swipe)

Flinging resembles a human’s fast swipe behavior. This operation will quickly swipe the screen and is suitable for simulating rapid browsing actions. The following example flings from top to bottom; the selector is empty in the example, but you still need to decide whether to provide a selector based on the actual situation.

```python
d().fling_from_top_to_bottom()
```

Fling from bottom to top:

```python
d().fling_from_bottom_to_top()
```
Fling from left to right:
```python
d().fling_from_left_to_right()
```
Fling from right to left:
```python
d().fling_from_right_to_left()
```

```{attention}
In simple cases, this operation does not require providing a selector parameter. If you encounter situations where flinging does not work, set the selector condition to a suitable element, such as an element with the `scrollable` attribute or the first‑level container of the list.
```

```python
>>> element = d(resourceId="com.tencent.news:id/important_list_content")
>>> element.fling_from_bottom_to_top()
True
```

## Element Dragging

Drag an element to the position of another element (e.g., dragging an app icon into a folder).

```python
element.drag_to(Selector(text="购物"))  # Drag to the target element's location
```

## Child and Sibling Queries

For repeated or featureless elements, you can first locate a parent container, then use `child` to get child elements and `sibling` to get sibling elements, thus narrowing down the scope.

```python
form = d(resourceId="login_form")              # Locate the parent container
form.child().get(1)                            # Get the first child element under form
form.sibling(textContains="找回密码")           # Get a sibling element of form whose text contains "找回密码"

# Here is a slightly more complex query
# Obtain the first result matching resourceId=com.example.com:id/resource, select its child node with resourceId=com.example.com:id/abc, and within that child node, query for an element whose description contains "一天内", then output its information.
d(resourceId="com.example.com:id/resource").get(0).child(resourceId="com.example.com:id/abc").child(descriptionContains="一天内").info()
```

For the example layout information below, you can use the following query to precisely select that element.

<p align="center">
<img src="/assets/images/example_child_sibling_layout.png" alt="Example Layout" width="100%">
</p>

```python
d(resourceId="com.zhiliaoapp.musically:id/bxa").child().get(3).child().get(1).info()
```

```{hint}
In most cases, you do not need such precise child/sibling query statements; a single `d(text="Continue with Google")` is sufficient, except when it is truly impossible to locate by text.
```

## Fling to Edge

Continuously fling in one direction until no further scrolling is possible; it may not always detect reaching the end, so you must specify `max_swipes`.

```python
d().fling_from_top_to_bottom_to_end(max_swipes=32)   # Fling down to the edge
d().fling_from_bottom_to_top_to_end(max_swipes=32)   # Fling up
d().fling_from_left_to_right_to_end(max_swipes=32)   # Fling right
d().fling_from_right_to_left_to_end(max_swipes=32)   # Fling left
```

## Scroll (Constant-speed Swipe)

Scroll with a fixed step size `step`, more mechanical than `swipe`, suitable for scenarios requiring steady stepping.

```python
d().scroll_from_top_to_bottom(step=60)    # Down
d().scroll_from_bottom_to_top(step=60)    # Up
d().scroll_from_left_to_right(step=60)    # Right
d().scroll_from_right_to_left(step=60)    # Left
```

## Scroll to Edge

Similar to flinging to the edge, but using constant‑speed scrolling instead of `fling`. Again you must specify `max_swipes` and `step`.

```python
d().scroll_from_top_to_bottom_to_end(max_swipes=32, step=60)   # Scroll down to the edge
d().scroll_from_bottom_to_top_to_end(max_swipes=32, step=60)   # Scroll up
d().scroll_from_left_to_right_to_end(max_swipes=32, step=60)   # Scroll right
d().scroll_from_right_to_left_to_end(max_swipes=32, step=60)   # Scroll left
```