Multi-touch¶

In addition to regular automation, we also provide advanced multi-touch interfaces, allowing you to implement complex finger operations. It includes features such as hardware gesture recording, playback, construction, and saving.

To help you quickly understand how to use it, we recommend enabling 'Show taps' and 'Pointer location' in Developer options -> Input (you may not see any feedback if these are not enabled). The following example code shows you how to draw an X symbol in the upper-left corner of the screen.

touch = d.touch()
finger0 = touch.contact(0)
finger1 = touch.contact(1)

# --- 1. Press down at the starting points ---
finger0.down(150, 320)
finger1.down(450, 320)
touch.wait(20)

# --- 2. Move along the path (10 coordinate points) ---
finger0.move(180, 350)
finger1.move(420, 350)
touch.wait(20)

finger0.move(210, 380)
finger1.move(390, 380)
touch.wait(20)

finger0.move(240, 410)
finger1.move(360, 410)
touch.wait(20)

finger0.move(270, 440)
finger1.move(330, 440)
touch.wait(20)

finger0.move(300, 470) # Intersection point
finger1.move(300, 470)
touch.wait(20)

finger0.move(330, 500)
finger1.move(270, 500)
touch.wait(20)

finger0.move(360, 530)
finger1.move(240, 530)
touch.wait(20)

finger0.move(390, 560)
finger1.move(210, 560)
touch.wait(20)

finger0.move(420, 590)
finger1.move(180, 590)
touch.wait(20)

finger0.move(450, 620) # End points
finger1.move(150, 620)
touch.wait(20)

# --- 3. Lift the fingers ---
finger0.up()
finger1.up()

# Perform the action
touch.perform()

Record Gestures¶

To make gesture construction easier, we also provide a recording feature. This feature requires you to operate a physical device.

# Get a multi-touch session
touch = d.touch()

# Record a gesture. After calling, it will wait for you to operate the physical device screen within the timeout period (120 seconds). It will return after your fingers are lifted.
touch.record()

# You can save the gesture as a binary file
touch.save("/sdcard/touch/bin")

# You can also play back the gesture directly
touch.perform()

Save Gestures¶

To avoid reconstructing the gesture with code every time, we provide a feature to save gestures, allowing you to persist gesture data to disk.

# Get a multi-touch session
touch = d.touch()

# You can save the just-constructed gesture for later loading
touch.save("/sdcard/track0.bin")

Load Gestures¶

You can directly load a gesture into a session from a gesture file stored on disk.

# Get a multi-touch session
touch = d.touch()

# Load a gesture from a file
touch.load("/sdcard/track0.bin")

Construct Gestures¶

Besides recording, you can also choose to construct touch gestures programmatically. Below, we will introduce manual construction and other available features. A valid touch operation consists of three types of events: down, move, and up. You can set pressure parameters for events like down and move.

# Get a multi-touch session
touch = d.touch()

# Get two (or more) finger contacts
finger0 = touch.contact(0)
finger1 = touch.contact(1)

# Finger 0 touches down at coordinates 150, 320
# The parameter z represents finger pressure, with a valid range of 1-255, defaulting to 128
finger0.down(150, 320, z=128)

# Finger 0 moves from 150, 320 to 180, 350
finger0.move(180, 350, z=128)

# Wait for 20 milliseconds (equivalent to finger0.wait(20))
touch.wait(20)

# Finger 0 lifts up
finger0.up()

# Clear all gestures
touch.reset()

# Perform the action (wait determines whether to wait for completion)
touch.perform(wait=True)