Image Matching Operations

This chapter introduces how to use template matching methods to find and click elements. FIRERPA supports both template matching and feature point matching (SIFT). Before starting, you need to understand basic information about image matching. Image matching is divided into template and feature point matching. Template matching is suitable for devices with the same resolution and is more efficient. Feature point matching is suitable for screens with different resolutions, but some thresholds may need to be fine-tuned according to the size of different device screens.

Hint

All matching processes take place on the FIRERPA server and do not occupy local resources. However, due to mobile performance limitations, efficiency is relatively low. For performance requirements, you can take screenshots and match them locally.

Matching Interface

The image matching interface is a slightly complex interface, but in most cases, the only parameters you need to adjust are threshold and distance. You can also find the meaning of each parameter in the parameter interpretation below. The main form of the interface is as follows. By default, full-screen matching is performed using the template matching method. This interface is mainly to save your host’s computing resources. If you have requirements for performance or recognition results, you can also use the screenshot interface to take screenshots and match them yourself.

d.find_similar_image(data, threshold=0.0, distance=250, scale=1.0, area=FindImageArea.FIA_WHOLE_SCREEN, method=FindImageMethod.FIM_TEMPLATE)
Field Description
data Byte data of the image to be matched (cropped image)
threshold Discard threshold (similarity)
distance Maximum feature point distance (feature matching only)
scale Scale matching (improve performance, not applicable to template matching)
area Matching area (limit area to improve performance)
method Matching method

Matching Methods

Matching methods support both template and feature point matching. Template matching is suitable for situations with obvious textures and fixed target shapes, but it is not robust enough for rotation, scale changes, and lighting changes. Feature point matching detects image feature points such as corners and edges, describes the features, and then matches them between two images. It has strong robustness to rotation, scale changes, lighting changes, etc., and is suitable for complex scene matching tasks.

Matching Method Description
FindImageMethod.FIM_TEMPLATE Template Matching
FindImageMethod.FIM_FEATURE Feature Point Matching

Matching Areas

Matching areas are mainly used to balance mobile performance. In most cases, the items to be matched are in a fixed area of the screen. You can reduce the image size by specifying its location area, thereby reducing the amount of calculation during matching.

Matching Area Description
FindImageArea.FIA_WHOLE_SCREEN Full-screen matching
FindImageArea.FIA_LEFT Match left half of screen
FindImageArea.FIA_TOP_LEFT Match top-left corner of screen
FindImageArea.FIA_TOP Match top half of screen
FindImageArea.FIA_TOP_RIGHT Match top-right corner of screen
FindImageArea.FIA_RIGHT Match right half of screen
FindImageArea.FIA_BOTTOM_RIGHT Match bottom-right corner of screen
FindImageArea.FIA_BOTTOM Match bottom half of screen
FindImageArea.FIA_BOTTOM_LEFT Match bottom-left corner of screen