Image Matching¶
This section introduces how to use template matching to locate and click elements. FIRERPA supports both template matching and feature point matching (SIFT). Before you begin, you need to understand some basics about image matching. Image matching is divided into template matching and feature point matching. Template matching is suitable for devices with the same resolution and is more efficient. Feature point matching is suitable for screens with different resolutions, but some thresholds may need fine-tuning according to the screen size of different devices.
Hint
All matching processes are performed on the FIRERPA server and do not consume local resources. However, due to mobile device performance limitations, the efficiency is lower. If performance is a concern, you can capture the screenshot locally and perform matching on your own machine.
Matching Interface¶
The image matching interface is a relatively complex interface, but in most cases, you only need to adjust the threshold and distance parameters. You can find the meanings of each parameter in the parameter explanation below. The main form of the interface is as follows, by default performing full-screen matching using the template matching method. This interface is mainly used to save your host's computing resources. If you have requirements for performance or recognition results, you can also use the screenshot interface to capture and match images yourself.
d.find_similar_image(data, threshold=0.0, distance=250, scale=1.0, area=FindImageArea.FIA_WHOLE_SCREEN, method=FindImageMethod.FIM_TEMPLATE)
Matching Method¶
The matching method supports template matching and feature point matching. Template matching is suitable for cases with obvious texture and fixed target shapes, but is not robust to rotation, scale changes, and illumination variations. Feature point matching detects image feature points such as corners and edges, describes the features, and then matches them between two images. It is more robust to rotation, scale changes, and illumination variations, making it suitable for complex scene matching tasks.
Matching Area¶
The matching area is mainly used to balance mobile device performance. In most cases, the items to be matched are in a fixed area of the screen. You can reduce the image size by specifying the area where they are located, thus reducing the computational load during matching.