Image Matching Operations¶
This chapter introduces how to use template matching methods to find and click elements. FIRERPA supports both template matching and feature point matching (SIFT). Before starting, you need to understand basic information about image matching. Image matching is divided into template and feature point matching. Template matching is suitable for devices with the same resolution and is more efficient. Feature point matching is suitable for screens with different resolutions, but some thresholds may need to be fine-tuned according to the size of different device screens.
Hint
All matching processes take place on the FIRERPA server and do not occupy local resources. However, due to mobile performance limitations, efficiency is relatively low. For performance requirements, you can take screenshots and match them locally.
Matching Interface¶
The image matching interface is a slightly complex interface, but in most cases, the only parameters you need to adjust are threshold and distance. You can also find the meaning of each parameter in the parameter interpretation below. The main form of the interface is as follows. By default, full-screen matching is performed using the template matching method. This interface is mainly to save your host’s computing resources. If you have requirements for performance or recognition results, you can also use the screenshot interface to take screenshots and match them yourself.
d.find_similar_image(data, threshold=0.0, distance=250, scale=1.0, area=FindImageArea.FIA_WHOLE_SCREEN, method=FindImageMethod.FIM_TEMPLATE)
| Field | Description |
|---|---|
| data | Byte data of the image to be matched (cropped image) |
| threshold | Discard threshold (similarity) |
| distance | Maximum feature point distance (feature matching only) |
| scale | Scale matching (improve performance, not applicable to template matching) |
| area | Matching area (limit area to improve performance) |
| method | Matching method |
Matching Methods¶
Matching methods support both template and feature point matching. Template matching is suitable for situations with obvious textures and fixed target shapes, but it is not robust enough for rotation, scale changes, and lighting changes. Feature point matching detects image feature points such as corners and edges, describes the features, and then matches them between two images. It has strong robustness to rotation, scale changes, lighting changes, etc., and is suitable for complex scene matching tasks.
| Matching Method | Description |
|---|---|
| FindImageMethod.FIM_TEMPLATE | Template Matching |
| FindImageMethod.FIM_FEATURE | Feature Point Matching |
Matching Areas¶
Matching areas are mainly used to balance mobile performance. In most cases, the items to be matched are in a fixed area of the screen. You can reduce the image size by specifying its location area, thereby reducing the amount of calculation during matching.
| Matching Area | Description |
|---|---|
| FindImageArea.FIA_WHOLE_SCREEN | Full-screen matching |
| FindImageArea.FIA_LEFT | Match left half of screen |
| FindImageArea.FIA_TOP_LEFT | Match top-left corner of screen |
| FindImageArea.FIA_TOP | Match top half of screen |
| FindImageArea.FIA_TOP_RIGHT | Match top-right corner of screen |
| FindImageArea.FIA_RIGHT | Match right half of screen |
| FindImageArea.FIA_BOTTOM_RIGHT | Match bottom-right corner of screen |
| FindImageArea.FIA_BOTTOM | Match bottom half of screen |
| FindImageArea.FIA_BOTTOM_LEFT | Match bottom-left corner of screen |