Image Matching Operations¶
This section introduces how to use template matching methods to locate and click elements. FIRERPA supports both template matching and feature point matching (SIFT). Before getting started, you should understand the basics of image matching. Image matching is divided into template matching and feature point matching. Template matching is suitable for devices with identical screen resolutions and offers higher efficiency. Feature point matching works across screens with different resolutions, but certain thresholds may require fine-tuning depending on the screen size of the device.
Hint
All matching processes are performed on the FIRERPA server side and do not consume local machine resources. However, due to performance limitations on mobile devices, matching efficiency may be relatively low. For better performance, users can perform screenshot matching locally.
Matching Interface¶
The image matching interface is slightly complex, but in most cases, you only need to adjust the threshold and distance parameters. The meanings of all parameters are explained below. The primary form of this interface is as follows. By default, it performs full-screen matching using the template matching method. This interface aims to save computational resources on your host machine. If higher performance or more accurate recognition results are required, you may alternatively use the screenshot interface to perform matching locally.
d.find_similar_image(data, threshold=0.0, distance=250, scale=1.0, area=FindImageArea.FIA_WHOLE_SCREEN, method=FindImageMethod.FIM_TEMPLATE)
| Field | Description |
|---|---|
| data | Byte data of the image to be matched (cropped image) |
| threshold | Discard threshold (similarity score) |
| distance | Maximum feature point distance (used only in feature matching) |
| scale | Scaling for matching (improves performance; not applicable for template matching) |
| area | Matching area (limiting area improves performance) |
| method | Matching method |
Matching Methods¶
Matching methods support both template matching and feature point matching. Template matching is suitable for targets with clear textures and fixed shapes, but it is not robust against rotation, scale changes, or lighting variations. Feature point matching detects key points in images such as corners and edges, describes their features, and then matches them between two images. It is highly robust to rotation, scale changes, lighting variations, and is suitable for complex scene-matching tasks.
| Matching Method | Description |
|---|---|
| FindImageMethod.FIM_TEMPLATE | Template matching |
| FindImageMethod.FIM_FEATURE | Feature point matching |
Matching Areas¶
Matching areas are primarily used to balance performance on mobile devices. In most cases, the target element resides in a fixed region of the screen. By specifying this region, you can reduce the image size and thus decrease computational load during matching.
| Matching Area | Description |
|---|---|
| FindImageArea.FIA_WHOLE_SCREEN | Full-screen matching |
| FindImageArea.FIA_LEFT | Match left half of the screen |
| FindImageArea.FIA_TOP_LEFT | Match top-left corner of the screen |
| FindImageArea.FIA_TOP | Match upper half of the screen |
| FindImageArea.FIA_TOP_RIGHT | Match top-right corner of the screen |
| FindImageArea.FIA_RIGHT | Match right half of the screen |
| FindImageArea.FIA_BOTTOM_RIGHT | Match bottom-right corner of the screen |
| FindImageArea.FIA_BOTTOM | Match lower half of the screen |
| FindImageArea.FIA_BOTTOM_LEFT | Match bottom-left corner of the screen |