Frida Data Reporting¶

The Frida data reporting feature is based on the persistence feature. You can use your own Frida scripts to automatically capture method call data and report it through our specific methods. The data reporting feature allows you to easily upload intercepted data directly to Redis, MQTT, or an external HTTP API. You can receive the reported data through Redis, MQTT, or an HTTP API. To maximize network efficiency, compressed reporting (zlib) is also supported.

Writing Reporting Scripts¶

Usually Frida scripts have features such as send and log that allow you to send data to the outside. However, for FIRERPA, you need to use specific methods to send data out. The following uses our template code for OkHttp traffic interception as a demonstration. It is only a demo script and may not work as-is. This script is not much different from a regular script; the only difference is the use of an emit method, which is a built-in FIRERPA method. You can use it to conveniently and systematically submit data to the outside.

Java.perform(function() {
        Java.use("com.android.okhttp.internal.http.HttpEngine").getResponse.implementation = function() {
                var response = this.getResponse()
                var data = {}
                data["url"] = response._request.value._url.value._url.value
                data["body"] = response.body().string()
                emit("report_data", JSON.stringify(data))
                return response
        }
})

The submission method has two parameters: emit(name, content). name represents the type of data. If the reporting destination you set is Redis, then this name represents the Redis queue name. You should describe it in English as accurately as possible, such as product_info. content represents the data content, and the type only supports strings and byte arrays. In the example, we converted it into a string for submission.

Now that you understand the format and calling requirements for writing scripts, that is, how to submit hooked data to the outside, you should continue reading below to learn how to configure the data reporting destination.

Data Reporting Destinations¶

The data reporting destination represents where the data emitted in your script should be sent. Supported destination types include HTTP APIs, Redis queues, and MQTT. There are some differences between them.

Generally, if you do not need to care about information such as data source, that is, you do not need to match data with the source device, we recommend using a Redis queue. Otherwise, you should use HTTP or MQTT (v5), because the features of these two protocols allow us to carry more metadata in the protocol, enabling more precise device matching.

Reporting Metadata¶

For data reported to HTTP APIs and MQTT destinations, in addition to the original reported data, you can also obtain metadata about the device and script at the protocol layer. The metadata included in the protocol is shown in the following table.

device is the unique ID of the device. You can find it in the information bar of the remote desktop. Usually this ID is unique and fixed. You can use it to identify the device and establish correspondence. encode is the data encoding. Supported encodings are none and zlib. If the encoding is zlib, you need to use zlib to decompress the data body. name is used to mark the type of this reported data, and it is also the first parameter when you use the emit method. sequence represents the index of the reported data. It starts from 0 and increments with each report. You can use this field to sort the data or check whether any report is lost.

Additional Parameters in Reporting Links¶

You can specify dynamic ID parameters in the reporting link using variable placeholders in the format ${name} inserted at specific positions in the link. For example: http://192.168.1.2/report/${device_id}. Supported variables are shown in the following table.

Reporting to HTTP¶

You need to write an HTTP service that receives the reported data yourself. The HTTP reporting endpoint must implement the POST method. FIRERPA will report data to the endpoint via POST, and the metadata fields will be encoded as HTTP query parameters, from which you can extract and process them. The data body is carried in the body of the POST request. FIRERPA automatically retries 3 times when receiving status codes 502, 503, or 504. If your backend correctly receives and processes the reported data, it should return plain text OK or SUCCESS and set the status code to 200 to indicate success.

Example Reporting Link¶

http://192.168.1.2/report/${device_id}?serialno=${serialno}

If standard HTTP authentication is required:

http://user:password@192.168.1.2/report/${device_id}?serialno=${serialno}

Reporting to MQTT¶

For reporting to MQTT, the reporting metadata can be extracted from UserProperty. TLS, username/password, and one-way certificate verification (server certificate verification) are supported.

Example Reporting Link¶

mqtt://test.mosquitto.org:1883/script/${device_id}/report

MQTT with password authentication:

mqtt://rw:readwrite@test.mosquitto.org:1884/script/${device_id}/report

Requiring server certificate verification:

mqtts://test.mosquitto.org:8883/script/${device_id}/report?verify=true&ca=LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVBekNDQXV1Z0F3SUJBZ0lVQlkxaGxDR3ZkajROaEJYa1ovdUxVWk5JTEF3d0RRWUpLb1pJaHZjTkFRRUwKQlFBd2daQXhDekFKQmdOVkJBWVRBa2RDTVJjd0ZRWURWUVFJREE1VmJtbDBaV1FnUzJsdVoyUnZiVEVPTUF3RwpBMVVFQnd3RlJHVnlZbmt4RWpBUUJnTlZCQW9NQ1UxdmMzRjFhWFIwYnpFTE1Ba0dBMVVFQ3d3Q1EwRXhGakFVCkJnTlZCQU1NRFcxdmMzRjFhWFIwYnk1dmNtY3hIekFkQmdrcWhraUc5dzBCQ1FFV0VISnZaMlZ5UUdGMFkyaHYKYnk1dmNtY3dIaGNOTWpBd05qQTVNVEV3TmpNNVdoY05NekF3TmpBM01URXdOak01V2pDQmtERUxNQWtHQTFVRQpCaE1DUjBJeEZ6QVZCZ05WQkFnTURsVnVhWFJsWkNCTGFXNW5aRzl0TVE0d0RBWURWUVFIREFWRVpYSmllVEVTCk1CQUdBMVVFQ2d3SlRXOXpjWFZwZEhSdk1Rc3dDUVlEVlFRTERBSkRRVEVXTUJRR0ExVUVBd3dOYlc5emNYVnAKZEhSdkxtOXlaekVmTUIwR0NTcUdTSWIzRFFFSkFSWVFjbTluWlhKQVlYUmphRzl2TG05eVp6Q0NBU0l3RFFZSgpLb1pJaHZjTkFRRUJCUUFEZ2dFUEFEQ0NBUW9DZ2dFQkFNRTBIS21JemZUT3drS0xUM1RISGUrT2JkaXphbVBnClVabUQ2NFRmM3pKZE5lWUdZbjRDRVhieVA2ZnkzdFdjOFMyYm9XNmR6ckg4U2RGZjl1bzMyMEdKQTlCN1UxRlcKVGUzeGRhL0xtM0pGZmFIamtXdzdqQndjYXVRWmpwR0lOSGFwSFJscGlDWnNxdUF0aE9neFc5U2dEZ1lsR3pFQQpzMDZwa0VGaU13K3FEZkxvL3N4RktCNnZRbEZla01lQ3ltakxDYk53UEp5cXloRm1QV3dpby9QRE1ydUJUelBICjNjaW9CbnJKV0tYYzNPalhkTEdGSk9majdwUDBqL2RyMkxINzJlU3Z2M1BRUUZsOTBDWlBGaHJDVWNSSFNTeG8KRTZ5akdPZG56N2Y2UHZlTElCNTc0a1FPUnd0OGVQbjB5aWRyVEMxaWN0aWtFRDNuSFloTVVPVUNBd0VBQWFOVApNRkV3SFFZRFZSME9CQllFRlBWVjZ4QlVGUGlHS0R5bzVWMytIYmg0TjlZU01COEdBMVVkSXdRWU1CYUFGUFZWCjZ4QlVGUGlHS0R5bzVWMytIYmg0TjlZU01BOEdBMVVkRXdFQi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUwKQlFBRGdnRUJBR2E5a1MyMU43MFRoTTYvSGo5RDdtYlZ4S0xCalZXZTJUUHNHZmJsM3JFRGZaK09LUloyajZBQwo2cjdqYjRUWk8zZHpGMnA2ZGdicmxVNzFZLzRLMFRkeklqUmozY1EzS1NtNDFKdlVRMGhaL2MwNGlHRGcveFdmCitwcDU4bmZQQVl3dWVycnVQTldtbFN0V0FYZjBVVHFSdGc0aFFEV0J1VUZESlR1V3V1QnZFWHVkejc0ZWgvd0sKc013ZnUxSEZ2ank1WjBpTURVOFBVRGVwalZvbE9DdWU5YXNobFM0RUI1SUVDZFNSMlRJdG5BSWlJd2lteDgzOQpMZFVkUnVkYWZNdTVUNVhtYTE4Mk9DMC91L3hSbEVtK3R2S0dHbWZGY04wcGlxVmw4T3JTUEJnSWxiKzFJS0pFCm0vWHJpV3IvQ3E0aC9KZkI3TlRzZXpWc2xna0Jhb1U9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K

In the above link, the relevant data will be sent to the script/${device_id}/report topic. You can subscribe to these messages using, for example, mosquitto_sub -L mqtt://test.mosquitto.org:1883/script/+/report.

Reporting to Redis¶

Redis reporting is relatively simple. Because it does not carry any metadata, the data source cannot be directly distinguished. You may need to dynamically modify the injected script to achieve this. For Redis reporting, FIRERPA will directly push the data body into the queue using LPUSH. For example, in the example script above, the reported data will be pushed into the report_data queue.

Example Reporting Link¶

redis://1.2.3.4/0

Redis with password authentication:

redis://:password@1.2.3.4/0

TLS + password authentication Redis:

rediss://:password@1.2.3.4/0?ssl_ca_data=LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURUVENDQWpXZ0F3SUJBZ0lVUFIvcmcxK0x2aU5tYzNsc0...

Reporting to IPC¶

Reporting to IPC is generally not used directly. This method can only be used in Pigeon dynamic scripts, so that your task script can directly listen to and process messages emitted by the emit method when using attach_script, rather than sending them directly to external HTTP or Redis. This gives you a way to process messages from JS scripts in Python to achieve JS-Python interaction.

Example Reporting Link¶

ipc://

Listening for Reported Data¶

Assume the injected application is com.android.settings, simply use ipc://.

app = d.application("com.android.settings")
app.attach_script(script, emit="ipc://")

Within a Pigeon task script, listen and output the script as follows. Please integrate it into your task class yourself.

from lamda.executor import InnerTaskThread

def on_message(name, data):
    print (name, data)

listener = HookScriptIPCListener("data", "com.android.settings", user=0,
                                              threadcls=InnerTaskThread)
listener.set_callback(on_message)
listener.start()

Stopping the Listener¶

listener.stop()

Reporting to IPC+EVENT¶

ipc+event is another form of link. It allows your task script to receive messages while simultaneously reporting the messages to the event system (the FIRERPA device-side status and control protocol; see the status/control protocol section of the Pigeon source code). Generally, you will not use this method directly unless you are modifying Pigeon or directly connecting to the device MQTT control protocol.

This method can automatically report app-related information to the MQTT backend when the device is not executing tasks, such as the interface through which JS hooks receive messages, so that messages received by the app can be sent to the backend for processing in real time.

Example Reporting Link¶

The event parameter is a filter, separated by commas. Only the events contained in it will be sent to the event system, corresponding to emit("nameA", ...) in the JS script.

ipc+event://?event=nameA,nameB,nameC

Listening for Reported Data¶

You need to integrate the MQTT event system yourself. Events will be reported as the script/emit type. Please refer to the Pigeon event system connection and receiving method.

Injecting Reporting Scripts¶

Of course, obtaining the app instance is the first step. You can obtain an app variable through the above call. It represents the application instance you need to inject into. Then you need to use it to perform subsequent injection or detachment operations.

app = d.application("com.android.settings")

Inject the above reporting script into the application. When data is captured, it will be submitted to the Redis report_data queue. In the example, your device must be able to directly access the relevant services on 192.168.1.10, otherwise it will not receive the data.

app.attach_script(script, emit="redis://192.168.1.10/0")

Calling it this way will submit your data to an HTTP endpoint instead of Redis (HTTPS is supported).

app.attach_script(script, emit="http://192.168.1.10/dataReport")

When the data you report is large, enabling compression can significantly improve network transmission throughput. You can use the encode parameter to enable reporting data compression, and you need to decompress the reported data on the receiving end.

app.attach_script(..., encode=DataEncode.DATA_ENCODE_ZLIB)

Decompressing Reported Data¶

By default, reported data is not compressed. If you set reporting compression, you also need to decompress the reported data using zlib on the receiving end. You can conveniently decompress the reported data using the decompress method from Python's official zlib library.

zlib.decompress(data)

Removing Reporting Scripts¶

The process of removing a reporting script is simple and is the same as in persistent scripts.

app.detach_script()