# Frida Data Reporting The data reporting functionality using Frida is based on the persistence feature. You can use your Frida scripts to automatically intercept method call data and report it via our specific method. The data reporting feature allows you to easily send the data intercepted by your script directly to Redis, MQTT, or an external HTTP interface, and you can receive the reported data content through Redis, MQTT, or an HTTP interface. To maximize network efficiency, data compression (zlib) is also supported before reporting. ```{attention} Since version 9.0, the built-in Frida 17.x requires you to bundle frida-java-bridge into your script yourself; otherwise, errors related to Java not defined will occur. This change is from Frida official. According to the official change instructions, you need to create a Node.js project and introduce frida-java-bridge. For details, refer to: https://github.com/oleavr/frida-agent-example , or use the [pack_frida_script.py](https://github.com/firerpa/lamda/blob/HEAD/tools/pack_frida_script.py) we provide to bundle the JS script. ``` ## Writing a Reporting Script Usually, Frida scripts have functions like `send` and `log` to send data to the outside, but for FIRERPA, you need to use a specific method to send data out. The following uses our template code for OkHttp traffic interception as a demonstration. It is just a demo script and may not work properly for you. The script below is not much different from regular scripts, the only difference being the use of an `emit` method, which is a built-in method of FIRERPA. You can use it to conveniently and systematically submit data to the outside. ```js Java.perform(function() { Java.use("com.android.okhttp.internal.http.HttpEngine").getResponse.implementation = function() { var response = this.getResponse() var data = {} data["url"] = response._request.value._url.value._url.value data["body"] = response.body().string() emit("report_data", JSON.stringify(data)) return response } }) ``` The submission method has two parameters: `emit(name, content)`. `name` represents the type of data; if your reporting destination is set to Redis, this name represents the Redis queue name. You should describe it as accurately as possible in English, e.g., `product_info`. `content` represents the data content, and the type supports only strings and byte arrays. In the example, we converted it to a string for submission. Now you understand the format and calling requirements for script writing, i.e., how to submit Hooked data to the outside. You also need to continue reading below to learn how to configure the **data reporting destination**. ## Data Reporting Destination The data reporting destination indicates where the data emitted in your script should be sent. Supported destination types include HTTP interfaces, Redis queues, and MQTT. There are some differences among them. In general, if you don't need to care about information such as data sources, i.e., you don't need to match **data** with **source device**, we recommend using Redis queues. Otherwise, you should use HTTP or MQTT (v5), because these protocols can carry more metadata, allowing you to perform more precise device matching. ### Reporting Metadata For data reported to HTTP interfaces and MQTT destinations, in addition to the original report data, you can also obtain metadata about the device and script at the protocol layer. The metadata carried in the protocol is shown in the table below. | Field | Description | |------------|--------------------------------------------------| | application| App package name (e.g., com.android.settings) | | device | Device ID (e.g., 67b2a3d7-5004-ea2a-0d44-194de6ede8de) | | encode | Data encoding (none | zlib) | | name | Data name (e.g., report_data) | | script | Script ID (e.g., 7c52530d) | | sequence | Reporting sequence (e.g., 30) | | timestamp | Reporting time (e.g., 1740023596914) | | user | Multi-app user ID (e.g., 0) | ```{attention} Due to the nature of the Redis protocol, data reported to a Redis destination does not include any of the above metadata. ``` `device` is the unique device ID; you can find it in the remote desktop information bar. Usually, this ID is unique and fixed. You can use it to mark the device and establish a correspondence. `encode` is the data encoding; supported encodings are `none` and `zlib`. If the encoding is zlib, you need to decompress the data body using zlib. `name` marks the type of this reported data; it is also the first parameter when you call the `emit` method. `sequence` represents the index of the reported data, starting from 0 and incrementing with each report. You can use this field to sort the data or check for report loss. ### Additional Parameters in the Reporting Link You can specify some dynamic id parameters in the reporting link using the variable placeholder format `${name}` inserted into specific parts of the link. For example, `http://192.168.1.2/report/${device_id}`. The supported variables are shown in the table below. | Name | Description | | ------------- | ----------- | | device_id | Unique device ID | | device_id_short | Unique device ID (short device ID encoded in BASE62) | | android_id | Android ID | | serialno | ro.serialno | ### Reporting to HTTP You need to write an HTTP service to receive the reported data yourself. The HTTP reporting interface must implement the POST method. FIRERPA will report data to the interface via POST, and will encode each field of the metadata as HTTP query parameters, which you can extract and process. The data body is carried in the POST request body. FIRERPA will automatically retry up to 3 times upon receiving 502, 503, or 504 status codes. If your backend correctly receives and processes the reported data, it should return plain text `OK` or `SUCCESS` with a `200` status code to indicate successful processing. ```{attention} HTTP requests are multi-threaded; the messages received by the backend may not follow the reporting sequence (sequence). ``` #### Example Reporting Link ```txt http://192.168.1.2/report/${device_id}?serialno=${serialno} ``` If standard HTTP protocol authentication is required: ```txt http://user:password@192.168.1.2/report/${device_id}?serialno=${serialno} ``` ### Reporting to MQTT When reporting to MQTT, the metadata can be extracted from `UserProperty`. It supports TLS, username/password, and one-way certificate verification (server certificate verification). #### Example Reporting Link ```txt mqtt://test.mosquitto.org:1883/script/${device_id}/report ``` MQTT with password authentication: ```txt mqtt://rw:readwrite@test.mosquitto.org:1884/script/${device_id}/report ``` Requiring server certificate authentication: ```txt mqtts://test.mosquitto.org:8883/script/${device_id}/report?verify=true&ca=LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVBekNDQXV1Z0F3SUJBZ0lVQlkxaGxDR3ZkajROaEJYa1ovdUxVWk5JTEF3d0RRWUpLb1pJaHZjTkFRRUwKQlFBd2daQXhDekFKQmdOVkJBWVRBa2RDTVJjd0ZRWURWUVFJREE1VmJtbDBaV1FnUzJsdVoyUnZiVEVPTUF3RwpBMVVFQnd3RlJHVnlZbmt4RWpBUUJnTlZCQW9NQ1UxdmMzRjFhWFIwYnpFTE1Ba0dBMVVFQ3d3Q1EwRXhGakFVCkJnTlZCQU1NRFcxdmMzRjFhWFIwYnk1dmNtY3hIekFkQmdrcWhraUc5dzBCQ1FFV0VISnZaMlZ5UUdGMFkyaHYKYnk1dmNtY3dIaGNOTWpBd05qQTVNVEV3TmpNNVdoY05NekF3TmpBM01URXdOak01V2pDQmtERUxNQWtHQTFVRQpCaE1DUjBJeEZ6QVZCZ05WQkFnTURsVnVhWFJsWkNCTGFXNW5aRzl0TVE0d0RBWURWUVFIREFWRVpYSmllVEVTCk1CQUdBMVVFQ2d3SlRXOXpjWFZwZEhSdk1Rc3dDUVlEVlFRTERBSkRRVEVXTUJRR0ExVUVBd3dOYlc5emNYVnAKZEhSdkxtOXlaekVmTUIwR0NTcUdTSWIzRFFFSkFSWVFjbTluWlhKQVlYUmphRzl2TG05eVp6Q0NBU0l3RFFZSgpLb1pJaHZjTkFRRUJCUUFEZ2dFUEFEQ0NBUW9DZ2dFQkFNRTBIS21JemZUT3drS0xUM1RISGUrT2JkaXphbVBnClVabUQ2NFRmM3pKZE5lWUdZbjRDRVhieVA2ZnkzdFdjOFMyYm9XNmR6ckg4U2RGZjl1bzMyMEdKQTlCN1UxRlcKVGUzeGRhL0xtM0pGZmFIamtXdzdqQndjYXVRWmpwR0lOSGFwSFJscGlDWnNxdUF0aE9neFc5U2dEZ1lsR3pFQQpzMDZwa0VGaU13K3FEZkxvL3N4RktCNnZRbEZla01lQ3ltakxDYk53UEp5cXloRm1QV3dpby9QRE1ydUJUelBICjNjaW9CbnJKV0tYYzNPalhkTEdGSk9majdwUDBqL2RyMkxINzJlU3Z2M1BRUUZsOTBDWlBGaHJDVWNSSFNTeG8KRTZ5akdPZG56N2Y2UHZlTElCNTc0a1FPUnd0OGVQbjB5aWRyVEMxaWN0aWtFRDNuSFloTVVPVUNBd0VBQWFOVApNRkV3SFFZRFZSME9CQllFRlBWVjZ4QlVGUGlHS0R5bzVWMytIYmg0TjlZU01COEdBMVVkSXdRWU1CYUFGUFZWCjZ4QlVGUGlHS0R5bzVWMytIYmg0TjlZU01BOEdBMVVkRXdFQi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUwKQlFBRGdnRUJBR2E5a1MyMU43MFRoTTYvSGo5RDdtYlZ4S0xCalZXZTJUUHNHZmJsM3JFRGZaK09LUloyajZBQwo2cjdqYjRUWk8zZHpGMnA2ZGdicmxVNzFZLzRLMFRkeklqUmozY1EzS1NtNDFKdlVRMGhaL2MwNGlHRGcveFdmCitwcDU4bmZQQVl3dWVycnVQTldtbFN0V0FYZjBVVHFSdGc0aFFEV0J1VUZESlR1V3V1QnZFWHVkejc0ZWgvd0sKc013ZnUxSEZ2ank1WjBpTURVOFBVRGVwalZvbE9DdWU5YXNobFM0RUI1SUVDZFNSMlRJdG5BSWlJd2lteDgzOQpMZFVkUnVkYWZNdTVUNVhtYTE4Mk9DMC91L3hSbEVtK3R2S0dHbWZGY04wcGlxVmw4T3JTUEJnSWxiKzFJS0pFCm0vWHJpV3IvQ3E0aC9KZkI3TlRzZXpWc2xna0Jhb1U9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K ``` In the above link, the relevant data will be sent to the topic `script/${device_id}/report`. You can subscribe to these messages using something like `mosquitto_sub -L mqtt://test.mosquitto.org:1883/script/+/report`. ### Reporting to Redis Redis reporting is relatively simple. Because it does not carry any metadata, you cannot directly distinguish the data source. You may need to achieve this by dynamically modifying the injected script. For Redis reporting, FIRERPA will directly push the data body into the queue via `LPUSH`. For example, in the example script above, the reported data will be pushed into the `report_data` queue. ```{attention} Only standalone Redis service is supported; Redis cluster is not supported. Except for the password field, do not add variable placeholders in the Redis link. The link is in the format supported by the standard Redis library; modifying other parts may cause parsing errors. ``` #### Example Reporting Link ```txt redis://1.2.3.4/0 ``` Redis requiring password authentication: ```txt redis://:password@1.2.3.4/0 ``` TLS + password authentication Redis: ```txt rediss://:password@1.2.3.4/0?ssl_ca_data=LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURUVENDQWpXZ0F3SUJBZ0lVUFIvcmcxK0x2aU5tYzNsc0... ``` ## Injecting the Reporting Script Of course, obtaining the app instance is the first step. You can obtain an `app` variable through the above calls, which represents the application instance you need to inject. You then need to use it to perform subsequent injection or detachment operations. ```python app = d.application("com.android.settings") ``` Inject the above reporting script into the application. When data is intercepted, it will be submitted to the Redis `report_data` queue. In the example, your device needs direct access to the relevant service on 192.168.1.10; otherwise, it will not receive the data. ```python app.attach_script(script, emit="redis://192.168.1.10/0") ``` Calling it this way will submit your data to an HTTP interface instead of Redis (HTTPS supported). ```python app.attach_script(script, emit="http://192.168.1.10/dataReport") ``` When the data you report is large, enabling compression can significantly improve network transmission throughput. You can use the `encode` parameter to enable report data compression and need to decompress the data on the receiving end. ```python app.attach_script(..., encode=DataEncode.DATA_ENCODE_ZLIB) ``` ## Decompressing Reported Data By default, reported data is not compressed. If you have enabled reporting compression, you also need to decompress the reported data on the receiving end using zlib. You can conveniently decompress the reported data using the standard Python library `zlib`'s `decompress` method. ```python zlib.decompress(data) ``` ## Removing the Reporting Script Removing the reporting script is straightforward, just like the usage in the persistence script. ```python app.detach_script() ```