BLE Log Compression Scheme (Preview)
1、Overview
This scheme scans the code of BLE stack-related components during the compilation phase, converting formatted strings and parameters in log statements into pure binary data. This improves log output efficiency and reduces the Flash footprint of the protocol stack.
Currently, the scheme supports log compression for both BLE-MESH and BLE-HOST-BLUEDROID components.
2、How to Use
This feature requires additional Python libraries. Please follow the steps below to set up the environment.
Step 1: Verify ESP-IDF Virtual Environment
Ensure all subsequent steps are performed within the ESP-IDF Python virtual environment.
Verify activation by running:
idf.py --version
If the output shows idf.py: command not found, the virtual environment is not active.
Refer to the official documentation to configure and activate the environment: ESP-IDF Setup Guide
After activation, run idf.py --version again. A version number confirms successful setup.
Step 2: Clean Build Cache
It is recommended to delete the existing build folder (if any) and rebuild the application to ensure a clean environment.
Step 3: Configure via Menuconfig
Run idf.py menuconfig and navigate to the following path to enable BLE-MESH log compression:
(Top) → Component config → Bluetooth → Common Options → BLE Log → Enable BLE Log Module (Experimental) → Settings of BLE Log Compression->Enable BLE Mesh log compression(Preview).
There are three configuration items under this submenu:
-
BLE Mesh log buffer length: Sets the maximum length of a single log entry.
-
Select the stack log tag to be compressed: Select the protocol stack log expect to compress.
-
Select the net buf log tag to be compressed: Select the log of the net_buf part of the protocol stack that expect to compress.
For example, In the BLE-Mesh component, logs are classified into four levels: BT_ERR, BT_WARN, BT_INFO, and BT_DBG.
Enabling Compress ERROR logs of ESP-BLE-MESH causes BT_ERR logs to be transmitted via the compression path.
If Keep the original error log statement is also checked, BT_ERR logs will be emitted through both the compressed channel and the legacy UART channel.
This dual-path approach increases binary size and extends the total log-output time (compression transmission latency + UART latency).
Default policy:
- ERROR & WARN → compressed + UART (dual path)
- INFO & DEBUG → compressed only (UART disabled)
Consequently, under the default configuration, even when BLE-Mesh INFO-level logging is turned on, no INFO messages appear on the terminal—they are redirected to the compressed interface and no longer pass through the serial port.
Step 4: Build the Application
After configuration, build the application with:
idf.py build
Watch for any warnings during the build process. For example:
CMake Warning at esp/esp-idf/components/bt/common/ble_log/log_compression/CMakeLists.txt:46 (message):
tree_sitter import failed, please check whether the package is installed
correctly,Please refer to the
file: esp/esp-idf/components/bt/common/ble_log/log_compression/README
for installation instructions.
This indicates that the dependencies were not installed correctly, and log compression has failed—falling back to a normal build. Please repeat Steps 1–4.
If log compression is successful, you will see output similar to:
[0/1285] Log compression is being performed, please wait...
Log compression underway, please wait...
Found module BLE_MESH for compression
Found 111 source files in module BLE_MESH requiring compression
3055 ble log(s) compressed
Header file for compressed logs generated
After a successful build, the following structure will be generated under build/ble_log/:
build/ble_log/
├── ble_log_database
│ └── BLE_MESH_logs.json
├── ble_script_log_{timestamp}.log
├── .compressed_srcs
│ └── esp_ble_mesh
├── include
│ └── mesh_log_index.h
└── module_info.yml
.compressed_srcs: Compressed C source files.mesh_log_index.h: Generated header file containing log macros.BLE_MESH_logs.json: Detailed information for each log entry.ble_script_log_{timestamp}.log: Log generated by the compression script.module_info.yml: Configuration file for compressed logging across modules.
Do not modify these auto-generated files.
Step 5: Receive Logs
With log compression enabled and under the default configuration, all log levels except ERR and WARN generated by the compressed component will be redirected to the compression-log interface for output. Please refer to the BLE Log module’s documentation for how to receive these logs: BLE Log module.
Step 6: Decode compressed Logs
The purpose of the log-compression scheme is to help users identify protocol-stack issues more quickly. As the parsing script has not yet been released, please submit the generated log file (.bin) along with the current IDF commit hash to the Espressif BLE team, who will handle the analysis.
Frequently Asked Questions
- If encoded logs cause compilation errors or missing macro definitions, delete the build folder and rebuild. If the issue persists, please report it to the Espressif BLE team.